www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DIP16: Transparently substitute module with package

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Starting a new thread from one in announce:

http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

Please comment, after which Walter will approve. Walter's approval means 
that he would approve a pull request implementing DIP16 (subject to 
regular correctness checks).


Destroy!

Andrei
Mar 30 2012
next sibling parent deadalnix <deadalnix gmail.com> writes:
Le 30/03/2012 16:46, Andrei Alexandrescu a écrit :
 Starting a new thread from one in announce:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

 Please comment, after which Walter will approve. Walter's approval means
 that he would approve a pull request implementing DIP16 (subject to
 regular correctness checks).


 Destroy!

 Andrei

I was waiting for it :D First, as all.d is already a convention in many D projects, why choose package.d ? I understand this is a keyword, but this will also be painfull for many developer. Second, what the rule of a .d file and a folder existing, but in different path (think -I switch of the compiler). In the first place, is the module declaration really usefull ? About the lookup rule, I understand that if I import a.b and use the function a.b.c.foo , then i must be able to refers to it as a.b.foo, but why a.foo ? It seems to me like going too far in the modification for no benefit (it can only increase the number of collision, and has no benefit I can think of).
Mar 30 2012
prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 03/30/2012 04:46 PM, Andrei Alexandrescu wrote:
 Starting a new thread from one in announce:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

 Please comment, after which Walter will approve. Walter's approval means
 that he would approve a pull request implementing DIP16 (subject to
 regular correctness checks).


 Destroy!

 Andrei

Those are two proposals. I think the first one should be implemented. I don't really like the second one. 1. It is an over-general solution, because it does not solve a general problem. 2. Library writers lose control. All symbol names in the package are reserved when the package is deployed. It will be a breaking change to introduce a sub-package that happens to have the same name as some symbol defined in any module in the same package hierarchy. 3. symbol lookup is already hard enough to get right, because compile time reflection and conditional code generation can introduce ambiguities and contradictions. DMD does not get it right. It is likely that this change would make fixing this in a general but not-too-conservative way considerably harder. Maybe it would be better to just interpret foo.bar.baz as foo.bar.package.baz if foo.bar is a package that has been imported via the foo.bar.package rewrite? Of course, issue 2 probably would remain.
Mar 30 2012
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On second thought, issue 2 is probably not that much of a problem.
Mar 30 2012
prev sibling parent "Nick Sabalausky" <a a.a> writes:
"Timon Gehr" <timon.gehr gmx.ch> wrote in message 
news:jl4jmg$2j1r$1 digitalmars.com...
 I don't really like the second one.

 1. It is an over-general solution, because it does not solve a general 
 problem.

 Maybe it would be better to just interpret foo.bar.baz as 
 foo.bar.package.baz if foo.bar is a package that has been imported via the 
 foo.bar.package rewrite?

That occurred to me, and I thought about proposing the same thing you're suggesting, but on second thought I wasn't so sure: If I need to disambiguate between "std.algorithm.find" and "foo.bar.baz.find", it might be nice to be able to just say "Meh, just...that one in Phobos, ie 'std'". Or "Just go with that 'foo' one". I could go either way, really.
Mar 30 2012
prev sibling next sibling parent reply Robert Clipsham <robert octarineparrot.com> writes:
On 30/03/2012 15:46, Andrei Alexandrescu wrote:
 Starting a new thread from one in announce:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

 Please comment, after which Walter will approve. Walter's approval means
 that he would approve a pull request implementing DIP16 (subject to
 regular correctness checks).


 Destroy!

 Andrei

The proposal doesn't say what happens when package.d is not found but foo/bar/ exists. Given that a lot of people will just use public import foo.bar.*; in that file, would it make sense for package.d missing to imply import foo.bar.*? That would save typing out every single file in there. Of course it would also be annoying if you wanted to import everything except one file, as you'd then have to type out every single import anyway. The other option is to error, which is probably a more sane option. -- Robert http://octarineparrot.com/
Mar 30 2012
parent "Nick Sabalausky" <a a.a> writes:
"Robert Clipsham" <robert octarineparrot.com> wrote in message 
news:jl4l5t$2m62$1 digitalmars.com...
 On 30/03/2012 15:46, Andrei Alexandrescu wrote:
 Starting a new thread from one in announce:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

 Please comment, after which Walter will approve. Walter's approval means
 that he would approve a pull request implementing DIP16 (subject to
 regular correctness checks).


 Destroy!

 Andrei

The proposal doesn't say what happens when package.d is not found but foo/bar/ exists. Given that a lot of people will just use public import foo.bar.*; in that file, would it make sense for package.d missing to imply import foo.bar.*? That would save typing out every single file in there. Of course it would also be annoying if you wanted to import everything except one file, as you'd then have to type out every single import anyway.

That would effectively be the same as Java's "import foo.*" and a lot of people have issues with that.
 The other option is to error, which is probably a more sane option.

That's what I'd suggest doing. Just treat it like importing any other missing package.
Mar 30 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 30 Mar 2012 10:46:19 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 Starting a new thread from one in announce:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

 Please comment, after which Walter will approve. Walter's approval means  
 that he would approve a pull request implementing DIP16 (subject to  
 regular correctness checks).

I think package.d should be allowed to specify module. Otherwise, something like /usr/include/d/std/datetime/package.d, what is the assumed package? If module is not useful here, it is not useful anywhere. I think the omission should be strictly anything after the package directory. For example, if you have std/algorithm/package.d and std/algorithm/sorting.d, where package.d imports sorting.d, you can refer to std.algorithm.sorting.sort by omitting sorting, but not by omitting algorithm or std. Other than that, this is a good change. -Steve
Mar 30 2012
prev sibling next sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 3/30/12, Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> wrote:
 Destroy!

"That means a program that imports std.algorithm may use "std.sort" for the symbol "std.algorithm.sort"." That's quite interesting. Would that also mean that you could do: import std.algorithm; // has indexOf import std.string; // has indexOf void main() { string.indexOf("foo", "foo"); -> std.string.indexOf } That would ease usage of Phobos a little bit. OTOH 'string' already is a keyword and things might get hairy.. Still this is one of the few proposals I like. My only caveat is the comment: "except the file is not allowed to use the "module" declaration.". Wouldn't it be better if we explicitly declared a module as a package instead? In foo\bar\package.d: package foo.bar; Since the "module" declaration must be on the first line (or second line after shebang), you could special-case DMD to allow the package keyword to be used here. I know D likes to abuse a keyword for multiple things (hello Mr. Static!), but I think we could live with it.
Mar 30 2012
parent "Nick Sabalausky" <a a.a> writes:
"Andrej Mitrovic" <andrej.mitrovich gmail.com> wrote in message 
news:mailman.1240.1333130858.4860.digitalmars-d puremagic.com...
 Still this is one of the few proposals I like. My only caveat is the
 comment: "except the file is not allowed to use the "module"
 declaration.". Wouldn't it be better if we explicitly declared a
 module as a package instead? In foo\bar\package.d:
 package foo.bar;

 Since the "module" declaration must be on the first line (or second
 line after shebang), you could special-case DMD to allow the package
 keyword to be used here. I know D likes to abuse a keyword for
 multiple things (hello Mr. Static!), but I think we could live with
 it.

Or maybe just require the module name ends with ".package"
Mar 30 2012
prev sibling next sibling parent Brad Anderson <eco gnuk.net> writes:
--e89a8f23531141977004bc79d403
Content-Type: text/plain; charset=ISO-8859-1

On Fri, Mar 30, 2012 at 12:06 PM, Andrej Mitrovic <
andrej.mitrovich gmail.com> wrote:

 On 3/30/12, Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> wrote:
 Destroy!

"That means a program that imports std.algorithm may use "std.sort" for the symbol "std.algorithm.sort"." That's quite interesting. Would that also mean that you could do: import std.algorithm; // has indexOf import std.string; // has indexOf void main() { string.indexOf("foo", "foo"); -> std.string.indexOf }

seems so natural to resolve ambiguity using as little context as necessary.
 That would ease usage of Phobos a little bit. OTOH 'string' already is
 a keyword and things might get hairy..

 Still this is one of the few proposals I like. My only caveat is the
 comment: "except the file is not allowed to use the "module"
 declaration.". Wouldn't it be better if we explicitly declared a
 module as a package instead? In foo\bar\package.d:
 package foo.bar;

 Since the "module" declaration must be on the first line (or second
 line after shebang), you could special-case DMD to allow the package
 keyword to be used here. I know D likes to abuse a keyword for
 multiple things (hello Mr. Static!), but I think we could live with
 it.

--e89a8f23531141977004bc79d403 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On Fri, Mar 30, 2012 at 12:06 PM, Andrej Mitrovic <span dir=3D"ltr">&lt;<a = href=3D"mailto:andrej.mitrovich gmail.com">andrej.mitrovich gmail.com</a>&g= t;</span> wrote:<br><div class=3D"gmail_quote"><blockquote class=3D"gmail_q= uote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1e= x"> On 3/30/12, Andrei Alexandrescu &lt;<a href=3D"mailto:SeeWebsiteForEmail er= dani.org">SeeWebsiteForEmail erdani.org</a>&gt; wrote:<br> &gt; Destroy!<br> <br> &quot;That means a program that imports std.algorithm may use &quot;std.sor= t&quot;<br> for the symbol &quot;std.algorithm.sort&quot;.&quot;<br> <br> That&#39;s quite interesting. Would that also mean that you could do:<br> import std.algorithm; =A0// has indexOf<br> import std.string; =A0// has indexOf<br> void main() {<br> =A0 =A0string.indexOf(&quot;foo&quot;, &quot;foo&quot;); -&gt; std.string.= indexOf<br> }<br> <br></blockquote><div><br></div><div>I was actually kind of surprised when = I found out this doesn&#39;t work. =A0It seems so natural to resolve ambigu= ity using as little context as necessary.</div><div>=A0</div><blockquote cl= ass=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;p= adding-left:1ex"> That would ease usage of Phobos a little bit. OTOH &#39;string&#39; already= is<br> a keyword and things might get hairy..<br> <br> Still this is one of the few proposals I like. My only caveat is the<br> comment: &quot;except the file is not allowed to use the &quot;module&quot;= <br> declaration.&quot;. Wouldn&#39;t it be better if we explicitly declared a<b= r> module as a package instead? In foo\bar\package.d:<br> package foo.bar;<br> <br> Since the &quot;module&quot; declaration must be on the first line (or seco= nd<br> line after shebang), you could special-case DMD to allow the package<br> keyword to be used here. I know D likes to abuse a keyword for<br> multiple things (hello Mr. Static!), but I think we could live with<br> it.<br> </blockquote></div><br> --e89a8f23531141977004bc79d403--
Mar 30 2012
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 3/30/12, Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> wrote:
 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

Btw, I bet with the help of hackers like e.g. Kenji Hara we'll have this implemented in a matter of days (if it gets accepted). Compare that to having a C++ committee that would spend the next 5 years writing a 100-page spec on what is or isn't a package, heheh. Didn't Bjarne say on Going Native that he doesn't really know what a module is or should be? He should try D sometime. ;)
Mar 30 2012
prev sibling next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, March 30, 2012 09:46:19 Andrei Alexandrescu wrote:
 Starting a new thread from one in announce:
 
 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16
 
 Please comment, after which Walter will approve. Walter's approval means
 that he would approve a pull request implementing DIP16 (subject to
 regular correctness checks).
 
 
 Destroy!

The first part with package.d seems like a good idea and certainly affects some of what I've been doing with std.datetime. In order to split it right now, you need a new package name, leaving the original for importing everything, and this provides a better way of dealing with that. However, I'm very nervous about the second part. e.g. std.sort instead of std.algorithm.sort seems like a bad idea to me. It increases the odds of name conflicts for little benefit. Not to mention, it'll make it a lot more confusing to find what modules stuff is actually in if people start doing stuff like std.sort(arr); In the case of sort, you may know where it's from - particularly since it's so common - but the less well-known the function is, the less likely that is at all obvious where it comes from, and if you're dealing with 3rd party software, then it wouldn't be at all obvious. For instance, how would you know that party.foo is really party.bar.foo? You wouldn't. Being so lax about importing could really harm code readibility (and maintainibility, since it increases the odds of name clashes). So, I'm inclined to say that that is a _bad_ idea. I'd propose that we make it so that if a module publicly imports another module, then you could treat it as if it were in that module. So, because std.datetime.package publicly imports std.datetime.systime, you could use std.datetime.SysTime instead of std.datetime.systime.SysTime. The compiler would need to realize that they're exactly the same symbol (we've had bugs relating to importing with : and the like which ended up creating new symbols, and we don't want that here), but that shouldn't be all that hard. That gives you control over which symbols are able to be treated as if they were in a given package rather than affecting everything indiscriminitely (and if you use : with public imports, it should then be possible, to restrict it to the _exact_ set of symbols that you want if you don't want all of the symbols to be treated as if they were in that module). Another question is how this affects the documentation. Does package.d generate a page just like the other modules do? The lack of a module declaration could make that difficult (not impossible, but it would probably require changes to ddoc in addition to the module stuff). Also, does that page get treated in any special manner in how the documentation is laid out, because it's for the package as a whole (probably more of a site question than a ddoc one though)? I'd like to be able to have a page describing the package as a whole for std.datetime in addition to having the individual pages rather than just splitting it up, and leaving the programmer to read each of the individual pages with no overview. And I think that however package.d works, it needs to enable that. - Jonathan M Davis
Mar 30 2012
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 3/30/12 1:39 PM, Jonathan M Davis wrote:
 However, I'm very nervous about the second part. e.g. std.sort instead of
 std.algorithm.sort seems like a bad idea to me. It increases the odds of name
 conflicts for little benefit.

Example?
 Not to mention, it'll make it a lot more confusing
 to find what modules stuff is actually in if people start doing stuff like

 std.sort(arr);

 In the case of sort, you may know where it's from - particularly since it's so
 common - but the less well-known the function is, the less likely that is at
 all obvious where it comes from, and if you're dealing with 3rd party
 software, then it wouldn't be at all obvious. For instance, how would you know
 that party.foo is really party.bar.foo? You wouldn't.

Why should you?
 Being so lax about
 importing could really harm code readibility (and maintainibility, since it
 increases the odds of name clashes). So, I'm inclined to say that that is a
 _bad_ idea.

Maybe if you produce a solid example, I'd be convinced. Andrei
Mar 30 2012
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 03/30/2012 11:35 PM, Jonathan M Davis wrote:
 But personally, I like the idea of making it so that publicly imported symbols
 can be accessed as if they were in the module that publicly imported them

+1. That is even better than treating the package module specially.
Mar 31 2012
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 03/31/2012 03:49 PM, Martin Nowak wrote:
 On Sat, 31 Mar 2012 13:06:36 +0200, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 03/30/2012 11:35 PM, Jonathan M Davis wrote:
 But personally, I like the idea of making it so that publicly
 imported symbols
 can be accessed as if they were in the module that publicly imported
 them

+1. That is even better than treating the package module specially.

That already works, doesn't it?

It already works indeed. My bad.
Mar 31 2012
prev sibling next sibling parent deadalnix <deadalnix gmail.com> writes:
Le 30/03/2012 21:33, Andrei Alexandrescu a écrit :
 On 3/30/12 1:39 PM, Jonathan M Davis wrote:
 However, I'm very nervous about the second part. e.g. std.sort instead of
 std.algorithm.sort seems like a bad idea to me. It increases the odds
 of name
 conflicts for little benefit.

Example?
 Not to mention, it'll make it a lot more confusing
 to find what modules stuff is actually in if people start doing stuff
 like

 std.sort(arr);

 In the case of sort, you may know where it's from - particularly since
 it's so
 common - but the less well-known the function is, the less likely that
 is at
 all obvious where it comes from, and if you're dealing with 3rd party
 software, then it wouldn't be at all obvious. For instance, how would
 you know
 that party.foo is really party.bar.foo? You wouldn't.

Why should you?
 Being so lax about
 importing could really harm code readibility (and maintainibility,
 since it
 increases the odds of name clashes). So, I'm inclined to say that that
 is a
 _bad_ idea.

Maybe if you produce a solid example, I'd be convinced. Andrei

You are reversing the logic. It have to be shown that it is not a problem to do the change. The other way around is flawed logic.
Mar 31 2012
prev sibling parent deadalnix <deadalnix gmail.com> writes:
Le 30/03/2012 23:35, Jonathan M Davis a écrit :
 On Friday, March 30, 2012 14:33:58 Andrei Alexandrescu wrote:
 On 3/30/12 1:39 PM, Jonathan M Davis wrote:
 However, I'm very nervous about the second part. e.g. std.sort instead of
 std.algorithm.sort seems like a bad idea to me. It increases the odds of
 name conflicts for little benefit.

Example?

std.sort works because there's only one sort. If there are two, you get a conflict (e.g. if you had std.path.sort which sorted paths in some path-specific manner). If std.path.sort existed now, then std.sort wouldn't work, and you'd be forced to specify std.algorithm.sort or std.path.sort, and that's fine. It would be similar to having to specify std.algorithm.indexOf when you've imported both std.string and std.algorithm. But the problem is when std.path.sort is added _later_.

I mentionned that as being a ptotential issue, so this is a +1 .
 But personally, I like the idea of making it so that publicly imported symbols
 can be accessed as if they were in the module that publicly imported them
 (with package.d being treated as if it had the same name as the package that
 it's in). That's essentially how it already works except when specifying the
 full import path for a symbol. And that way, you can specify in
 std.algorithm.package.d exactly what you want to be imported when
 std.algorithm is imported (including using : to restrict it to specific symbols
 in a module), and only those symbols will be treated as if they were part of
 std.algorithm - both for importing purposes and when specifying the import
 path when using a symbol. The library maintainer then has control over which
 symbols get used with which import paths.

 - Jonathan M Davis

I did propose that in another thread, so this is a +1 too.
Mar 31 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, March 30, 2012 20:06:57 Andrej Mitrovic wrote:
 On 3/30/12, Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> wrote:
 Destroy!

"That means a program that imports std.algorithm may use "std.sort" for the symbol "std.algorithm.sort"." That's quite interesting. Would that also mean that you could do: import std.algorithm; // has indexOf import std.string; // has indexOf void main() { string.indexOf("foo", "foo"); -> std.string.indexOf }

No, I don't think so. If I understand the proposal correctly, it would enable std.indexOf (which doesn't help you at all in this case), not string.indexOf. It's trying to make it so that you can treat a symbol in a sub-module as it were in a higher module, and string.indexOf doesn't help with that at all. - Jonathan M Davis
Mar 30 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, March 30, 2012 12:15:44 Brad Anderson wrote:
 On Fri, Mar 30, 2012 at 12:06 PM, Andrej Mitrovic <
 
 andrej.mitrovich gmail.com> wrote:
 On 3/30/12, Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> wrote:
 Destroy!

"That means a program that imports std.algorithm may use "std.sort" for the symbol "std.algorithm.sort"." That's quite interesting. Would that also mean that you could do: import std.algorithm; // has indexOf import std.string; // has indexOf void main() { string.indexOf("foo", "foo"); -> std.string.indexOf }

I was actually kind of surprised when I found out this doesn't work. It seems so natural to resolve ambiguity using as little context as necessary.

It would certainly be desirable in some cases, but I believe that the reason that it doesn't work is due to the ambiguities that it would create. I'd have to go dig up old discussions on it though to remember all of the details. alias is supposed to solve the problem, but it doesn't really work all that well for it, since private doesn't hide symbols, it only makes them inaccessible (just like with C++). So, creating aliases in a module causes problems in other modules that import that module, even if the aliases are private. There are definitely some folks pushing for private to actually start hiding symbols (IIRC, there's even a pull request for it), but I don't know what the odds of convincing Walter are. If/Once that happens, alias will actually become usable for this sort of situation, and the inability to do string.indexOf won't be as big a deal. - Jonathan M Davis
Mar 30 2012
prev sibling next sibling parent "Rene Zwanenburg" <renezwanenburg gmail.com> writes:
On Friday, 30 March 2012 at 18:39:44 UTC, Jonathan M Davis wrote:
 I'd propose that we make it so that if a module publicly 
 imports another
 module, then you could treat it as if it were in that module. 
 So, because
 std.datetime.package publicly imports std.datetime.systime, you 
 could use
 std.datetime.SysTime instead of std.datetime.systime.SysTime.

I'm not sure if that's a good idea. I'd prefer a new kind of import statement, perhaps something like: // module std.datetime.package alias import std.datetime.systime; which is similar to a public alias of everything in that module?
Mar 30 2012
prev sibling next sibling parent "Nick Sabalausky" <a a.a> writes:
My comments:

1. My first impression was that using "foo/bar/package.d" instead of 
"foo/bar.d" seemed a bit odd and messy. But I realize now that cleverly 
solves the issue where "foo/bar.d" would be considered to be inside a 
different package from "foo/bar/*.d". So I like that. Personally, I think I 
would have gone with "foo/bar/_.d" as that sorts much, much better, but 
naming debates can go on forever, and I can live with "package.d"

2. I don't understand any of this:

-------------------------------------------
When looking up the symbol "foo.bar.baz", currently an exact match is 
needed. However. when looking up ".baz" or simply "baz", a flexible lookup 
is used that has many advantages (less verbose, hijacking detection etc). 
Therefore we think similar flexibility should be imparted to "foo.bar.baz", 
as follows:

If a qualified symbol "foo.bar.baz" appears in code, the compiler considers 
"foo.bar" a prefix that sets the starting point of the lookup, and then 
proceeds with looking up "baz" from that starting point. That means a 
program that imports std.algorithm may use "std.sort" for the symbol 
"std.algorithm.sort".
-------------------------------------------

I *do* understand "a program that imports std.algorithm may use "std.sort" 
for the symbol 'std.algorithm.sort'", and I think that's a good idea. It 
solves a problem I hadn't even thought of. But I don't understand that stuff 
I quoted above. Perhaps you could reword/clarify?

3. Other than that stuff, I'm very much in favor of this. I'll have some of 
that!
Mar 30 2012
prev sibling next sibling parent "F i L" <witte2008 gmail.com> writes:
On Friday, 30 March 2012 at 18:15:57 UTC, Brad Anderson wrote:
 On Fri, Mar 30, 2012 at 12:06 PM, Andrej Mitrovic <
 andrej.mitrovich gmail.com> wrote:

 On 3/30/12, Andrei Alexandrescu 
 <SeeWebsiteForEmail erdani.org> wrote:
 Destroy!

"That means a program that imports std.algorithm may use "std.sort" for the symbol "std.algorithm.sort"." That's quite interesting. Would that also mean that you could do: import std.algorithm; // has indexOf import std.string; // has indexOf void main() { string.indexOf("foo", "foo"); -> std.string.indexOf }

work. It seems so natural to resolve ambiguity using as little context as necessary.

Ya that was the behavior I expected as well. Would be great if it worked like that. Just back trace the reference until the ambiguity is resolved. // ----- Also, I'm probably missing something here, but I never understood why importing a package doesn't work like it does in Actionscript/Java/others... import foo.bar.*; // everything import foo.bar.all; // custom That makes a lot of sense to me.
Mar 30 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, March 30, 2012 14:33:58 Andrei Alexandrescu wrote:
 On 3/30/12 1:39 PM, Jonathan M Davis wrote:
 However, I'm very nervous about the second part. e.g. std.sort instead of
 std.algorithm.sort seems like a bad idea to me. It increases the odds of
 name conflicts for little benefit.

Example?

std.sort works because there's only one sort. If there are two, you get a conflict (e.g. if you had std.path.sort which sorted paths in some path-specific manner). If std.path.sort existed now, then std.sort wouldn't work, and you'd be forced to specify std.algorithm.sort or std.path.sort, and that's fine. It would be similar to having to specify std.algorithm.indexOf when you've imported both std.string and std.algorithm. But the problem is when std.path.sort is added _later_. All of a sudden, code which used std.sort and worked is now broken. The problem does currently exist in that if we added indexOf to another module - say std.array - then code which imported either std.string or std.algorithm as well as std.array would break with the addition of std.array.indexOf, but your proposal makes it worse. Not only does it provide another way in which adding a function could result in conflicts when existing code is recompiled, but it makes it so that if you add any function anywhere in the _entire standard library_ which has the name as an existing one, you get a conflict (if anyone uses std.x rather than x or the full import path). D does a good job of providing ways to fix name conflicts, but it doesn't do a good job of preventing them when adding new symbols to a library (primarily because it doesn't use static imports by default), and your proposal makes that part of the problem worse. If std.x were to become common practice, then any time that you added a symbol to a library when that symbol was already used by another module, you'd create conflicts (combined with the fact that private doesn't hide symbol names but merely makes them inaccessible, this could result in a lot of symbol name conflicts).
 Not to mention, it'll make it a lot more confusing
 to find what modules stuff is actually in if people start doing stuff like
 
 std.sort(arr);
 
 In the case of sort, you may know where it's from - particularly since
 it's so common - but the less well-known the function is, the less likely
 that is at all obvious where it comes from, and if you're dealing with
 3rd party software, then it wouldn't be at all obvious. For instance, how
 would you know that party.foo is really party.bar.foo? You wouldn't.

Why should you?

Do you know what the foo function does? If you don't, you're going to have to look it up. And if you don't know what module it comes from, you can't do that. You also have to know where foo is from if a foo function is added to another module and causes a conflict, because you're going to have to give the full import path to actually use it. That's currently true with just bare foo as well, but party.foo gives the illusion of specifying where foo is from without actually specifying where it's from. At least right now, if foo is used with its import path, you know that that's actually its import path. Also, what happens if we want to add a module named sort later? The fact that people are using std.sort means that adding std.sort as a module will break code. Granted, it's not very likely that we're going to add a module named sort, but there are plenty of other symbol names that it could happen with. But then again, if we decided to provide a module with all of the major sort algorithms, then maybe we _would_ create a module named std.sort. Just because we don't see a need now doesn't mean that we won't later. In either case, by allowing std.x where x is a symbol in any sub-module of std, you're going to create conflicts any time that you add a module which has the same name as an existing symbol anywhere in the library.
 Being so lax about
 importing could really harm code readibility (and maintainibility, since
 it
 increases the odds of name clashes). So, I'm inclined to say that that is
 a
 _bad_ idea.

Maybe if you produce a solid example, I'd be convinced.

Well, as I've pointed with a few examples here, your proposal will increase the chances of adding symbol conflicts any time that a symbol is added to a library all just so that you can do std.algorithm.sort instead of std.algorithm.submodule1.sort once sort has been moved to std.algorithm.sumodule1. And we could make it possible to do std.algorithm.sort without adding all of those possible conflicts. The simplest solution would simply be to make it so that if std.algorithm.sort is used, and std.algorithm is a package with a std.algorithm.package module, then the compiler looks in all of the sub-modules of std.algorithm to find sort. That solves the problem right there without increasing the odds of symbol conflicts across the entire library like your proposal does. But personally, I like the idea of making it so that publicly imported symbols can be accessed as if they were in the module that publicly imported them (with package.d being treated as if it had the same name as the package that it's in). That's essentially how it already works except when specifying the full import path for a symbol. And that way, you can specify in std.algorithm.package.d exactly what you want to be imported when std.algorithm is imported (including using : to restrict it to specific symbols in a module), and only those symbols will be treated as if they were part of std.algorithm - both for importing purposes and when specifying the import path when using a symbol. The library maintainer then has control over which symbols get used with which import paths. - Jonathan M Davis
Mar 30 2012
prev sibling next sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2012-03-30 14:46:19 +0000, Andrei Alexandrescu 
<SeeWebsiteForEmail erdani.org> said:

 Destroy!

Since you're asking… One thing the current system avoids is unresolvable symbols. If two symbol name clashes, you just qualify them fully and it'll always be unambiguous. For instance: .std.algorithm.sort(…) Now, if std.algorithm becomes both a module and a package, you could have both a sort function and a sort submodule with no way to distinguish between the two, even when fully qualified. I think this is why D does not allow modules to have the same name as packages. I understand that you try to work around this problem by inventing a .std.algorithm.package scope. Then you make it's content imported automatically inside the .std.algorithm scope for backward compatibility (and convenience). The problem is that if .std.algorithm.package contains a sort function and there is also a module called std.algorithm.sort, the fully-qualified name of that 'sort' module will become ambiguous. Moreover, whether the fully-qualified name .std.algorithm.sort is ambiguous or not depends on what modules were imported, which is not a very desirable behaviour. So to make sure there is no unresolvable fully-qualified names, when importing std.algorithm.sort the compiler should make sure that no symbol called 'sort' already exist in the .std.algorithm scope (which includes the symbols in .std.algorithm.package and all other packages inside std.algorithm). This is clearly untenable. - - - I recognize the need. If I may, I'll propose something simpler: 1. allow both std/algorithm.d and std/algorithm/sort.d to exist 2. importing std.algorithm.sort will also implicitly import std.algorithm (if it exists) and std (if it exists) 3. if any symbol of std.algorithm clash with the std.algorithm.sort module name, you get an error when importing std.algorithm.sort. Effectively, importing std.algorithm.sort becomes synonymous to importing std, std.algorithm, and std.algorithm.sort. This is what's needed to detect clashes in fully-qualified names. The only issue now (beside a few more imports) is that if std.algorithm imports any of its submodule it becomes a circular import. That's usually fine in D, but not when you have module constructors. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Mar 30 2012
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 30 Mar 2012 17:45:36 -0400, Michel Fortin
<michel.fortin michelf.com> wrote:

 On 2012-03-30 14:46:19 +0000, Andrei Alexandrescu  =

 <SeeWebsiteForEmail erdani.org> said:

 Destroy!

Since you're asking=E2=80=A6 One thing the current system avoids is unresolvable symbols. If two =

 symbol name clashes, you just qualify them fully and it'll always be  =

 unambiguous. For instance:

 	.std.algorithm.sort(=E2=80=A6)

 Now, if std.algorithm becomes both a module and a package, you could  =

 have both a sort function and a sort submodule with no way to  =

 distinguish between the two, even when fully qualified. I think this i=

 why D does not allow modules to have the same name as packages.

 I understand that you try to work around this problem by inventing a  =

 .std.algorithm.package scope. Then you make it's content imported  =

 automatically inside the .std.algorithm scope for backward compatibili=

 (and convenience). The problem is that if .std.algorithm.package  =

 contains a sort function and there is also a module called  =

 std.algorithm.sort, the fully-qualified name of that 'sort' module wil=

 become ambiguous. Moreover, whether the fully-qualified name  =

 .std.algorithm.sort is ambiguous or not depends on what modules were  =

 imported, which is not a very desirable behaviour.

So this becomes an error. I don't see this as a major problem. Just don't name a module sort inside std/algorithm. This is no different than ambiguous templates, which are allowed until y= ou want to instantiate one. -Steve
Apr 02 2012
parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2012-04-02 13:04:31 +0000, "Steven Schveighoffer" 
<schveiguy yahoo.com> said:

 On Fri, 30 Mar 2012 17:45:36 -0400, Michel Fortin
 
 The problem is that if .std.algorithm.package
 contains a sort function and there is also a module called
 std.algorithm.sort, the fully-qualified name of that 'sort' module wil l
 become ambiguous. Moreover, whether the fully-qualified name
 .std.algorithm.sort is ambiguous or not depends on what modules were
 imported, which is not a very desirable behaviour.

So this becomes an error. I don't see this as a major problem. Just don't name a module sort inside std/algorithm. This is no different than ambiguous templates, which are allowed until y ou want to instantiate one.

If you have ambiguous templates in the same module, it'll always be ambiguous irrespective of what you import (and you can blame the module's designer for it). If you have ambiguous templates residing in different modules the symbol will be unambiguous until you've imported the second module (same as overloaded functions). At that point you can disambiguate using the fully-qualified name of the template (or function). Whereas if the fully-qualified name of a module becomes ambiguous because of a symbol in another module, there is no way to disambiguate. All you can do is avoid importing the two conflicting modules together, just like when you encounter two headers trying to define the same symbol in C/C++. With the way D modules were designed, this cannot happen because you can't have a module with the same name as a package. I always thought this was done on purpose, but I might be wrong. Whatever we do, I think it'd be a nice property to preserve that fully-qualified names should always work. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Apr 02 2012
parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2012-04-04 14:08:34 +0000, "Steven Schveighoffer" 
<schveiguy yahoo.com> said:

 On Mon, 02 Apr 2012 20:44:09 -0400, Michel Fortin  
 <michel.fortin michelf.com> wrote:
 
 Whereas if the fully-qualified name of a module becomes ambiguous  
 because of a symbol in another module, there is no way to disambiguate. 
  All you can do is avoid importing the two conflicting modules 
 together,  just like when you encounter two headers trying to define 
 the same  symbol in C/C++.

How does this happen? The FQN cannot be ambiguous.

Sure it can if I follow DIP16, because module names can become ambiguous. Let's try this with an example. First, let's define a pretty standard module: std/algorithm/sort.d: module std.algorithm.sort; void sort(T)(T[] array); Here the fully-qualified name of the sort function is .std.algorithm.sort.sort. But according to DIP16's lookup rules, the sort function is also available (if not ambiguous) at: std.sort std.algorithm.sort Question 1: since there is already a module at .std.algorithm.sort, doesn't the module name become ambiguous with the sort function it itself contains? Let's assume the module's name take priority and does not conflict so we can continue. Now we create the package.d file: std/algorithm/package.d: import std.algorithm.sort; And now I write this somewhere in my code: std.algorithm.sort Question 2: does std.algorithm.sort refer to the std.algorithm.sort *module* or to std.algorithm.package.sort *function* publicly imported from the std.algorithm.sort module? Again, we could decide that the module takes priority. But having symbols take priority over one another is not how D has resolved ambiguities up to now; what D does usually is make it a hard error. If we make it an error the fully-qualified name of anything in the std.algorithm.sort module becomes inaccessible. If we do not make it an error, the module name shadows the function imported in the package. And the problem with shadowing is that it can silently change what code you're calling depending on what you've imported (if you need an example, just ask). You might think I'm trying to split hair in four to find flaws, that no one is going to do things that dumb, but I unfortunately think the problematic pattern is already quite common. How many times have we seen modules containing a class, variable, or function having the same name as the module's name? What should happen when you publicly import those modules in the package.d file? The practice might not be too prevalent in Phobos because modules tend to do a lot of things and are therefore named more generically, but it still happens. For instance: std.array.array std.getopt.getopt std.regex.regex Say you wanted to create a package.d file directly for the whole package std, what should be done for those? -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Apr 04 2012
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 04/04/2012 07:53 PM, Steven Schveighoffer wrote:
 On Wed, 04 Apr 2012 12:33:26 -0400, Michel Fortin
 <michel.fortin michelf.com> wrote:

 On 2012-04-04 14:08:34 +0000, "Steven Schveighoffer"
 <schveiguy yahoo.com> said:

 The FQN cannot be ambiguous.

Sure it can if I follow DIP16, because module names can become ambiguous. Let's try this with an example. First, let's define a pretty standard module: std/algorithm/sort.d: module std.algorithm.sort; void sort(T)(T[] array); Here the fully-qualified name of the sort function is .std.algorithm.sort.sort. But according to DIP16's lookup rules, the sort function is also available (if not ambiguous) at: std.sort std.algorithm.sort Question 1: since there is already a module at .std.algorithm.sort, doesn't the module name become ambiguous with the sort function it itself contains?

OK, but when is it ever valid to refer to a module when the semantic expectations are for something other than a module? I can only think of two places where module names are used, inside an import statement and inside a module statement (three if you count the prefix of a FQN). Maybe I'm missing some case...

__traits(allMembers, pack.age.mod.ule);
...
 You might think I'm trying to split hair in four to find flaws, that
 no one is going to do things that dumb, but I unfortunately think the
 problematic pattern is already quite common. How many times have we
 seen modules containing a class, variable, or function having the same
 name as the module's name?

Tango anyone? :) But yes, I think the issue really becomes, we need to look at context when deciding the semantic meaning of a symbol. I don't think this violates the context-free grammar, because wouldn't this only come into play at the semantic level? Not a compiler writer/hacker, so I don't know.

No symbol is resolved until semantic, but I don't think hiding the module/package symbol if any clashing symbol in the module/any subpackage exists is a satisfactory solution.
Apr 04 2012
parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2012-04-04 19:48:32 +0000, "Steven Schveighoffer" 
<schveiguy yahoo.com> said:

 On Wed, 04 Apr 2012 14:03:07 -0400, Timon Gehr <timon.gehr gmx.ch> wrote:
 
 No symbol is resolved until semantic, but I don't think hiding the  
 module/package symbol if any clashing symbol in the module/any  
 subpackage exists is a satisfactory solution.

Then we must come up with a way to hide the submodules of a virtual module. I tried this, which fails on the current compiler: import x; int x; So it seems we have two choices here: 1. DIP16 needs to get more complex to make package submodules not accesible as individual modules. 2. Start contextually interpreting identifiers at least in the case of modules vs. non-modules. I'd suggest option 2 allows for better backwards compatibility and more flexibility.

I don't think option 2 is realistic (see my other post). I don't think option 1 is an improvement over what we have. I mean, if you're going to hide the submodules, what is the benefit compared to just using a different package name for the implementation modules? You can already refactor std.algorithm this way with no change in the compiler: module std.algorithm; public import std.algorithm_impl.sort; public import std.algorithm_impl.map; public import std.algorithm_impl.blah_blah_blah; … If we add a language feature, it should be an noticeable improvement over this situation. I think we need a third option. Here's an idea: we could allow modules having a single symbol with the same name as the module to behave as if they were the symbol itself, just like templates behaves. For instance: module std.algorithm.sort; void sort(int[] t); Now you can import std.algorithm.sort and then use the std.algorithm.sort fully qualified name as if it was a function, even though it's the module name (std.algorithm.sort.sort would be the function's name). Or maybe we could just allow "alias sort this" at module level? Or is it allowed already? -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Apr 04 2012
next sibling parent Michel Fortin <michel.fortin michelf.com> writes:
On 2012-04-05 00:50:49 +0000, Michel Fortin <michel.fortin michelf.com> said:

 I think we need a third option.
 
 Here's an idea: we could allow modules having a single symbol with the 
 same name as the module to behave as if they were the symbol itself, 
 just like templates behaves. For instance:
 
 	module std.algorithm.sort;
 
 	void sort(int[] t);
 
 Now you can import std.algorithm.sort and then use the 
 std.algorithm.sort fully qualified name as if it was a function, even 
 though it's the module name (std.algorithm.sort.sort would be the 
 function's name).
 
 Or maybe we could just allow "alias sort this" at module level? Or is 
 it allowed already?

Forgot to follow through with what "import std.algorithm" would do. If std.algorithm is a package, then it should only contain modules (which can each pose for a function/class/struct/variable thanks to the feature described above). Importing the package std.algorithm would open the std/algorithm.d file, read a list of package from the file, and proceed to import each of them (just as if you'd have imported each of them separately). Here is an idea for declaring the list of modules to import when you import a package in std/algorithm.d: package std.algorithm; module std.algorithm.sort; module std.algorithm.map; ... (I'm _not_ using a package.d file because that'd incur two lookups: one for std/algorithm.d in case it's a module, and one for std/algorithm/package.d in case it's a package. It also removes the possibility of both files existing at the same time which would be confusing.) This way, modules and packages stay exactly the same in the language as they are now, except that you can now import a package directly instead of importing each module separately. And modules with a single symbol with the same name as the module are treated as if they were that symbol. This allows migrating a module to a package without breaking the existing code base, but the drawback is that each symbol in the module becoming a package need to become its own submodule. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Apr 04 2012
prev sibling next sibling parent reply deadalnix <deadalnix gmail.com> writes:
Le 05/04/2012 13:46, Steven Schveighoffer a écrit :
 I don't like this proposal, simply because this means one
 function/symbol per submodule. It should be more flexible than that. But
 I like the line of thinking.

 Let's re-examine the issue. We need to allow splitting of a module X.d
 into pieces for maintenance (or possibly accessibility -- disallowing
 friends). But we don't want to break code which currently uses FQN to
 access X's symbols.

 I think the following might work:

 algorithm.d:

 import this = std.algorithm_impl.sort;

 Which then imports std.algorithm_impl.sort, and effectively aliases all
 its symbols into algorithm.d. If std.algorithm_impl.sort defines a
 function called sort, then it's also aliased to std.algorithm.sort. In
 essence, sort has *two* FQN, but there are no FQN that refer to more
 than one symbol.

 I purposely left out the package.d idea because it's orthogonal to this.

 -Steve

The behavior you described has been proposed for public import, and have been discussed. This is interesting. You propose an alternative syntax with is fine and have the advantage to not disturb already existing public imports, but have the drawback to create a new syntax, again. If such a syntax is adopted, what would be the point of public imports ? If it is still useful, then your syntax is a better choice, otherwise, we'd better modify public import.
Apr 05 2012
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 04/05/2012 02:58 PM, Steven Schveighoffer wrote:
 No, public imports simply mean that you can view the publicly imported
 module. It does *not* add aliases to the importing module.

Have you tried it?
Apr 05 2012
next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, April 05, 2012 09:49:59 Steven Schveighoffer wrote:
 On Thu, 05 Apr 2012 09:23:25 -0400, Timon Gehr <timon.gehr gmx.ch> wrote:
 On 04/05/2012 02:58 PM, Steven Schveighoffer wrote:
 No, public imports simply mean that you can view the publicly imported
 module. It does *not* add aliases to the importing module.

Have you tried it?

I just did. OK, what the hell are we arguing about then?! DIP16 is worksforme :) See this part of the spec: http://dlang.org/module.html Read the part on public modules. You may understand why I didn't know about that "feature" (which seems to work on all the installed compilers I have, back to 2.033). I just read the public import part of TDPL, and there it is, all spelled out quite nicely. I'm going to file a bug against the spec... grrr... I'm now firmly in the "we don't need to change anything, just refactor the modules and use public import" camp. We don't even need to change ANYTHING in the compiler! Forget everything I said before in this thread ;) I suppose the only thing we don't get is being able to have a module and a package with the same FQN. I don't see that being a major issue.

What doesn't work is being able to turn a module into a package with the same name. Right now, we could create a std.alg package with sub-modules containing all of std.algorithm's functionality and change std.algorithm to pubicly import them all, but you can't turn std.algorithm itself into a package without breaking code. The package.d portion of the proposal makes it so that you can. Now, public import's current behavior is _almost_ enough to make the second part completely unnecessary. The only change that would be required would be to make it so that std.algorithm.x looks in std/algorithm/package.d if there's no std.algorithm.x module, because otherwise the public imports in package.d would be putting all of the symbols in std.algorithm.package, not std.algorithm (that, and package.d isn't legal at present). So, the whole point of this proposal - to seemlessly allow the transition of a module to a package in place - _does_ require a language/compiler change. But moving stuff from a module to a new package does work already. - Jonathan M Davis
Apr 05 2012
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/5/12 4:26 PM, Steven Schveighoffer wrote:
 On Thu, 05 Apr 2012 17:00:56 -0400, Jonathan M Davis
 <jmdavisProg gmx.com> wrote:

 On Thursday, April 05, 2012 15:30:17 Steven Schveighoffer wrote:

 I don't see how. Just move the code into another module and publicly
 import that module from std/algorithm.d. Problem pretty much solved.

The issue is code organization. If you want to split up std.algorithm (or std.datetime or whatever) into multiple modules, you have to create a new package with a completely different name with no connection to the original save for the fact that the original publicly imports it.

My view is that people will not import the smaller modules, they will only ever import std.algorithm.

I think we should be looking for a solution that not only allows replacing module -> package transparently, but also allows people to import the newly introduced fine-grained modules. Andrei
Apr 05 2012
next sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2012-04-05 21:43:24 +0000, Andrei Alexandrescu 
<SeeWebsiteForEmail erdani.org> said:

 I think we should be looking for a solution that not only allows 
 replacing module -> package transparently, but also allows people to 
 import the newly introduced fine-grained modules.

I think it'd be valuable too. But how do you do that without creating ambiguous fully qualified names? One way would be that by importing a module inside a package, the compiler would check first that the package file contains no symbol with that module name. In other words, if you import std.algorithm.sorting, it'd check that "algorithm" is not a symbol defined in the "std" package file (if it exists for std) and that no "sorting" symbol exists in the "std.algorithm" package file. That would work. But the problem is that if the package file publicly imports all of its submodules, then you'll need to parse all the imported files to determine if the module you're trying to import conflicts with a package-level symbol. A more practical option would be limit the package files to only two things: 1. a list of modules to import when you import the package (importing the package would be akin importing all modules in this list) 2. a list of symbols aliased in the package's namespace (aliased symbols can be accessed using the package name as a prefix and through selective imports) Since the list of aliases is stored directly in the package file, reading the package file to get the names of each alias is enough to tell there is no conflict with the module name when you're trying to import it. No need to open the other modules the package refers to, or to resolve the symbols the aliases refer to: you only need the name of each alias to verify there is no conflict. For instance, a package file could look like this: package std.algorithm; // modules to import when a file is importing the package // (those are not imported inside this package's namespace) module std.algorithm.sorting; module std.algorithm.mapping; ... // symbols to alias to this package namespace // (importing a module with one of these names is an error) alias std.algorithm.sorting.completeSort completeSort; alias std.algorithm.sorting.isSorted isSorted; alias std.algorithm.sorting.partialSort partialSort; alias std.algorithm.sorting.schwartzSort schwartzSort; alias std.algorithm.sorting.sort sort; ... The drawback is that it's hard to maintain the list of fully-qualified names up to date if you change it in the various modules. But if the goal is only to preserve backward compatibility, creating that list of aliases could be a one-time thing. New symbols would continue to be imported when you import std.algorithm, but they'd not be available directly under std.algorithm: you'd need to use std.algorithm.sorting.newSort if you find yourself in a situation that requires a fully-qualified name. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Apr 05 2012
parent reply deadalnix <deadalnix gmail.com> writes:
Le 06/04/2012 01:32, Michel Fortin a écrit :
 On 2012-04-05 21:43:24 +0000, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> said:

 I think we should be looking for a solution that not only allows
 replacing module -> package transparently, but also allows people to
 import the newly introduced fine-grained modules.

I think it'd be valuable too. But how do you do that without creating ambiguous fully qualified names?

It isn't possible. But as already mentioned, all name doesn't make sense in all situation, so most of the time, disambiguation can be done. Plus, we want to be able to split module when they grow, and in such a situation, collisions will never happen, because all symbols comes from the same module in a first place.
Apr 06 2012
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/6/12 4:43 AM, deadalnix wrote:
 Le 06/04/2012 01:32, Michel Fortin a écrit :
 On 2012-04-05 21:43:24 +0000, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> said:

 I think we should be looking for a solution that not only allows
 replacing module -> package transparently, but also allows people to
 import the newly introduced fine-grained modules.

I think it'd be valuable too. But how do you do that without creating ambiguous fully qualified names?

It isn't possible. But as already mentioned, all name doesn't make sense in all situation, so most of the time, disambiguation can be done. Plus, we want to be able to split module when they grow, and in such a situation, collisions will never happen, because all symbols comes from the same module in a first place.

One other desirable feature is library distribution. Library writers should be able to offer library xyz as a sheer directory. Then library users should be able to import path.to.xyz and simply get going, or even rename xyz to xyz1 and import path.to.xyz1 without a problem. Andrei
Apr 06 2012
parent reply deadalnix <deadalnix gmail.com> writes:
Le 06/04/2012 16:49, Andrei Alexandrescu a écrit :
 On 4/6/12 4:43 AM, deadalnix wrote:
 Le 06/04/2012 01:32, Michel Fortin a écrit :
 On 2012-04-05 21:43:24 +0000, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> said:

 I think we should be looking for a solution that not only allows
 replacing module -> package transparently, but also allows people to
 import the newly introduced fine-grained modules.

I think it'd be valuable too. But how do you do that without creating ambiguous fully qualified names?

It isn't possible. But as already mentioned, all name doesn't make sense in all situation, so most of the time, disambiguation can be done. Plus, we want to be able to split module when they grow, and in such a situation, collisions will never happen, because all symbols comes from the same module in a first place.

One other desirable feature is library distribution. Library writers should be able to offer library xyz as a sheer directory. Then library users should be able to import path.to.xyz and simply get going, or even rename xyz to xyz1 and import path.to.xyz1 without a problem. Andrei

Good point. That is an argument for package.d, all.d or _.d .
Apr 06 2012
parent reply David Gileadi <gileadis NSPMgmail.com> writes:
On 4/6/12 10:52 AM, deadalnix wrote:
 Good point. That is an argument for package.d, all.d or _.d .

Or [packagename].d, where [packagename] is the name of its sibling folder (std/algorithm.d in the case of std/algorithm/sort.d).
Apr 06 2012
parent deadalnix <deadalnix gmail.com> writes:
Le 06/04/2012 20:03, David Gileadi a écrit :
 On 4/6/12 10:52 AM, deadalnix wrote:
 Good point. That is an argument for package.d, all.d or _.d .

Or [packagename].d, where [packagename] is the name of its sibling folder (std/algorithm.d in the case of std/algorithm/sort.d).

No, because in this case, this isn't in the same folder. Still I prefer that opetion because : 1/ You mostly distribute lib as archives, or in a managed way (package manager, build system, etc . . .) 2/ It is simpler as import a.b.c always does the same thing, and other solution introduce complexity in that process, and possible cases of error.
Apr 06 2012
prev sibling next sibling parent reply deadalnix <deadalnix gmail.com> writes:
Le 05/04/2012 23:43, Andrei Alexandrescu a écrit :
 On 4/5/12 4:26 PM, Steven Schveighoffer wrote:
 On Thu, 05 Apr 2012 17:00:56 -0400, Jonathan M Davis
 <jmdavisProg gmx.com> wrote:

 On Thursday, April 05, 2012 15:30:17 Steven Schveighoffer wrote:

 I don't see how. Just move the code into another module and publicly
 import that module from std/algorithm.d. Problem pretty much solved.

The issue is code organization. If you want to split up std.algorithm (or std.datetime or whatever) into multiple modules, you have to create a new package with a completely different name with no connection to the original save for the fact that the original publicly imports it.

My view is that people will not import the smaller modules, they will only ever import std.algorithm.

I think we should be looking for a solution that not only allows replacing module -> package transparently, but also allows people to import the newly introduced fine-grained modules. Andrei

Why not limit name collision to name which make sense ? For instance, import std.a.b.c is a module. if it refers also to a function, this import doesn't make any sense, so, even if we have a name collision, this isn't a big deal (except maybe for reflection ?). Same goes for std.a.b.c(); which is a function call, and obviously not the module. Here what I propose to resolve names : 1/ import does always find the .d corresponding file. No exception. 2/ Module a.b.c is in package a, a.b and a.b.c . Any package declaration in a.b.c match package a.b (one level is removed). 3/ When a name is used in the code and have to be resolved, the following process occurs : - The compiler find all stuff that have this name. - The compiler discard all stuffs that have this name and doesn't make sense. - If all remaining items are overload of the same item, then standard best match rule apply. - If all remaining items aren't in the same module, or overload or different items, an error occurs. This is never a problem in the case of big modules splitted in submodules. Some examples : a.d public import a.b; // import a/b.d class b { static void foo() {} } **************** a/b.d public import a.b.c; // import a/b/c.d void foo(int i) {} **************** a/b/c.d void foo() {} **************** main.d import a; foo(); // foo from a/b/c.d foo(2); // foo from a/b.d a.foo(); // foo from a/b/c.d a.b.foo(); // Error, match both a/b.d and a/b/c.d a.b.foo(2); // foo from a/b.d a.b.c.foo(); // foo from a/b/c.d
Apr 06 2012
parent Michel Fortin <michel.fortin michelf.com> writes:
On 2012-04-06 09:41:27 +0000, deadalnix <deadalnix gmail.com> said:

 Why not limit name collision to name which make sense ?

You're proposing that fully qualified names sometime work and sometime do not work based on what you've imported and the context of usage. This makes any FQN conflict hard to detect, because the error will be dependent on the context in which you use the name. You say it'll never happen for modules spitted in submodules, but I disagree. Someday, someone will add a symbol to a submodule without checking every package/module file underneath. Unit tests in the module won't catch the problem because for them to catch the problem the parent module/package has to be imported. Only on certain usages, with the right imports and in the right context will the error occur. In fact, if you add disambiguation based on what "makes sense" you're making the error more obscure and harder to detect in the first place because sometime it'll occur and sometime not depending on hard to predict conditions. If we allow arbitrary symbols to live inside a package, any conflict in FQN should be caught as soon as possible, ideally as soon as you compile after having added the symbol, and even if you don't use the symbol anywhere. Just like how you get an error today when you try to import a module matching a known package name, you shouldn't be allowed to import a module matching a known symbol name, or create a symbol matching a module fully qualified name. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Apr 06 2012
prev sibling next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, April 06, 2012 08:09:28 Steven Schveighoffer wrote:
 I feel like most people will still import the main package module, and not
 the submodules.  I don't think I ever wrote a piece of java code that
 didn't have:
 
 import java.io.*;

Which is actually considered bad practice in Java, though a lot of people do like to do it. What's generally considered good practice is to explicitly import every module/class, and Eclipse likes to underline imports in red if you don't.
 I keep coming back to std.container.  Already it's large, and full of
 unrelated types.  It's only going to get worse.
 
 Now, I fully agree that having some way to import a package by itself
 without having to import all its modules would be ideal (as well as
 splitting a large module into submodules that live in the same
 namespace).  I think DIP15 has that covered.

DIP15 doesn't fix the explicit path problem though. You can't change std/algorithm.d into std/algorithm/ (with sorting.d, search.d, etc.) without breaking code. You could make std/algorithm.d publicly import std/alg/* and then DIP15 would allow you to import std.alg to get all of its sub-modules, but you're still forced to use a module to publicly import symbols as part of a migration path, and you can't split a module in place. DIP15 also less powerful than having a package.d, since it'll automatically import ever public symbol when it may be desirable to have it only import a subset of them or to publicly import something from another module (e.g. as part of a migration path - like moving the benchmarking code in std.datetime to std.benchmark rather than lumping it with the std.datetime stuff when it's split into separate modules). That may or may not be a big deal, but it does mean that DIP15 isn't as powerful as DIP16's package.d solution in that regard. DIP15 also doesn't give a means of documentating a package. By having a package.d file, we have a really good place to provide an overarching description for the package in ddoc. So, while I don't like the second part of DIP16 at all (i.e. the std.sort stuff), I think that the package.d part is a much better way to go. - Jonathan M Davis
Apr 06 2012
parent Jacob Carlborg <doob me.com> writes:
On 2012-04-07 02:25, Jonathan M Davis wrote:
 On Friday, April 06, 2012 08:09:28 Steven Schveighoffer wrote:
 I feel like most people will still import the main package module, and not
 the submodules.  I don't think I ever wrote a piece of java code that
 didn't have:

 import java.io.*;

Which is actually considered bad practice in Java, though a lot of people do like to do it. What's generally considered good practice is to explicitly import every module/class, and Eclipse likes to underline imports in red if you don't.

import foo.bar.* Is used all over the place in SWT. -- /Jacob Carlborg
Apr 07 2012
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 06 Apr 2012 20:25:23 -0400, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:

 DIP15 doesn't fix the explicit path problem though. You can't change
 std/algorithm.d into std/algorithm/ (with sorting.d, search.d, etc.)  
 without
 breaking code. You could make std/algorithm.d publicly import std/alg/*  
 and
 then DIP15 would allow you to import std.alg to get all of its  
 sub-modules,
 but you're still forced to use a module to publicly import symbols as  
 part of
 a migration path, and you can't split a module in place.

I think either you or I am missing something. In DIP15, if you define std/algorithm/_.d, and then import std.algorithm, it imports std/algorithm/_.d, which then 1. publicly imports other modules, and 2. aliases symbols to the name std.algorithm.symbol. At least, this is how I understand the intent. It seems equivalent to me to the package.d proposal, it's just using _.d instead of package.d. If you import std.algorithm.sorting, and try and use std.algorithm.sort, yes it will not work. But this does not break existing code (which does not import std.algorithm.sorting), and I find it odd that we want to make std.algorithm.sort work if you don't import std.algorithm. -Steve
Apr 09 2012
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 14:24:10 -0400, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:

 On Thursday, April 05, 2012 09:49:59 Steven Schveighoffer wrote:
 On Thu, 05 Apr 2012 09:23:25 -0400, Timon Gehr <timon.gehr gmx.ch> I  
 suppose the only thing we don't get is being able to have a module and a
 package with the same FQN. I don't see that being a major issue.

What doesn't work is being able to turn a module into a package with the same name. Right now, we could create a std.alg package with sub-modules containing all of std.algorithm's functionality and change std.algorithm to pubicly import them all, but you can't turn std.algorithm itself into a package without breaking code.

But so what? nobody has any code like: import std.algorithm.sort; So who cares where that module goes? I agree it would be ideal to put it there, but I don't think it's strictly necessary. And there is no need for the shortcut for fully qualified names.
 So, the whole point of this proposal - to seemlessly allow the  
 transition of a
 module to a package in place - _does_ require a language/compiler change.

I don't see how. Just move the code into another module and publicly import that module from std/algorithm.d. Problem pretty much solved. BTW, importing a directory was already proposed in DIP15. http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP15 -Steve
Apr 05 2012
prev sibling parent deadalnix <deadalnix gmail.com> writes:
Le 05/04/2012 14:58, Steven Schveighoffer a écrit :
 On Thu, 05 Apr 2012 08:49:24 -0400, deadalnix <deadalnix gmail.com> wrote:

 Le 05/04/2012 13:46, Steven Schveighoffer a écrit :
 I don't like this proposal, simply because this means one
 function/symbol per submodule. It should be more flexible than that. But
 I like the line of thinking.

 Let's re-examine the issue. We need to allow splitting of a module X.d
 into pieces for maintenance (or possibly accessibility -- disallowing
 friends). But we don't want to break code which currently uses FQN to
 access X's symbols.

 I think the following might work:

 algorithm.d:

 import this = std.algorithm_impl.sort;

 Which then imports std.algorithm_impl.sort, and effectively aliases all
 its symbols into algorithm.d. If std.algorithm_impl.sort defines a
 function called sort, then it's also aliased to std.algorithm.sort. In
 essence, sort has *two* FQN, but there are no FQN that refer to more
 than one symbol.

 I purposely left out the package.d idea because it's orthogonal to this.

 -Steve

The behavior you described has been proposed for public import, and have been discussed. This is interesting. You propose an alternative syntax with is fine and have the advantage to not disturb already existing public imports, but have the drawback to create a new syntax, again. If such a syntax is adopted, what would be the point of public imports ? If it is still useful, then your syntax is a better choice, otherwise, we'd better modify public import.

No, public imports simply mean that you can view the publicly imported module. It does *not* add aliases to the importing module. for example: foo.d: module foo; int x; bar.d: module bar; public import foo; int y; main.d: import bar; // publicly imports foo. void main() { foo.x = 5; // ok, publicly imported via bar (non public, this would be an error) bar.y = 5; // ok bar.x = 5; // error, public import doesn't create new fully qualified name for foo.x } With the system I propose, using the specific technique would make it so bar.x would also be valid, and would refer to foo.x I think we need new syntax, or a new language feature, because you don't want to alter the operation of existing code. Simply changing public imports would cause lots of existing code to be reinterpreted, possibly in a way not intended, or that would be ambiguous. -Steve

I know. I think you answered too fast. I wasn't stating that public import are the same as what you propose. I was discussing the fact that both probably answer the same need and that 2 different syntax is something to avoid. Code wouldn't be broken in unexpected way, because thing will cause either a compile error (collision of names, but the problem exists with other solutions too) or is illegal in the current shape of the language.
Apr 05 2012
prev sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2012-04-05 11:46:38 +0000, "Steven Schveighoffer" 
<schveiguy yahoo.com> said:

 I don't like this proposal, simply because this means one function/symbo l
 per submodule.  It should be more flexible than that.  But I like the li ne
 of thinking.

I have the same reserve about it's lack of flexibility too. It would be a nice thing to have for all those libraries having one module per class as it'd reduce the length of the fully qualified name you have to use when you need to disambiguate. But for moving a module like std.algorithm to a package, it's cumbersome.
 Let's re-examine the issue.  We need to allow splitting of a module X.d
 into pieces for maintenance (or possibly accessibility -- disallowing
 friends).  But we don't want to break code which currently uses FQN to
 access X's symbols.
 
 I think the following might work:
 
 algorithm.d:
 
 import this = std.algorithm_impl.sort;
 
 Which then imports std.algorithm_impl.sort, and effectively aliases all
 its symbols into algorithm.d.  If std.algorithm_impl.sort defines a
 function called sort, then it's also aliased to std.algorithm.sort.  In
 essence, sort has *two* FQN, but there are no FQN that refer to more tha n
 one symbol.

This is what a public import should do. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Apr 05 2012
parent deadalnix <deadalnix gmail.com> writes:
Le 05/04/2012 16:47, Steven Schveighoffer a écrit :
 But for moving a module like std.algorithm to a package, it's cumbersome.

Not really. Just move the files to another directory (i.e. std/internal/algorithm) and publicly import them.

std/internal isn't good. If I just want to import sort, I would have to do std.internal.algorithm.sort, which isn't good. Plus, the package accessibility modifier would be broken with such a pattern. The all.d, package.d, _.d, or, as I propose foldername.d (and submodules into foldername) do not break package accessibility modifier, which is better.
Apr 05 2012
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-04-04 18:33, Michel Fortin wrote:

 You might think I'm trying to split hair in four to find flaws, that no
 one is going to do things that dumb, but I unfortunately think the
 problematic pattern is already quite common. How many times have we seen
 modules containing a class, variable, or function having the same name
 as the module's name? What should happen when you publicly import those
 modules in the package.d file?

I do that all the time in my libraries. -- /Jacob Carlborg
Apr 04 2012
prev sibling next sibling parent Michel Fortin <michel.fortin michelf.com> writes:
On 2012-04-04 17:53:24 +0000, "Steven Schveighoffer" 
<schveiguy yahoo.com> said:

 But yes, I think the issue really becomes, we need to  look at context 
 when deciding the semantic meaning of a symbol.  I don't  think this 
 violates the context-free grammar, because wouldn't this only  come 
 into play at the semantic level?  Not a compiler writer/hacker, so I  
 don't know.

You'd need a whole lot of context. Let's say I write: .a.b.c.d = .e.f.g.h; Can you tell me which letters are the name of a package/module, and which are the name of something else? It could be this: module a.b; struct c { static int d; } module e.f.g; int h; or it could be this: module a; struct b { struct c { struct d { static void opAssign(int); } } } module e; struct f { enum g { h = 1 } } or a multitude of other variants. If you can't know by looking at the assignment above, how will the parser know? Fully qualified names really need to refer to a single thing.
 Say you wanted to create a package.d file directly for the whole 
 package  std, what should be done for those?

No, let's not do that. Ever. :)

Phobos is just an example. People will want to use "package.d" files for their own libraries too, and will complain if it doesn't work. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Apr 04 2012
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/4/12 10:53 AM, Steven Schveighoffer wrote:
 On Wed, 04 Apr 2012 12:33:26 -0400, Michel Fortin
 <michel.fortin michelf.com> wrote:
 Question 2: does std.algorithm.sort refer to the std.algorithm.sort
 *module* or to std.algorithm.package.sort *function* publicly imported
 from the std.algorithm.sort module?

The function. A symbol that is not specified as the module name in import statement or in a module statement is always *not* a module. I think our one saving grace here is that when you want to import a specific symbol from a module, this is not the syntax: import std.stdio.writefln; So there is never any ambiguity as to whether you mean a module identifier or other symbol.

Interesting. But isn't there an ambiguity when the symbol is not the last one in the chain? Consider: a.b.c.x Could be static member d of class c in module a.b, or module member d in module a.b.c. Andrei
Apr 05 2012
next sibling parent deadalnix <deadalnix gmail.com> writes:
Le 05/04/2012 16:35, Andrei Alexandrescu a écrit :
 On 4/4/12 10:53 AM, Steven Schveighoffer wrote:
 On Wed, 04 Apr 2012 12:33:26 -0400, Michel Fortin
 <michel.fortin michelf.com> wrote:
 Question 2: does std.algorithm.sort refer to the std.algorithm.sort
 *module* or to std.algorithm.package.sort *function* publicly imported
 from the std.algorithm.sort module?

The function. A symbol that is not specified as the module name in import statement or in a module statement is always *not* a module. I think our one saving grace here is that when you want to import a specific symbol from a module, this is not the syntax: import std.stdio.writefln; So there is never any ambiguity as to whether you mean a module identifier or other symbol.

Interesting. But isn't there an ambiguity when the symbol is not the last one in the chain? Consider: a.b.c.x Could be static member d of class c in module a.b, or module member d in module a.b.c. Andrei

The whole point of this thread is to import symbols from submodules into a module. Obviously, this tends to create collisions. This one is hard to solve and should probably be an error. However, such a collision will also appear with DIP16, and DIP16 cause also other type of collisions. So that one is superior - even if not perfect
Apr 05 2012
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/5/12 7:43 AM, Steven Schveighoffer wrote:
 I currently think DIP16 is invalid/worksforme (public imports allows
 splitting a module into a package). All that is left is how we could
 specifically import a package with one import statement.

Not entirely (I was aware of the way public import works). An issue does exist - there are "too many names", i.e. the alias pulled in the importing module and also the name being imported. This makes for odd synonyms such as std.algorithm_package.sort.sort being the same as std.algorithm.sort. The other issue is that obviously algorithm_package and algorithm must have distinct names, which makes the scheme a bit awkward at least until we define a compelling convention. I guess we can live with all that. Andrei
Apr 05 2012
next sibling parent deadalnix <deadalnix gmail.com> writes:
Le 05/04/2012 17:23, Andrei Alexandrescu a écrit :
 On 4/5/12 7:43 AM, Steven Schveighoffer wrote:
 I currently think DIP16 is invalid/worksforme (public imports allows
 splitting a module into a package). All that is left is how we could
 specifically import a package with one import statement.

Not entirely (I was aware of the way public import works). An issue does exist - there are "too many names", i.e. the alias pulled in the importing module and also the name being imported. This makes for odd synonyms such as std.algorithm_package.sort.sort being the same as std.algorithm.sort. The other issue is that obviously algorithm_package and algorithm must have distinct names, which makes the scheme a bit awkward at least until we define a compelling convention. I guess we can live with all that. Andrei

The first one isn't a problem. It isn't too many names, it is 2 names, and it happen when explicitly told to do so. The second is.
Apr 05 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, April 05, 2012 11:30:26 Steven Schveighoffer wrote:
 A couple issues that still need consideration:
 
 1. If std.algorithm the module becomes std.algorithm the package, what
 happens with ddoc? We probably *do* need a compiler solution to this.

That's assuming that you insist on keeping all of the documentation in one file. That arguably defeats the purpose of splitting up the modules. If there isn't enough in the module to split the documentation, then why do you need to split the module? What _would_ be valuable and the package.d could provide is an overview of the package. The ddoc comment for the package.d module can become the documentation for the package as a whole.
 2. deadalnix pointed out that if we come up with a scheme where the
 package module and its submodules are in the same directory, the package
 accessibility qualifier can be used (hey look, a use for the package
 keyword!).

Yes. std.datetime will need that once it's split. Without that, much of it can't be split and/or code would have to be needlessly duplicated. - Jonathan M Davis
Apr 05 2012
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 14:33:22 -0400, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:

 On Thursday, April 05, 2012 11:30:26 Steven Schveighoffer wrote:
 A couple issues that still need consideration:

 1. If std.algorithm the module becomes std.algorithm the package, what
 happens with ddoc? We probably *do* need a compiler solution to this.

That's assuming that you insist on keeping all of the documentation in one file. That arguably defeats the purpose of splitting up the modules. If there isn't enough in the module to split the documentation, then why do you need to split the module?

I thought the whole point was code maintenance? Not documentation splitting... I would have expected people to continue to treat std.algorithm like it was one module, even though it imports several sub-modules for its implementation.
 What _would_ be valuable and the package.d could provide is an overview  
 of the
 package. The ddoc comment for the package.d module can become the
 documentation for the package as a whole.

 2. deadalnix pointed out that if we come up with a scheme where the
 package module and its submodules are in the same directory, the package
 accessibility qualifier can be used (hey look, a use for the package
 keyword!).

Yes. std.datetime will need that once it's split. Without that, much of it can't be split and/or code would have to be needlessly duplicated.

Hm.. I just thought of something, as long as the main "package" module imports everything from the same directory, and doesn't define anything, this isn't an issue. -Steve
Apr 05 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 02 Apr 2012 20:44:09 -0400, Michel Fortin  
<michel.fortin michelf.com> wrote:

 On 2012-04-02 13:04:31 +0000, "Steven Schveighoffer"  
 <schveiguy yahoo.com> said:

 On Fri, 30 Mar 2012 17:45:36 -0400, Michel Fortin

 The problem is that if .std.algorithm.package
 contains a sort function and there is also a module called
 std.algorithm.sort, the fully-qualified name of that 'sort' module wil  
 l
 become ambiguous. Moreover, whether the fully-qualified name
 .std.algorithm.sort is ambiguous or not depends on what modules were
 imported, which is not a very desirable behaviour.

don't name a module sort inside std/algorithm. This is no different than ambiguous templates, which are allowed until y ou want to instantiate one.

If you have ambiguous templates in the same module, it'll always be ambiguous irrespective of what you import (and you can blame the module's designer for it). If you have ambiguous templates residing in different modules the symbol will be unambiguous until you've imported the second module (same as overloaded functions). At that point you can disambiguate using the fully-qualified name of the template (or function). Whereas if the fully-qualified name of a module becomes ambiguous because of a symbol in another module, there is no way to disambiguate. All you can do is avoid importing the two conflicting modules together, just like when you encounter two headers trying to define the same symbol in C/C++.

How does this happen? The FQN cannot be ambiguous. For example, if we change std/algorithm.d into: std/algorithm/sorting.d which defines sort std/algorithm/finding.d which defines find std/algorithm/package.d which imports sorting.d and finding.d Then how can importing some other module make the FQN ambiguous? The FQN for sort is std.algorithm.sorting.sort. Now, if you define the symbol sort inside std/algorithm/package.d, you have an ambiguity, but so what? If you are responsible for the std.algorithm module, aren't you responsible for all the files in that pseudo-package? I don't see how it changes things from today. Basically, my point is the only ambiguity FQN in the package.d file can create is with other symbols within that same directory, therefore within the same module. All we have to do is adopt the practice that phobos' package.d will only publicly import other modules. It won't define any symbols. Then there will always be an unambiguous FQN. Hm... maybe you mean if you can have std/algorithm/package.d do something like import std.file, which contains another sort function, is that now packaged under std.algorithm.sort? I don't think that's possible in the given DIP. -Steve
Apr 04 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 04 Apr 2012 12:33:26 -0400, Michel Fortin  
<michel.fortin michelf.com> wrote:

 On 2012-04-04 14:08:34 +0000, "Steven Schveighoffer"  
 <schveiguy yahoo.com> said:

  The FQN cannot be ambiguous.

Sure it can if I follow DIP16, because module names can become ambiguous. Let's try this with an example. First, let's define a pretty standard module: std/algorithm/sort.d: module std.algorithm.sort; void sort(T)(T[] array); Here the fully-qualified name of the sort function is .std.algorithm.sort.sort. But according to DIP16's lookup rules, the sort function is also available (if not ambiguous) at: std.sort std.algorithm.sort Question 1: since there is already a module at .std.algorithm.sort, doesn't the module name become ambiguous with the sort function it itself contains?

OK, but when is it ever valid to refer to a module when the semantic expectations are for something other than a module? I can only think of two places where module names are used, inside an import statement and inside a module statement (three if you count the prefix of a FQN). Maybe I'm missing some case...
 Let's assume the module's name take priority and does not conflict so we  
 can continue. Now we create the package.d file:

 std/algorithm/package.d:

 	import std.algorithm.sort;

 And now I write this somewhere in my code:

 	std.algorithm.sort

 Question 2: does std.algorithm.sort refer to the std.algorithm.sort  
 *module* or to std.algorithm.package.sort *function* publicly imported  
 from the std.algorithm.sort module?

The function. A symbol that is not specified as the module name in import statement or in a module statement is always *not* a module. I think our one saving grace here is that when you want to import a specific symbol from a module, this is not the syntax: import std.stdio.writefln; So there is never any ambiguity as to whether you mean a module identifier or other symbol.
 You might think I'm trying to split hair in four to find flaws, that no  
 one is going to do things that dumb, but I unfortunately think the  
 problematic pattern is already quite common. How many times have we seen  
 modules containing a class, variable, or function having the same name  
 as the module's name?

Tango anyone? :) But yes, I think the issue really becomes, we need to look at context when deciding the semantic meaning of a symbol. I don't think this violates the context-free grammar, because wouldn't this only come into play at the semantic level? Not a compiler writer/hacker, so I don't know.
 Say you wanted to create a package.d file directly for the whole package  
 std, what should be done for those?

No, let's not do that. Ever. :) -Steve
Apr 04 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 04 Apr 2012 14:03:07 -0400, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 04/04/2012 07:53 PM, Steven Schveighoffer wrote:
 OK, but when is it ever valid to refer to a module when the semantic
 expectations are for something other than a module? I can only think of
 two places where module names are used, inside an import statement and
 inside a module statement (three if you count the prefix of a FQN).
 Maybe I'm missing some case...

__traits(allMembers, pack.age.mod.ule);

hm... maybe we'd have to have new __traits that would disambiguate, like __traits(allModuleMembers...) This doesn't seem like a huge barrier.
 Tango anyone? :) But yes, I think the issue really becomes, we need to
 look at context when deciding the semantic meaning of a symbol. I don't
 think this violates the context-free grammar, because wouldn't this only
 come into play at the semantic level? Not a compiler writer/hacker, so I
 don't know.

No symbol is resolved until semantic, but I don't think hiding the module/package symbol if any clashing symbol in the module/any subpackage exists is a satisfactory solution.

Then we must come up with a way to hide the submodules of a virtual module. I tried this, which fails on the current compiler: import x; int x; So it seems we have two choices here: 1. DIP16 needs to get more complex to make package submodules not accesible as individual modules. 2. Start contextually interpreting identifiers at least in the case of modules vs. non-modules. I'd suggest option 2 allows for better backwards compatibility and more flexibility. -Steve
Apr 04 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 04 Apr 2012 20:50:49 -0400, Michel Fortin  =

<michel.fortin michelf.com> wrote:

 On 2012-04-04 19:48:32 +0000, "Steven Schveighoffer"  =

 <schveiguy yahoo.com> said:

 On Wed, 04 Apr 2012 14:03:07 -0400, Timon Gehr <timon.gehr gmx.ch>  =


 wrote:

 No symbol is resolved until semantic, but I don't think hiding the  =



 module/package symbol if any clashing symbol in the module/any   =



 subpackage exists is a satisfactory solution.



 module.
  I tried this, which fails on the current compiler:
  import x;
  int x;
  So it seems we have two choices here:
  1. DIP16 needs to get more complex to make package submodules not   =


 accesible as individual modules.
 2. Start contextually interpreting identifiers at least in the case o=


 modules vs. non-modules.
  I'd suggest option 2 allows for better backwards compatibility and  =


 more  flexibility.

I don't think option 2 is realistic (see my other post).

I saw your other post. I see that a major issue with the way DIP16 = intends to shortcut fully qualified names is demonstrated by this simple= = example: a/b/c.d: module a.b.c; void foo() {} struct c { static void foo() {} } main.d: import a.b.c; void main() { a.b.c.foo(); // cannot determine what this is. Is it the global = function, or a shortcut FQN to a.b.c.c.foo? } This should be counter-case enough to disqualify that idea -- it will = break existing code without ever adding any package.d files. Thanks for= = explaining.
 I don't think option 1 is an improvement over what we have. I mean, if=

 you're going to hide the submodules, what is the benefit compared to  =

 just using a different package name for the implementation modules?

The advantage is you can split your logical module into submodules for = maintenance purposes.
 You can already refactor std.algorithm this way with no change in the =

 compiler:

 	module std.algorithm;

 	public import std.algorithm_impl.sort;
 	public import std.algorithm_impl.map;
 	public import std.algorithm_impl.blah_blah_blah;
 	=E2=80=A6

 If we add a language feature, it should be an noticeable improvement  =

 over this situation.

I agree, option 1 is not very convincing.
 I think we need a third option.

 Here's an idea: we could allow modules having a single symbol with the=

 same name as the module to behave as if they were the symbol itself,  =

 just like templates behaves. For instance:

 	module std.algorithm.sort;

 	void sort(int[] t);

 Now you can import std.algorithm.sort and then use the  =

 std.algorithm.sort fully qualified name as if it was a function, even =

 though it's the module name (std.algorithm.sort.sort would be the  =

 function's name).

 Or maybe we could just allow "alias sort this" at module level? Or is =

 allowed already?

I don't like this proposal, simply because this means one function/symbo= l = per submodule. It should be more flexible than that. But I like the li= ne = of thinking. Let's re-examine the issue. We need to allow splitting of a module X.d = = into pieces for maintenance (or possibly accessibility -- disallowing = friends). But we don't want to break code which currently uses FQN to = access X's symbols. I think the following might work: algorithm.d: import this =3D std.algorithm_impl.sort; Which then imports std.algorithm_impl.sort, and effectively aliases all = = its symbols into algorithm.d. If std.algorithm_impl.sort defines a = function called sort, then it's also aliased to std.algorithm.sort. In = = essence, sort has *two* FQN, but there are no FQN that refer to more tha= n = one symbol. I purposely left out the package.d idea because it's orthogonal to this.= -Steve
Apr 05 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 08:49:24 -0400, deadalnix <deadalnix gmail.com> wrot=
e:

 Le 05/04/2012 13:46, Steven Schveighoffer a =C3=A9crit :
 I don't like this proposal, simply because this means one
 function/symbol per submodule. It should be more flexible than that. =


 I like the line of thinking.

 Let's re-examine the issue. We need to allow splitting of a module X.=


 into pieces for maintenance (or possibly accessibility -- disallowing=


 friends). But we don't want to break code which currently uses FQN to=


 access X's symbols.

 I think the following might work:

 algorithm.d:

 import this =3D std.algorithm_impl.sort;

 Which then imports std.algorithm_impl.sort, and effectively aliases a=


 its symbols into algorithm.d. If std.algorithm_impl.sort defines a
 function called sort, then it's also aliased to std.algorithm.sort. I=


 essence, sort has *two* FQN, but there are no FQN that refer to more
 than one symbol.

 I purposely left out the package.d idea because it's orthogonal to th=


 -Steve

The behavior you described has been proposed for public import, and ha=

 been discussed. This is interesting.

 You propose an alternative syntax with is fine and have the advantage =

 not disturb already existing public imports, but have the drawback to =

 create a new syntax, again.

 If such a syntax is adopted, what would be the point of public imports=

 If it is still useful, then your syntax is a better choice, otherwise,=

 we'd better modify public import.

No, public imports simply mean that you can view the publicly imported = module. It does *not* add aliases to the importing module. for example: foo.d: module foo; int x; bar.d: module bar; public import foo; int y; main.d: import bar; // publicly imports foo. void main() { foo.x =3D 5; // ok, publicly imported via bar (non public, this woul= d be = an error) bar.y =3D 5; // ok bar.x =3D 5; // error, public import doesn't create new fully qualif= ied = name for foo.x } With the system I propose, using the specific technique would make it so= = bar.x would also be valid, and would refer to foo.x I think we need new syntax, or a new language feature, because you don't= = want to alter the operation of existing code. Simply changing public = imports would cause lots of existing code to be reinterpreted, possibly = in = a way not intended, or that would be ambiguous. -Steve
Apr 05 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 08:49:24 -0400, deadalnix <deadalnix gmail.com> wrote:

 If such a syntax is adopted, what would be the point of public imports ?  
 If it is still useful, then your syntax is a better choice, otherwise,  
 we'd better modify public import.

Another way to look at it: A public import saves you from having to import dependent modules that you should be importing anyway. A 'this' import treats the other modules as if they were actually part of the imported module in terms of namespace. To give an example, std.range imports std.array. std.array is a module all on its own, and has a specific set of functionality. std.range has a different set of functionality, but you would never want to have std.range imported without also importing std.array. Another example would be a derived class module publicly importing the base class module. On the other hand, something like std.container could have 15 different container types in it. Each of these container types should really live in their own module, in terms of maintenance and separation of private data. For example, std.container.RedBlackTree has no business accessing the private members of std.container.Array. But because they live in the same module, it can. However, we don't want to define std.container.RedBlackTree.RedBlackTree in terms of namespace. So we have one of these combining modules, and everything still lives in the std.container namespace, but we get all the benefits of separating the code into individual modules. -Steve
Apr 05 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 09:23:25 -0400, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 04/05/2012 02:58 PM, Steven Schveighoffer wrote:
 No, public imports simply mean that you can view the publicly imported
 module. It does *not* add aliases to the importing module.

Have you tried it?

I just did. OK, what the hell are we arguing about then?! DIP16 is worksforme :) See this part of the spec: http://dlang.org/module.html Read the part on public modules. You may understand why I didn't know about that "feature" (which seems to work on all the installed compilers I have, back to 2.033). I just read the public import part of TDPL, and there it is, all spelled out quite nicely. I'm going to file a bug against the spec... grrr... I'm now firmly in the "we don't need to change anything, just refactor the modules and use public import" camp. We don't even need to change ANYTHING in the compiler! Forget everything I said before in this thread ;) I suppose the only thing we don't get is being able to have a module and a package with the same FQN. I don't see that being a major issue. -Steve
Apr 05 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 10:26:16 -0400, deadalnix <deadalnix gmail.com> wrot=
e:

 Le 05/04/2012 14:58, Steven Schveighoffer a =C3=A9crit :
 On Thu, 05 Apr 2012 08:49:24 -0400, deadalnix <deadalnix gmail.com>  =


 wrote:

 Le 05/04/2012 13:46, Steven Schveighoffer a =C3=A9crit :



 module. It does *not* add aliases to the importing module.


[snip]
 I know. I think you answered too fast.

 I wasn't stating that public import are the same as what you propose. =

 was discussing the fact that both probably answer the same need and th=

 2 different syntax is something to avoid.

 Code wouldn't be broken in unexpected way, because thing will cause  =

 either a compile error (collision of names, but the problem exists wit=

 other solutions too) or is illegal in the current shape of the languag=

My assertion above is 100% *WRONG*. See this bug I just filed. = http://d.puremagic.com/issues/show_bug.cgi?id=3D7830 In fact public imports today are *exactly* the same as what I proposed. = I = had no idea. I think we are (well, at least I am) on a wild goose chase here. All th= at = is left to discuss is if/how we want to allow importing of packages. But I think DIP15 already covers that = http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP15 I think DIP16 is worksforme/invalid, and should be closed. -Steve
Apr 05 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 10:35:52 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 On 4/4/12 10:53 AM, Steven Schveighoffer wrote:
 On Wed, 04 Apr 2012 12:33:26 -0400, Michel Fortin
 <michel.fortin michelf.com> wrote:
 Question 2: does std.algorithm.sort refer to the std.algorithm.sort
 *module* or to std.algorithm.package.sort *function* publicly imported
 from the std.algorithm.sort module?

The function. A symbol that is not specified as the module name in import statement or in a module statement is always *not* a module. I think our one saving grace here is that when you want to import a specific symbol from a module, this is not the syntax: import std.stdio.writefln; So there is never any ambiguity as to whether you mean a module identifier or other symbol.

Interesting. But isn't there an ambiguity when the symbol is not the last one in the chain? Consider: a.b.c.x Could be static member d of class c in module a.b, or module member d in module a.b.c.

Stop reading all my posts except for the ones I just posted. I was completely wrong in what I thought the compiler couldn't do, because the spec is lacking. I currently think DIP16 is invalid/worksforme (public imports allows splitting a module into a package). All that is left is how we could specifically import a package with one import statement. DIP15 covers that. But I don't even think we need that, we can fix Phobos without any compiler changes. -Steve
Apr 05 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 10:33:14 -0400, Michel Fortin  
<michel.fortin michelf.com> wrote:

 On 2012-04-05 11:46:38 +0000, "Steven Schveighoffer"  
 <schveiguy yahoo.com> said:

 I don't like this proposal, simply because this means one  
 function/symbo l
 per submodule.  It should be more flexible than that.  But I like the  
 li ne
 of thinking.

I have the same reserve about it's lack of flexibility too. It would be a nice thing to have for all those libraries having one module per class as it'd reduce the length of the fully qualified name you have to use when you need to disambiguate.

I see the point you are going for. I don't really like the one-class-per-module style as much as how phobos has one module per category. But that's not an excuse not to examine this possibility. I think it's a different issue than what DIP16/DIP15 is trying to solve.
 But for moving a module like std.algorithm to a package, it's cumbersome.

Not really. Just move the files to another directory (i.e. std/internal/algorithm) and publicly import them.
 Let's re-examine the issue.  We need to allow splitting of a module X.d
 into pieces for maintenance (or possibly accessibility -- disallowing
 friends).  But we don't want to break code which currently uses FQN to
 access X's symbols.
  I think the following might work:
  algorithm.d:
  import this = std.algorithm_impl.sort;
  Which then imports std.algorithm_impl.sort, and effectively aliases all
 its symbols into algorithm.d.  If std.algorithm_impl.sort defines a
 function called sort, then it's also aliased to std.algorithm.sort.  In
 essence, sort has *two* FQN, but there are no FQN that refer to more  
 tha n
 one symbol.

This is what a public import should do.

I wasn't aware of this. Mostly because the spec omits that feature. I have to retract almost everything I've argued, because we already have the means to fix what I think DIP16 is trying to solve without any compiler changes. -Steve
Apr 05 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 10:53:34 -0400, deadalnix <deadalnix gmail.com> wrot=
e:

 Le 05/04/2012 16:47, Steven Schveighoffer a =C3=A9crit :
 But for moving a module like std.algorithm to a package, it's  =



 cumbersome.

Not really. Just move the files to another directory (i.e. std/internal/algorithm) and publicly import them.

std/internal isn't good. If I just want to import sort, I would have t=

 do std.internal.algorithm.sort, which isn't good.

import std.algorithm : sort; BTW, this doesn't work today, I'll file a bug.
 Plus, the package accessibility modifier would be broken with such a  =

 pattern.

 The all.d, package.d, _.d, or, as I propose foldername.d (and submodul=

 into foldername) do not break package accessibility modifier, which is=

 better.

I don't really know what you mean, I didn't read that post. Would you = mind posting a link or repeating that argument? -Steve
Apr 05 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 10:59:31 -0400, Steven Schveighoffer  =

<schveiguy yahoo.com> wrote:

 On Thu, 05 Apr 2012 10:53:34 -0400, deadalnix <deadalnix gmail.com>  =

 wrote:

 Le 05/04/2012 16:47, Steven Schveighoffer a =C3=A9crit :
 Not really. Just move the files to another directory (i.e.
 std/internal/algorithm) and publicly import them.

std/internal isn't good. If I just want to import sort, I would have =


 do std.internal.algorithm.sort, which isn't good.

import std.algorithm : sort; BTW, this doesn't work today, I'll file a bug.

Nevermind, it works the same as if sort is defined in a std.algorithm = module. I'm kind of surprised the FQN doesn't work though... -Steve
Apr 05 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 10:59:31 -0400, Steven Schveighoffer  
<schveiguy yahoo.com> wrote:

 On Thu, 05 Apr 2012 10:53:34 -0400, deadalnix <deadalnix gmail.com>  
 wrote:
 Plus, the package accessibility modifier would be broken with such a  
 pattern.

 The all.d, package.d, _.d, or, as I propose foldername.d (and  
 submodules into foldername) do not break package accessibility  
 modifier, which is better.

I don't really know what you mean, I didn't read that post. Would you mind posting a link or repeating that argument?

Also, nevermind :) I realize what you are talking about now. This is a good point. Of course, there are other mechanisms to do this, and DIP15 already proposes an improvement for this. However, at least in the case of phobos, I don't think there's a lot of instances of package accessibility modifier. -Steve
Apr 05 2012
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 11:23:12 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 On 4/5/12 7:43 AM, Steven Schveighoffer wrote:
 I currently think DIP16 is invalid/worksforme (public imports allows
 splitting a module into a package). All that is left is how we could
 specifically import a package with one import statement.

Not entirely (I was aware of the way public import works). An issue does exist - there are "too many names", i.e. the alias pulled in the importing module and also the name being imported. This makes for odd synonyms such as std.algorithm_package.sort.sort being the same as std.algorithm.sort.

Right, but if one only ever imports std.algorithm, who cares what the submodule FQNs are? AIUI, DIP16 also doesn't really reduce the number of names either.
 The other issue is that obviously algorithm_package and algorithm must  
 have distinct names, which makes the scheme a bit awkward at least until  
 we define a compelling convention. I guess we can live with all that.

Agreed. I think Don mentioned std.math already does something, we may want to look at that model. A couple issues that still need consideration: 1. If std.algorithm the module becomes std.algorithm the package, what happens with ddoc? We probably *do* need a compiler solution to this. 2. deadalnix pointed out that if we come up with a scheme where the package module and its submodules are in the same directory, the package accessibility qualifier can be used (hey look, a use for the package keyword!). -Steve
Apr 05 2012
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Mar 30, 2012 at 05:35:25PM -0400, Jonathan M Davis wrote:
[...]
 But personally, I like the idea of making it so that publicly imported
 symbols can be accessed as if they were in the module that publicly
 imported them (with package.d being treated as if it had the same name
 as the package that it's in). That's essentially how it already works
 except when specifying the full import path for a symbol. And that
 way, you can specify in std.algorithm.package.d exactly what you want
 to be imported when std.algorithm is imported (including using : to
 restrict it to specific symbols in a module), and only those symbols
 will be treated as if they were part of std.algorithm - both for
 importing purposes and when specifying the import path when using a
 symbol. The library maintainer then has control over which symbols get
 used with which import paths.

+1. I also have my doubts about the wisdom of std.sort implicitly binding to std.algorithm.sort. It's nice syntactic sugar in the short term, but as Jonathan points out, it can cause headaches in the long term. I say we should hold off on that part of the proposal. T -- Microsoft is to operating systems & security ... what McDonalds is to gourmet cooking.
Mar 30 2012
prev sibling next sibling parent Ary Manzana <ary esperanto.org.ar> writes:
On 3/30/12 10:46 PM, Andrei Alexandrescu wrote:
 Starting a new thread from one in announce:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

 Please comment, after which Walter will approve. Walter's approval means
 that he would approve a pull request implementing DIP16 (subject to
 regular correctness checks).

Great. Large modules are my main complaint about D. :-) If I correctly understand the second part (because I couldn't understand the text in the proposal until I read some comments here), then it makes sense. Is it like this? sort(...) -> search sort in every module out there std.sort(...) -> search sort in every module that's in the std package If both std.algorithm.sort and std.path.sort exist, or something like that, then you would anyway get a clash so you'd have to fully qualify it. But if std.algorithm.sort and foo.bar.sort and you'd import both: import std.algorithm.package; import foo.bar.package; and you'd wanted to use both, then it could be convenient: std.sort(...) foo.sort(...) Though I wonder if this indeed happens a lot. That's why I would wait until there's a real need for it. The main complaint people have is not having a way to import all files in a directory, which is the first point, but I never heard a complaint about the second point. Also, I think it would make sense to change the first part to this: * If the compiler sees a request for importing "foo.bar" and "foo/bar" is a directory, then automatically look for the file "foo/bar/package.d". *If it doesn't exist, automatically expand the import to import all files under that directory.* If both "foo/bar.d" and "foo/bar/" exists, compilation halts with an error. That way you have convenience and safety. Most of the time people just put in package.d a list of all the files in that directory. Maybe sometimes (not sure) people restrict that list to some modules. And in those cases you can just restrict the list in package.d Please, it's the year 2012. Compilers need to be smarter. Save people some typing time. You save them typing all the imports. But then you make them typing them in that pacakge.d file. Hmm...
Mar 30 2012
prev sibling next sibling parent reply "Chris NS" <ibisbasenji gmail.com> writes:
I'm pretty impressed with the idea, and look forward to its 
implementation, but I do have one question.  How does this affect 
(if at all) the implicit "friend" relationship of declarations?  
Since package "foo.bar" is treated as a single module, do the 
classes in "foo/bar/alpha.d" and "foo/bar/beta.d" have access to 
each other's private members?

I'm not sure whether I favor them losing or keeping their 
"friend" status.

----------
Chris NS
Mar 30 2012
parent "Nick Sabalausky" <a a.a> writes:
"Chris NS" <ibisbasenji gmail.com> wrote in message 
news:ugopmohijjcnnrchuwbe forum.dlang.org...
 I'm pretty impressed with the idea, and look forward to its 
 implementation, but I do have one question.  How does this affect (if at 
 all) the implicit "friend" relationship of declarations?  Since package 
 "foo.bar" is treated as a single module, do the classes in 
 "foo/bar/alpha.d" and "foo/bar/beta.d" have access to each other's private 
 members?

 I'm not sure whether I favor them losing or keeping their "friend" status.

There's always the "package" access specifier.
Mar 30 2012
prev sibling next sibling parent "Chris NS" <ibisbasenji gmail.com> writes:
On Saturday, 31 March 2012 at 06:39:07 UTC, Nick Sabalausky wrote:
 "Chris NS" <ibisbasenji gmail.com> wrote in message
 news:ugopmohijjcnnrchuwbe forum.dlang.org...
 I'm pretty impressed with the idea, and look forward to its 
 implementation, but I do have one question.  How does this 
 affect (if at all) the implicit "friend" relationship of 
 declarations?  Since package "foo.bar" is treated as a single 
 module, do the classes in "foo/bar/alpha.d" and 
 "foo/bar/beta.d" have access to each other's private members?

 I'm not sure whether I favor them losing or keeping their 
 "friend" status.

There's always the "package" access specifier.

True, though it bears revisiting in its own right. I've never been completely satisfied with the horizontal visibility, and would have preferred 'package' be defined as "visible within current package, *and subpackages* of the current." But now I'm getting a bit off-topic.
Mar 30 2012
prev sibling next sibling parent Derek <ddparnell bigpond.com> writes:
On Sat, 31 Mar 2012 01:46:19 +1100, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 Starting a new thread from one in announce:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

 Please comment ...

We solved this issue in the Euphoria Programming language by introducing the concept of 'public include'. The 'include' directive is similar to DPL's 'import' directive. Normally when a file (module) is included, its public identifiers are *only* visible to the including module and not further module up the include tree. However, if a module is 'public include'd then its public identifiers are set up *as if* they were actually declared in the including module rather than the included module. Thus if Euphoria's module called datetime.e gets too large, we split it into, say two or three new modules and replace the code in datetime.e with corresponding 'public include' statements, thus making it a type of package definition. This means that existing application code does not have to be modified and future code can choose to include either the entire package called datetime.e or individual modules that go into making that package. From the POV of a developer, creating a package out of what was a module is transparent. They do not even know or have to care in order to use it. There are no complicated access rules that can one day trip people up. We made one tweak to the public identifier declaration though to cater for situations in which an identifier which was currently local to a module, but when moved to a new module when the original was transformed to a package, still had to be visible to the code in the package file but invisible to code including the package. We created the 'export' qualifier on an identifier declaration to signal such an item. Original module code ... function to_seconds(datetime x) /* A local function to the module */ public function local_to_gmt(datetime x) /* A publicly exposed function */ New module code ... export function to_seconds(datetime x) /* A function visible to the package file only */ public function local_to_gmt(datetime x) /* A publicly exposed function */ This means that application that include the package datetime.e will not see to_seconds() but code in the datetime.e will see it, and local_to_gmt() will be seen by the application code and the package code. -- Derek Parnell Melbourne, Australia
Mar 31 2012
prev sibling next sibling parent "foobar" <foo bar.com> writes:
On Friday, 30 March 2012 at 14:46:16 UTC, Andrei Alexandrescu 
wrote:
 Starting a new thread from one in announce:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

 Please comment, after which Walter will approve. Walter's 
 approval means that he would approve a pull request 
 implementing DIP16 (subject to regular correctness checks).


 Destroy!

 Andrei

Hooray! I was loudly complaining about this issue for years. I'm glad this is finally going to be taken care of. 1. Regarding naming - as others mentioned it's inconvenient for sorting. How about changing the file extension instead of the name? e.g have a "algorithm.package" instead of "algorithm.d"? The compiler could even ignore the name of the file as long as it has the proper extension - check for existence of "*.package". The compiler should enforce that there's at most one such package file to prevent ambiguities - regardless of the chosen naming scheme. 2. Regarding documentation - I'd encourage (via a D style guideline) putting the overview of the package in this file. A tool such as Ruby's codnar would nicely compliment the reference generated by DDoc. see https://www.ruby-toolbox.com/gems/codnar for details. 3. std.algorithm should be deprecated and eventually removed. All code *is* essentially algorithms and as such importing this module makes as much sense as doing: import code; // generic much? 4. regarding the look-up rules - I'm unsure about this part of the proposal. Others already voiced concerns about issues this might bring.
Mar 31 2012
prev sibling next sibling parent reply "Martin Nowak" <dawg dawgfoto.de> writes:
On Sat, 31 Mar 2012 13:06:36 +0200, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 03/30/2012 11:35 PM, Jonathan M Davis wrote:
 But personally, I like the idea of making it so that publicly imported  
 symbols
 can be accessed as if they were in the module that publicly imported  
 them

+1. That is even better than treating the package module specially.

That already works, doesn't it?
Mar 31 2012
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday, March 31, 2012 17:30:36 Timon Gehr wrote:
 On 03/31/2012 03:49 PM, Martin Nowak wrote:
 On Sat, 31 Mar 2012 13:06:36 +0200, Timon Gehr <timon.gehr gmx.ch> wrote:
 On 03/30/2012 11:35 PM, Jonathan M Davis wrote:
 But personally, I like the idea of making it so that publicly
 imported symbols
 can be accessed as if they were in the module that publicly imported
 them

+1. That is even better than treating the package module specially.

That already works, doesn't it?

It already works indeed. My bad.

Does it? I thought that std.range.replace wouldn't work (even though std.range publicly imports std.array), because replace isn't part of std.range. You don't need to import std.array if you import std.range, because std.range does, but you can't specificy it's path as if it were in std.range. I'll have to check... Okay. You're right. std.range.replace _does_ work. So then the only issue is making it so that public imports in std.datetime.package are treated as if they were in std.datetime. That seems like the cleanest solution to me. It goes along quite nicely with making it so that anything in std.datetime.package is imported when importing std.datetime. - Jonathan M Davis
Mar 31 2012
prev sibling next sibling parent "Martin Nowak" <dawg dawgfoto.de> writes:
 However, I'm very nervous about the second part. e.g. std.sort instead of
 std.algorithm.sort seems like a bad idea to me. It increases the odds of  
 name
 conflicts for little benefit. Not to mention, it'll make it a lot more  
 confusing
 to find what modules stuff is actually in if people start doing stuff  
 like

 std.sort(arr);

You can turn that argument around, std.sort(arr) instead of sort(arr) provides some extra context for disambiguation. You'd be able to do that for public imports in the package.d module anyhow, so it makes sense to support it for implicit packages as well.
Mar 31 2012
prev sibling next sibling parent deadalnix <deadalnix gmail.com> writes:
Le 30/03/2012 16:46, Andrei Alexandrescu a écrit :
 Starting a new thread from one in announce:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

 Please comment, after which Walter will approve. Walter's approval means
 that he would approve a pull request implementing DIP16 (subject to
 regular correctness checks).


 Destroy!

 Andrei

OK, here is an alternative proposal, based on what has been said here, and in other thread. I think it is a better alternative because it is almost already working. 1/ you can defined both a.d (d source code) and a (folder) in the same folder. 2/ import a; will always look for a.d . a folder. a existing isn't an error. 3/ import a.b will look for b.d in a folder. If a.d exists, it isn't an error. 4/ The package a include what in a.d and everything defined in d files in the folder a and its subfolders. 5/ What is publicly imported get automatically aliased in the importing module. See example below if it is not clear : a.d : public import a.b; b.d : void foo() {} automatic aliasing in a.d would look like : public import a.b; // Automaticaly added aliases : alias b.foo foo; And usage from third party code : import a; void main() { foo(); // OK a.foo(); // OK a.b.foo(); // OK } And with static import : static import a; void main() { foo(); // Error a.foo(); // OK a.b.foo(); // OK }
Mar 31 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Sat, 31 Mar 2012 00:23:32 -0400, Chris NS <ibisbasenji gmail.com> wrote:

 I'm pretty impressed with the idea, and look forward to its  
 implementation, but I do have one question.  How does this affect (if at  
 all) the implicit "friend" relationship of declarations?  Since package  
 "foo.bar" is treated as a single module, do the classes in  
 "foo/bar/alpha.d" and "foo/bar/beta.d" have access to each other's  
 private members?

 I'm not sure whether I favor them losing or keeping their "friend"  
 status.

They would lose their friend status. Classes that you need to be "friends" would have to go into the same submodule, which makes perfect sense to me. -Steve
Apr 02 2012
prev sibling next sibling parent reply "Martin Nowak" <dawg dawgfoto.de> writes:
On Fri, 30 Mar 2012 16:46:19 +0200, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 Starting a new thread from one in announce:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

 Please comment, after which Walter will approve. Walter's approval means  
 that he would approve a pull request implementing DIP16 (subject to  
 regular correctness checks).


 Destroy!

 Andrei

What about supporting package initalization? I basically proposed that if a submodule of a package was imported, a static import of the package is implicitly added. http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP15
Apr 03 2012
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
Le 03/04/2012 19:44, Martin Nowak a écrit :
 On Fri, 30 Mar 2012 16:46:19 +0200, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 Starting a new thread from one in announce:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

 Please comment, after which Walter will approve. Walter's approval
 means that he would approve a pull request implementing DIP16 (subject
 to regular correctness checks).


 Destroy!

 Andrei

What about supporting package initalization? I basically proposed that if a submodule of a package was imported, a static import of the package is implicitly added. http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP15

That is pretty much what was proposed by people in this thread. Basically, it boils down to adding automatically aliases with public imports. I would prefer use pkg.d file instead of pkg/_.d (I though of it a lot recently, and this is what make more sense, even if not my initial proposal). With pkg/package.d or pkg/_.d you can ends up with unnecessary complexity in choosing the file wich is imported, and create error cases. For instance : What happen if both pkg.d and pkg/_.d exists ? If it is not in the same path (think -I compiler option). In one case, this is an issue, in the other this isn't. This file convention solution is superior to the _.d one or package.d one. DIP15 is superior to D16 IMO.
Apr 04 2012
parent deadalnix <deadalnix gmail.com> writes:
Le 05/04/2012 02:55, Martin Nowak a écrit :
 What happen if both pkg.d and pkg/_.d exists ? If it is not in the
 same path (think -I compiler option). In one case, this is an issue,
 in the other this isn't.

We'd need to directly search for a subdirectory to decide whether it's a package.

This is true, but why would it be a conflict ? Package already is a tree structure, so it shouldn't cause much trouble.
 Maybe someone with experience of big Python projects has some valuable
 insights, but I think one point of adding the file inside a folder is to
 prevent from unintended pickup of folders.

Using eponymous trick, you'd always pick up a file, never a folder. Hence, lookup rules get easier. Plus it is D-ish. I have no doubt that this approach is working in python, and could work in D. This is exactly why this was my first proposal. But this doesn't means that we can't do better. I have tested quite a lot of way to achieve that, in several languages, and this is usually a messy topic, with no single solution.
Apr 05 2012
prev sibling parent "Martin Nowak" <dawg dawgfoto.de> writes:
 What happen if both pkg.d and pkg/_.d exists ? If it is not in the same  
 path (think -I compiler option). In one case, this is an issue, in the  
 other this isn't.

We'd need to directly search for a subdirectory to decide whether it's a package. Maybe someone with experience of big Python projects has some valuable insights, but I think one point of adding the file inside a folder is to prevent from unintended pickup of folders.
Apr 04 2012
prev sibling next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 30 Mar 2012 10:46:19 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 Starting a new thread from one in announce:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

 Please comment, after which Walter will approve. Walter's approval means  
 that he would approve a pull request implementing DIP16 (subject to  
 regular correctness checks).


 Destroy!

BTW, this case makes the part of DIP16 which wants to shortcut fully qualified names invalid, or at least costly (I posted this code in another part of the thread, but I thought I'd bring it up higher). The following is valid code today: a/b.d: module a.b; void foo() {} struct b { static void foo() {} } main.d: import a.b; void main() { a.b.foo(); } If DIP16 were to be implemented, this becomes ambiguous. Is a.b.foo() the module function foo() from a.b, or is it a shortcut for a.b.b.foo()? The main issue is, because you can shortcut the FQN, and a chain of identifiers can have repeated identifiers in them, ambiguity is possible. Michel Fortin pointed out at least three cases of this in phobos (although I can't see how they could be ambiguous, he has a good point). -Steve
Apr 05 2012
parent Ary Manzana <ary esperanto.org.ar> writes:
On 4/5/12 10:55 PM, Steven Schveighoffer wrote:
 On Fri, 30 Mar 2012 10:46:19 -0400, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 Starting a new thread from one in announce:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

 Please comment, after which Walter will approve. Walter's approval
 means that he would approve a pull request implementing DIP16 (subject
 to regular correctness checks).


 Destroy!

BTW, this case makes the part of DIP16 which wants to shortcut fully qualified names invalid, or at least costly (I posted this code in another part of the thread, but I thought I'd bring it up higher). The following is valid code today: a/b.d: module a.b; void foo() {} struct b { static void foo() {} } main.d: import a.b; void main() { a.b.foo(); } If DIP16 were to be implemented, this becomes ambiguous. Is a.b.foo() the module function foo() from a.b, or is it a shortcut for a.b.b.foo()?

It's a shortcut to the module function foo(). If I replace main.d with: void main() { foo(); } Is foo() the module function foo() from a.b, or is it a shortcut for a.b.b.foo()? Here you have no doubts. What your mind does it: find a top level symbol "foo" in all imported modules. If you find more than one, error. You must apply the same logic for "a.b.foo()". First you search "foo" in the "a.b" symbol. Here you find it: it's the top level function "foo" in "a.b". Then you stop searching. However, if you can't find it in the module "a.b", you search a top level symbol "foo" in all modules that are in package "a.b". That's it. You don't search "foo" in every possible nesting: just module nesting.
 The main issue is, because you can shortcut the FQN, and a chain of
 identifiers can have repeated identifiers in them, ambiguity is possible.

As I said before, it's not a shortcut of the FQN: it's just a shortcut for the module name.
Apr 05 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, April 05, 2012 15:30:17 Steven Schveighoffer wrote:
 On Thu, 05 Apr 2012 14:24:10 -0400, Jonathan M Davis <jmdavisProg gmx.com>
 
 wrote:
 On Thursday, April 05, 2012 09:49:59 Steven Schveighoffer wrote:
 On Thu, 05 Apr 2012 09:23:25 -0400, Timon Gehr <timon.gehr gmx.ch> I
 suppose the only thing we don't get is being able to have a module and a
 package with the same FQN. I don't see that being a major issue.

What doesn't work is being able to turn a module into a package with the same name. Right now, we could create a std.alg package with sub-modules containing all of std.algorithm's functionality and change std.algorithm to pubicly import them all, but you can't turn std.algorithm itself into a package without breaking code.

But so what? nobody has any code like: import std.algorithm.sort; So who cares where that module goes? I agree it would be ideal to put it there, but I don't think it's strictly necessary. And there is no need for the shortcut for fully qualified names.
 So, the whole point of this proposal - to seemlessly allow the
 transition of a
 module to a package in place - _does_ require a language/compiler change.

I don't see how. Just move the code into another module and publicly import that module from std/algorithm.d. Problem pretty much solved.

The issue is code organization. If you want to split up std.algorithm (or std.datetime or whatever) into multiple modules, you have to create a new package with a completely different name with no connection to the original save for the fact that the original publicly imports it. For instance, with the work that I've done thus far on splitting std.datetime, I've had to create a std.dtime package to hold the modules and have std.datetime pubicly import them. This automatically creates the issue of what the difference between them is (for anyone new to Phobos) and does not indicate their relation at all in the hierarchy. It would be much cleaner to be able to turn std/datetime.d into std/datetime/ with a package.d in it along with the new modules. No, we don't _have_ to do something to make it so that std.algorithm can be turned into std/algorithm/ while still not breaking code. But it would be very nice. Certainly, I don't understand why this DIP would ever have been proposed if Andrei didn't find it valuable.
 BTW, importing a directory was already proposed in DIP15.
 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP15

Yes, but having a package.d with the public imports gives you much finer- grained control over what gets imported, and DIP15 doesn't solve the problem of fully qualified uses of std.alorgithm.sort not breaking when std.algorithm.sort gets moved to something like std.algorithm.sorting.d. So, DIP15 doesn't work as a means of seemlessly breaking up a module. It just _mostly_ works as one (since people usually don't fully qualify symbols). Also, package.d would give us a means for documenting a package, which I would very much like to be able to do. Having std.datetime give an overview of the std.dtime package is definitely worse than having a means of having the package document itself - which the package.d file should be able to give us. The package.d portion of DIP16 allows a means of controlling what importing a package would mean, it provides a means of turning a module into package in place, and it potentially provides a way of documenting a package - all of which are valuable. Whether they're valuable enough to merit a language change is obviously up for debate, but certainly, as the designer and primary maintainer of one of the main targets for being split up, I very much like the idea of being able to split up a module in place rather than having to create a new package with a new name with no obvious relation to the original. - Jonathan M Davis
Apr 05 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, April 05, 2012 15:26:14 Steven Schveighoffer wrote:
 On Thu, 05 Apr 2012 14:33:22 -0400, Jonathan M Davis <jmdavisProg gmx.com>
 
 wrote:
 On Thursday, April 05, 2012 11:30:26 Steven Schveighoffer wrote:
 A couple issues that still need consideration:
 
 1. If std.algorithm the module becomes std.algorithm the package, what
 happens with ddoc? We probably *do* need a compiler solution to this.

That's assuming that you insist on keeping all of the documentation in one file. That arguably defeats the purpose of splitting up the modules. If there isn't enough in the module to split the documentation, then why do you need to split the module?

I thought the whole point was code maintenance? Not documentation splitting... I would have expected people to continue to treat std.algorithm like it was one module, even though it imports several sub-modules for its implementation.

If the module isn't large enough to be split for documentation, I find it hard to believe that it needs to be split for maintenance. And if all you care about is sub-modules for implementation and want all of the functions in the same module still, then this DIP is pointless. All you have to do is declare undocumented sub-modules which hold the various implementations and have the actual module call them. We already do this sort of thing in Phobos to get around static destructors screaming about circular dependencies. I only see the need to split of a module as the DIP suggests when it's too big, and if the documentation isn't large enough to be an issue, then I don't see how the module itself is going to be too large unless it has a ton of helper stuff, and that stuff can be seemlessy put in another, undocumented module without needing to create a package. - Jonathan M Davis
Apr 05 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 17:00:56 -0400, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:

 On Thursday, April 05, 2012 15:30:17 Steven Schveighoffer wrote:

 I don't see how. Just move the code into another module and publicly
 import that module from std/algorithm.d. Problem pretty much solved.

The issue is code organization. If you want to split up std.algorithm (or std.datetime or whatever) into multiple modules, you have to create a new package with a completely different name with no connection to the original save for the fact that the original publicly imports it.

My view is that people will not import the smaller modules, they will only ever import std.algorithm. Look at std.bigint, which imports modules from std.internal.math. Nobody ever imports std.internal.math.??? because they just import std.bigint.
 BTW, importing a directory was already proposed in DIP15.
 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP15

Yes, but having a package.d with the public imports gives you much finer- grained control over what gets imported, and DIP15 doesn't solve the problem of fully qualified uses of std.alorgithm.sort not breaking when std.algorithm.sort gets moved to something like std.algorithm.sorting.d.

I think you are forgetting that all current code imports std.algorithm, which will register the symbol std.algorithm.sort via public import. No code will break, even if it uses FQN. The only way to "break" code is to write new import statements. I don't think anyone will do that unknowingly. -Steve
Apr 05 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 17:02:13 -0400, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:

 On Thursday, April 05, 2012 15:26:14 Steven Schveighoffer wrote:
 On Thu, 05 Apr 2012 14:33:22 -0400, Jonathan M Davis  
 <jmdavisProg gmx.com>

 wrote:
 On Thursday, April 05, 2012 11:30:26 Steven Schveighoffer wrote:
 A couple issues that still need consideration:

 1. If std.algorithm the module becomes std.algorithm the package,  


 happens with ddoc? We probably *do* need a compiler solution to this.

That's assuming that you insist on keeping all of the documentation in one file. That arguably defeats the purpose of splitting up the modules.

 there
 isn't enough in the module to split the documentation, then why do you
 need to
 split the module?

I thought the whole point was code maintenance? Not documentation splitting... I would have expected people to continue to treat std.algorithm like it was one module, even though it imports several sub-modules for its implementation.

If the module isn't large enough to be split for documentation, I find it hard to believe that it needs to be split for maintenance.

Why do we ever need to split modules for documentation? Just fix the doc generator so it's not as monolithic. For instance, have one page per class or struct.
 And if all you care
 about is sub-modules for implementation and want all of the functions in  
 the
 same module still, then this DIP is pointless. All you have to do is  
 declare
 undocumented sub-modules which hold the various implementations and have  
 the
 actual module call them. We already do this sort of thing in Phobos to  
 get
 around static destructors screaming about circular dependencies.

You are starting to see my point :) But I think the issue is not so much that you are splitting the implementation, but splitting up the API into related modules. For example, std.container. Imagine we have a robust set of 15 containers. Why should those all be in one file? They have nothing to do with eachother except they are in the same namespace. Why does RedBlackTree need to be able to access the internals of Array? As the writer of RedBlackTree, I want to be able to test and develop my container without having to worry about the rest of std.container. But I also would like to have the FQN of it to be std.container.RedBlackTree. Public imports allow this *today* without any changes. Note that size isn't so much an issue for me as being able to compartmentalize and develop individual pieces of a module. -Steve
Apr 05 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, April 05, 2012 17:33:50 Steven Schveighoffer wrote:
 On Thu, 05 Apr 2012 17:02:13 -0400, Jonathan M Davis <jmdavisProg gmx.com>
 If the module isn't large enough to be split for documentation, I find
 it hard
 to believe that it needs to be split for maintenance.

Why do we ever need to split modules for documentation? Just fix the doc generator so it's not as monolithic. For instance, have one page per class or struct.

That may or may not be desirable (certainly in the case of smaller types, I'd argue that it isn't). By doing it on a module basis, you have far more control. But regardless, that would be a major change to ddoc. In either case, the size of the documentation page for a module is currently closely tied to the number of public symbols in the module, so if you have a large API, it can become desirable to split it up simply because the documentation page for it is too large. And by splitting up the API, you fix that problem. Not to mention, if the module is mostly free functions, putting the documentation for each class or struct on its own page doesn't help anyway. So, while that may be a good change to ddoc in at least some cases, it doesn't solve the problem in general. For instance, it would help std.datetime, but it wouldn't help std.algorithm at all.
 And if all you care
 about is sub-modules for implementation and want all of the functions in
 the
 same module still, then this DIP is pointless. All you have to do is
 declare
 undocumented sub-modules which hold the various implementations and have
 the
 actual module call them. We already do this sort of thing in Phobos to
 get
 around static destructors screaming about circular dependencies.

You are starting to see my point :) But I think the issue is not so much that you are splitting the implementation, but splitting up the API into related modules.

As I understand it, the entire point of this DIP is to enable splitting up the API cleanly without breaking code. The implementation can already be split up seemlessly. - Jonathan M Davis
Apr 05 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, April 05, 2012 16:43:24 Andrei Alexandrescu wrote:
 On 4/5/12 4:26 PM, Steven Schveighoffer wrote:
 On Thu, 05 Apr 2012 17:00:56 -0400, Jonathan M Davis
 
 <jmdavisProg gmx.com> wrote:
 On Thursday, April 05, 2012 15:30:17 Steven Schveighoffer wrote:
 I don't see how. Just move the code into another module and publicly
 import that module from std/algorithm.d. Problem pretty much solved.

The issue is code organization. If you want to split up std.algorithm (or std.datetime or whatever) into multiple modules, you have to create a new package with a completely different name with no connection to the original save for the fact that the original publicly imports it.

My view is that people will not import the smaller modules, they will only ever import std.algorithm.

I think we should be looking for a solution that not only allows replacing module -> package transparently, but also allows people to import the newly introduced fine-grained modules.

Yeah. If all we want to do is continue to always import std.algorithm, then the DIP is more or less pointless. It's the splitting of the API among multiple packages while allowing the programmer to either call/import it like he has been or to call/import it from the new module explicitly that the DIP is trying to enable. If we make it possible to split std.algorithm into multiple modules in place, then we avoid breaking code, keep the code organized in the same hierarchy - only more detailed - and allow the programmer to import on either a roughly or finely grained level, depending on which they prefer. And I really like how this could enable us to have package-specific documentation, whereas all documentation is currently module-specific and doesn't enable us to provide a page which gaves an overview of a package. That's not always necessary, but there are times when it would be quite nice (e.g. std.datetime). - Jonathan M Davis
Apr 05 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 23:18:07 -0400, Ary Manzana <ary esperanto.org.ar>  
wrote:

 On 4/5/12 10:55 PM, Steven Schveighoffer wrote:
 On Fri, 30 Mar 2012 10:46:19 -0400, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 Starting a new thread from one in announce:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

 Please comment, after which Walter will approve. Walter's approval
 means that he would approve a pull request implementing DIP16 (subject
 to regular correctness checks).


 Destroy!

BTW, this case makes the part of DIP16 which wants to shortcut fully qualified names invalid, or at least costly (I posted this code in another part of the thread, but I thought I'd bring it up higher). The following is valid code today: a/b.d: module a.b; void foo() {} struct b { static void foo() {} } main.d: import a.b; void main() { a.b.foo(); } If DIP16 were to be implemented, this becomes ambiguous. Is a.b.foo() the module function foo() from a.b, or is it a shortcut for a.b.b.foo()?

It's a shortcut to the module function foo(). If I replace main.d with: void main() { foo(); } Is foo() the module function foo() from a.b, or is it a shortcut for a.b.b.foo()? Here you have no doubts. What your mind does it: find a top level symbol "foo" in all imported modules. If you find more than one, error.

That's slightly different, because you must *always* qualify struct b's foo with a preceeding b.
 You must apply the same logic for "a.b.foo()". First you search "foo" in  
 the "a.b" symbol. Here you find it: it's the top level function "foo" in  
 "a.b". Then you stop searching.

 However, if you can't find it in the module "a.b", you search a top  
 level symbol "foo" in all modules that are in package "a.b". That's it.  
 You don't search "foo" in every possible nesting: just module nesting.

What if a.b is a struct, and it's the only possible match? We don't search for it? At some point the struct b has to come into play. Or are you saying we cannot shortcut struct FQN's? I suppose if we prefer to match modules before types, then name lookup for fully qualified names only becomes ambiguous with packages allowed to have their own modules, so it shouldn't affect existing code.
 The main issue is, because you can shortcut the FQN, and a chain of
 identifiers can have repeated identifiers in them, ambiguity is  
 possible.

As I said before, it's not a shortcut of the FQN: it's just a shortcut for the module name.

My example *was* a shortcut for the module name. I did not imply that you could shortcut the other parts of the FQN. What about hijacking though? For example: module a.b struct c { static void foo() {} } people now use a.c.foo() to avoid having to type the whole thing But along comes someone who creates: module a.c; void foo() {} Now, doesn't this usurp a.c.foo() without warning? -Steve
Apr 06 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 19:14:42 -0400, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:

 On Thursday, April 05, 2012 17:33:50 Steven Schveighoffer wrote:
 Why do we ever need to split modules for documentation? Just fix the doc
 generator so it's not as monolithic. For instance, have one page per
 class or struct.

That may or may not be desirable (certainly in the case of smaller types, I'd argue that it isn't). By doing it on a module basis, you have far more control. But regardless, that would be a major change to ddoc.

ddoc's output leaves a lot to be desired. The unorganized links at the top suck. Using module order to show symbols instead of category of symbols. How is a doc page ever "too big"? Even std.datetime loads in a second. It's more that it's "too disorganized".
 And if all you care
 about is sub-modules for implementation and want all of the functions  

 the
 same module still, then this DIP is pointless. All you have to do is
 declare
 undocumented sub-modules which hold the various implementations and  

 the
 actual module call them. We already do this sort of thing in Phobos to
 get
 around static destructors screaming about circular dependencies.

You are starting to see my point :) But I think the issue is not so much that you are splitting the implementation, but splitting up the API into related modules.

As I understand it, the entire point of this DIP is to enable splitting up the API cleanly without breaking code. The implementation can already be split up seemlessly.

As can the API via public imports. -Steve
Apr 06 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 17:43:24 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 On 4/5/12 4:26 PM, Steven Schveighoffer wrote:
 On Thu, 05 Apr 2012 17:00:56 -0400, Jonathan M Davis
 <jmdavisProg gmx.com> wrote:

 On Thursday, April 05, 2012 15:30:17 Steven Schveighoffer wrote:

 I don't see how. Just move the code into another module and publicly
 import that module from std/algorithm.d. Problem pretty much solved.

The issue is code organization. If you want to split up std.algorithm (or std.datetime or whatever) into multiple modules, you have to create a new package with a completely different name with no connection to the original save for the fact that the original publicly imports it.

My view is that people will not import the smaller modules, they will only ever import std.algorithm.

I think we should be looking for a solution that not only allows replacing module -> package transparently, but also allows people to import the newly introduced fine-grained modules.

Of course you *can* do this. I think you mean "and allows one to refer to the fine grained module as if it were imported from the top-level module". I don't think that benefit is very great. Why shouldn't we expect people to use the module name they actually imported to refer to a module? If you want selective import, we have: import std.algorithm : sort; I think the real benefit to splitting up the module comes from being able to draw clear lines between which pieces of a large module are related. I feel like most people will still import the main package module, and not the submodules. I don't think I ever wrote a piece of java code that didn't have: import java.io.*; I keep coming back to std.container. Already it's large, and full of unrelated types. It's only going to get worse. Now, I fully agree that having some way to import a package by itself without having to import all its modules would be ideal (as well as splitting a large module into submodules that live in the same namespace). I think DIP15 has that covered. -Steve
Apr 06 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, April 06, 2012 07:56:41 Steven Schveighoffer wrote:
 On Thu, 05 Apr 2012 19:14:42 -0400, Jonathan M Davis <jmdavisProg gmx.com>
 
 wrote:
 On Thursday, April 05, 2012 17:33:50 Steven Schveighoffer wrote:
 Why do we ever need to split modules for documentation? Just fix the doc
 generator so it's not as monolithic. For instance, have one page per
 class or struct.

That may or may not be desirable (certainly in the case of smaller types, I'd argue that it isn't). By doing it on a module basis, you have far more control. But regardless, that would be a major change to ddoc.

ddoc's output leaves a lot to be desired. The unorganized links at the top suck. Using module order to show symbols instead of category of symbols. How is a doc page ever "too big"? Even std.datetime loads in a second. It's more that it's "too disorganized".

There's no question that the links at the top suck, and std.datetime is the poster child for why they suck. The page is too large for how its organized, and it _does_ take significantly longer to load than other Phobos pages, even if it's still usable. There's no question that better links would help significantly though. Regardless, there's enough content there that it would arguably be better served if it were multiple pages.
 And if all you care
 about is sub-modules for implementation and want all of the functions

in
 the
 same module still, then this DIP is pointless. All you have to do is
 declare
 undocumented sub-modules which hold the various implementations and

have
 the
 actual module call them. We already do this sort of thing in Phobos to
 get
 around static destructors screaming about circular dependencies.

You are starting to see my point :) But I think the issue is not so much that you are splitting the implementation, but splitting up the API into related modules.

As I understand it, the entire point of this DIP is to enable splitting up the API cleanly without breaking code. The implementation can already be split up seemlessly.

As can the API via public imports.

Not and keep stuff in the same place in the hierarchy. You have to create a completely separate package. The point of the DIP was to make it so that you could do it _in place_ by turning a module like std/algorithm.d into the package std/algorithm/ with modules inside it. Right now, I can go and split up std.datetime into a new package (e.g. std.dtime) and then publicly import it all from std.datetime, but then the new package is completely separate from the module which imports everything. It also makes it look like std.datetime should go away in favor of just keeping std.dtime (the same with std.algorithm if it gets split), which may or may not actually be desirable. Organizationally, it would be much nicer to be able to turn std/datetime.d into std/datetime/, and _that_ is what the DIP is trying to enable. The rest can be done now but not that. You can't split up the API in place. - Jonathan M Davis
Apr 06 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday, April 07, 2012 18:45:15 Jacob Carlborg wrote:
 On 2012-04-07 02:25, Jonathan M Davis wrote:
 On Friday, April 06, 2012 08:09:28 Steven Schveighoffer wrote:
 I feel like most people will still import the main package module, and
 not
 the submodules.  I don't think I ever wrote a piece of java code that
 didn't have:
 
 import java.io.*;

Which is actually considered bad practice in Java, though a lot of people do like to do it. What's generally considered good practice is to explicitly import every module/class, and Eclipse likes to underline imports in red if you don't.

import foo.bar.* Is used all over the place in SWT.

Like I said, some people do like to do it, but Eclipse doesn't like you to, and there are quite a few Java folks who argue that it's bad practice. I forget what the reasons were (maybe increased buld times due to pulling in more symbols or more issues with symbol conflicts - I don't recall), but I wasn't particularly convinced when I heard them. Regardless though, there's a definite contingent against importing with * in the Java community, and it was my understanding that that contigent was the majority of that community, but I don't know. - Jonathan M Davis
Apr 07 2012
prev sibling next sibling parent "Kapps" <opantm2+spam gmail.com> writes:
On Saturday, 7 April 2012 at 17:44:20 UTC, Jonathan M Davis wrote
 Like I said, some people do like to do it, but Eclipse doesn't 
 like you to,
 and there are quite a few Java folks who argue that it's bad 
 practice. I
 forget what the reasons were (maybe increased buld times due to 
 pulling in
 more symbols or more issues with symbol conflicts - I don't 
 recall), but I
 wasn't particularly convinced when I heard them. Regardless 
 though, there's a
 definite contingent against importing with * in the Java 
 community, and it was
 my understanding that that contigent was the majority of that 
 community, but I
 don't know.

 - Jonathan M Davis

http://stackoverflow.com/questions/147454/why-is-using-a-wild-card-with-a-java-import-statement-bad Their reasoning sounds more due to various packages reinventing things or being poorly split than actual flaws with package imports themselves. Besides, D already addresses the issues of indicating which one you want, without the horribly long package names that Java has. Seems to me their examples are more like 'import std.*' than 'import std.datetime.*'.
Apr 07 2012
prev sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Monday, April 09, 2012 08:55:27 Steven Schveighoffer wrote:
 On Fri, 06 Apr 2012 20:25:23 -0400, Jonathan M Davis <jmdavisProg gmx.com>
 
 wrote:
 DIP15 doesn't fix the explicit path problem though. You can't change
 std/algorithm.d into std/algorithm/ (with sorting.d, search.d, etc.)
 without
 breaking code. You could make std/algorithm.d publicly import std/alg/*
 and
 then DIP15 would allow you to import std.alg to get all of its
 sub-modules,
 but you're still forced to use a module to publicly import symbols as
 part of
 a migration path, and you can't split a module in place.

I think either you or I am missing something. In DIP15, if you define std/algorithm/_.d, and then import std.algorithm, it imports std/algorithm/_.d, which then 1. publicly imports other modules, and 2. aliases symbols to the name std.algorithm.symbol. At least, this is how I understand the intent. It seems equivalent to me to the package.d proposal, it's just using _.d instead of package.d. If you import std.algorithm.sorting, and try and use std.algorithm.sort, yes it will not work. But this does not break existing code (which does not import std.algorithm.sorting), and I find it odd that we want to make std.algorithm.sort work if you don't import std.algorithm.

Okay. I reread DIP15 again. I guess that I scanned over it too quickly before and/or misremembered it. I had understood that it was proposing that importing std.algorithm where std.algorithm was a package would be the equivalent of importing std.algorithm.* in Java and that there were no extra files involved. So clearly, I've been misunderstanding things here. So, yeah. DIP15 is basically the same as DIP16 except without the std.sort nonsense and the fact that it uses _.d instead of package.d. Using package.d has the advantage of package being a keyword, making it so that no one is going to accidentally create a module that will be treated specially, but it has the downside of likely requiring more special handling by the compiler. I don't really care which we pick though. My main point though, misunderstandings aside, is that it would be _really_ nice to be able to split up  a package in place and that without an enhancement of some kind, we can't do  that without breaking code. DIP15 appears to fit the bill quite nicely in that regard though. The part of DIP16 which is really bad is the std.sort stuff which. Public importing combined with either the first part of DIP16 or with DIP15 seems to take care of the problem quite nicely. - Jonathan M Davis
Apr 09 2012