www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DIP10005: Dependency-Carrying Declarations is now available for

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Destroy.

https://github.com/dlang/DIPs/pull/51/files


Andrei
Dec 13 2016
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
(eh I was off by one zero in the title)
Dec 13 2016
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/13/2016 05:33 PM, Andrei Alexandrescu wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files
The "View" mode offers formatting before merging: https://github.com/andralex/DIPs/blob/10ca2e22c0d759c81e4b3afbdc085d7131f6 b85/DIPs/DIP1005.md -- Andrei
Dec 13 2016
prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 13.12.2016 23:33, Andrei Alexandrescu wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei
1. The syntax is ambiguous. Is import.foo.bar.baz the symbol baz in module foo.bar, or the symbol bar.baz in module foo? I'd prefer syntax like (import foo.bar).baz and (import foo).bar.baz. (I.e., the syntax of import expressions would closely mirror that of import declarations, and would be unambiguous.) 2. The behaviour of aliases to import expressions should be defined explicitly. I.e. alias short = import very.long.module_name; void foo(int i)(short.T a){ ... } does this import the module if foo is not instantiated?
Dec 13 2016
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/13/16 6:03 PM, Timon Gehr wrote:
 On 13.12.2016 23:33, Andrei Alexandrescu wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei
1. The syntax is ambiguous. Is import.foo.bar.baz the symbol baz in module foo.bar, or the symbol bar.baz in module foo? I'd prefer syntax like (import foo.bar).baz and (import foo).bar.baz. (I.e., the syntax of import expressions would closely mirror that of import declarations, and would be unambiguous.)
That is a problem. I switched to the most attractive alternate syntax. https://github.com/andralex/DIPs/blob/2e5859c0f64ac4949123fe8de39ccf2ccf72d859/DIPs/DIP1005.md Andrei
Dec 13 2016
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 14.12.2016 01:14, Andrei Alexandrescu wrote:
 On 12/13/16 6:03 PM, Timon Gehr wrote:
 On 13.12.2016 23:33, Andrei Alexandrescu wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei
1. The syntax is ambiguous. Is import.foo.bar.baz the symbol baz in module foo.bar, or the symbol bar.baz in module foo? I'd prefer syntax like (import foo.bar).baz and (import foo).bar.baz. (I.e., the syntax of import expressions would closely mirror that of import declarations, and would be unambiguous.)
That is a problem. I switched to the most attractive alternate syntax. https://github.com/andralex/DIPs/blob/2e5859c0f64ac4949123fe8de39ccf2ccf72d859/DIPs/DIP1005.md Andrei
I'd still advise to match the syntax of other import declarations. What justifies the difference? (Note that import(foo.bar.baz) is a string import.)
Dec 13 2016
parent Yuxuan Shui <yshuiv7 gmail.com> writes:
On Wednesday, 14 December 2016 at 00:21:23 UTC, Timon Gehr wrote:
 On 14.12.2016 01:14, Andrei Alexandrescu wrote:
 On 12/13/16 6:03 PM, Timon Gehr wrote:
 [...]
That is a problem. I switched to the most attractive alternate syntax. https://github.com/andralex/DIPs/blob/2e5859c0f64ac4949123fe8de39ccf2ccf72d859/DIPs/DIP1005.md Andrei
I'd still advise to match the syntax of other import declarations. What justifies the difference? (Note that import(foo.bar.baz) is a string import.)
Maybe it's a good idea to make it a property? e.g.: import("std.stdio") void process(File f);
Dec 13 2016
prev sibling parent reply Chris M. <chrismohrfeld comcast.net> writes:
On Tuesday, 13 December 2016 at 23:03:39 UTC, Timon Gehr wrote:
 On 13.12.2016 23:33, Andrei Alexandrescu wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei
1. The syntax is ambiguous. Is import.foo.bar.baz the symbol baz in module foo.bar, or the symbol bar.baz in module foo? I'd prefer syntax like (import foo.bar).baz and (import foo).bar.baz. (I.e., the syntax of import expressions would closely mirror that of import declarations, and would be unambiguous.)
Why not something like import foo.bar : baz or import foo : bar.baz to distinguish between module and symbol? It's already used anyway. Also I like Yuxuan's idea, the other ideas seem to add more clutter after the function name
Dec 13 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/13/16 8:07 PM, Chris M. wrote:
 Why not something like import foo.bar : baz or import foo : bar.baz to
 distinguish between module and symbol? It's already used anyway.

 Also I like Yuxuan's idea, the other ideas seem to add more clutter
 after the function name
So we have: // 1 struct Buffer(R) if ((import std.range).isInputRange!R) { ... } // 2 struct Buffer(R) if (import std.range:isInputRange!R) { ... } // 3 import("std.stdio") struct Buffer(R) if (isInputRange!R) { ... } The first two require repeating the import for each separate declaration, e.g.: // 1 bool equal(R1, R2) if ((import std.range).isInputRange!R1 && (import std.range).isInputRange!R2) { ... } // 2 bool equal(R1, R2) if (import std.range:isInputRange!R1 && import std.range:isInputRange!R2) { ... } The last is surprising because it uses a string where otherwise there would be an unquoted list of dot-separated names. I prefer the current form of the proposal: bool equal(R1, R2) import (std.range) if (isInputRange!R1 && isInputRange!R2) { ... } The point has been brought up that the syntax import(std.range) is also used for string imports. It is a drawback. Andrei
Dec 13 2016
next sibling parent reply Chris M. <chrismohrfeld comcast.net> writes:
On Wednesday, 14 December 2016 at 01:39:01 UTC, Andrei 
Alexandrescu wrote:

 I prefer the current form of the proposal:

 bool equal(R1, R2)
 import (std.range)
 if (isInputRange!R1 && isInputRange!R2)
 { ... }

 The point has been brought up that the syntax import(std.range) 
 is also used for string imports. It is a drawback.


 Andrei
How about using "imports" instead of "import"? Simple enough change, and it still makes sense bool equal(R1, R2) imports (std.range) if (isInputRange!R1 && isInputRange!R2) { ... }
Dec 13 2016
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/13/16 8:53 PM, Chris M. wrote:
 On Wednesday, 14 December 2016 at 01:39:01 UTC, Andrei Alexandrescu wrote:

 I prefer the current form of the proposal:

 bool equal(R1, R2)
 import (std.range)
 if (isInputRange!R1 && isInputRange!R2)
 { ... }

 The point has been brought up that the syntax import(std.range) is
 also used for string imports. It is a drawback.


 Andrei
How about using "imports" instead of "import"? Simple enough change, and it still makes sense bool equal(R1, R2) imports (std.range) if (isInputRange!R1 && isInputRange!R2) { ... }
I fear that (a) a lot of folks will believe "imports" is a keyword; (b) another lot of folks will write "import" instead of "imports" and wonder why it doesn't work. -- Andrei
Dec 13 2016
parent Chris M. <chrismohrfeld comcast.net> writes:
On Wednesday, 14 December 2016 at 02:23:07 UTC, Andrei 
Alexandrescu wrote:
 On 12/13/16 8:53 PM, Chris M. wrote:
 On Wednesday, 14 December 2016 at 01:39:01 UTC, Andrei 
 Alexandrescu wrote:

 I prefer the current form of the proposal:

 bool equal(R1, R2)
 import (std.range)
 if (isInputRange!R1 && isInputRange!R2)
 { ... }

 The point has been brought up that the syntax 
 import(std.range) is
 also used for string imports. It is a drawback.


 Andrei
How about using "imports" instead of "import"? Simple enough change, and it still makes sense bool equal(R1, R2) imports (std.range) if (isInputRange!R1 && isInputRange!R2) { ... }
I fear that (a) a lot of folks will believe "imports" is a keyword; (b) another lot of folks will write "import" instead of "imports" and wonder why it doesn't work. -- Andrei
Could use something like "using" or "with" instead of "import" then. bool equal(R1, R2) using(std.range) if (isInputRange!R1 && isInputRange!R2) { ... } Of course, the collision with the syntax for string imports might not be a huge deal anyway, since you could easily distinguish between when it's for a string import and when it's for a inline import just by the context you see it in. Leaving it as "import" would probably be fine too.
Dec 13 2016
prev sibling parent reply pineapple <meapineapple gmail.com> writes:
On Wednesday, 14 December 2016 at 01:53:44 UTC, Chris M. wrote:
 How about using "imports" instead of "import"? Simple enough 
 change, and it still makes sense

 bool equal(R1, R2)
 imports (std.range)
 if (isInputRange!R1 && isInputRange!R2)
 { ... }
On Tuesday, 13 December 2016 at 23:03:39 UTC, Timon Gehr wrote:
 I'd prefer syntax like (import foo.bar).baz and (import 
 foo).bar.baz. (I.e., the syntax of import expressions would 
 closely mirror that of import declarations, and would be 
 unambiguous.)

 2. The behaviour of aliases to import expressions should be 
 defined explicitly.

 I.e.

 alias short = import very.long.module_name;

 void foo(int i)(short.T a){ ... }

 does this import the module if foo is not instantiated?
I am most in favor of making the function signature either `imports` or ` imports`, or doing this: struct Buffer(R) if (import std.range:isInputRange!R) { ... } I also think actually being able to write `alias short = import very.long.module_name;` would be great, if only so that the contents of a module can be imported into their own user-defined namespace rather than the global scope.
Dec 14 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/16 5:55 AM, pineapple wrote:
 On Wednesday, 14 December 2016 at 01:53:44 UTC, Chris M. wrote:
 How about using "imports" instead of "import"? Simple enough change,
 and it still makes sense

 bool equal(R1, R2)
 imports (std.range)
 if (isInputRange!R1 && isInputRange!R2)
 { ... }
On Tuesday, 13 December 2016 at 23:03:39 UTC, Timon Gehr wrote:
 I'd prefer syntax like (import foo.bar).baz and (import foo).bar.baz.
 (I.e., the syntax of import expressions would closely mirror that of
 import declarations, and would be unambiguous.)

 2. The behaviour of aliases to import expressions should be defined
 explicitly.

 I.e.

 alias short = import very.long.module_name;

 void foo(int i)(short.T a){ ... }

 does this import the module if foo is not instantiated?
I am most in favor of making the function signature either `imports` or ` imports`, or doing this:
No inflections please. Walter requires the use of the "import" keyword, and I agree with him.
 struct Buffer(R) if (import std.range:isInputRange!R) { ... }

 I also think actually being able to write `alias short = import
 very.long.module_name;` would be great, if only so that the contents of
 a module can be imported into their own user-defined namespace rather
 than the global scope.
The alias fails to meet the DCD principle: the declaration carries dependencies with it. Andrei
Dec 14 2016
prev sibling next sibling parent reply Hatem Oraby <hatemoraby gmail.com> writes:
On Wednesday, 14 December 2016 at 01:39:01 UTC, Andrei 
Alexandrescu wrote:
 On 12/13/16 8:07 PM, Chris M. wrote:
 [...]
So we have: // 1 struct Buffer(R) if ((import std.range).isInputRange!R) { ... } [...]
What about?: with(import std.range) bool equal(R1, R2) if (isInputRange!R1 && isInputRange!R2) { ... }
Dec 13 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/13/16 9:22 PM, Hatem Oraby wrote:
 On Wednesday, 14 December 2016 at 01:39:01 UTC, Andrei Alexandrescu wrote:
 On 12/13/16 8:07 PM, Chris M. wrote:
 [...]
So we have: // 1 struct Buffer(R) if ((import std.range).isInputRange!R) { ... } [...]
What about?: with(import std.range) bool equal(R1, R2) if (isInputRange!R1 && isInputRange!R2) { ... }
I considered this, then figured with is superfluous. -- Andrei
Dec 13 2016
parent reply Jacob Carlborg <doob me.com> writes:
On 2016-12-14 03:23, Andrei Alexandrescu wrote:
 On 12/13/16 9:22 PM, Hatem Oraby wrote:
 with(import std.range)
 bool equal(R1, R2) if (isInputRange!R1 && isInputRange!R2)
 { ... }
I considered this, then figured with is superfluous. -- Andrei
It could allow to have a better control of the scope which the import affects, i.e.: with(import std.range) { void foo(T) if (isInputRange!T) void bar(T) if (isInputRange!T) } -- /Jacob Carlborg
Dec 13 2016
next sibling parent reply Suliman <evermind live.ru> writes:
On Wednesday, 14 December 2016 at 07:17:57 UTC, Jacob Carlborg 
wrote:
 On 2016-12-14 03:23, Andrei Alexandrescu wrote:
 On 12/13/16 9:22 PM, Hatem Oraby wrote:
 with(import std.range)
 bool equal(R1, R2) if (isInputRange!R1 && isInputRange!R2)
 { ... }
I considered this, then figured with is superfluous. -- Andrei
It could allow to have a better control of the scope which the import affects, i.e.: with(import std.range) { void foo(T) if (isInputRange!T) void bar(T) if (isInputRange!T) }
Look very nice!
Dec 14 2016
parent reply Yuxuan Shui <yshuiv7 gmail.com> writes:
On Wednesday, 14 December 2016 at 08:16:56 UTC, Suliman wrote:
 On Wednesday, 14 December 2016 at 07:17:57 UTC, Jacob Carlborg 
 wrote:
 On 2016-12-14 03:23, Andrei Alexandrescu wrote:
 On 12/13/16 9:22 PM, Hatem Oraby wrote:
 with(import std.range)
 bool equal(R1, R2) if (isInputRange!R1 && isInputRange!R2)
 { ... }
I considered this, then figured with is superfluous. -- Andrei
It could allow to have a better control of the scope which the import affects, i.e.: with(import std.range) { void foo(T) if (isInputRange!T) void bar(T) if (isInputRange!T) }
Look very nice!
If this looks nice to you then I would like to propose: import("std.range"): void foo(T) if (isInputRange!T) And: import("std.range") { void foo(T) if (isInputRange!T) void bar(T) if (isInputRange!T) } In my eye this whole thing fits really well into the property syntax. Yes, it's a little strange to pass module name as string. Otherwise it's perfect. Maybe we can allow properties to take symbol as argument?
Dec 14 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/16 3:43 AM, Yuxuan Shui wrote:
  import("std.range"):
This seems to break the requirement that declaration-local imports should offer the same amenities as regular imports. It would seem much if we started putting all of the import syntax in strings. -- Andrei
Dec 14 2016
prev sibling next sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Wednesday, 14 December 2016 at 07:17:57 UTC, Jacob Carlborg 
wrote:
 On 2016-12-14 03:23, Andrei Alexandrescu wrote:
 On 12/13/16 9:22 PM, Hatem Oraby wrote:
 with(import std.range)
 bool equal(R1, R2) if (isInputRange!R1 && isInputRange!R2)
 { ... }
I considered this, then figured with is superfluous. -- Andrei
It could allow to have a better control of the scope which the import affects, i.e.: with(import std.range) { void foo(T) if (isInputRange!T) void bar(T) if (isInputRange!T) }
Trouble is, there's no real difference between doing that, vs. creating a standalone module containing `foo` and `bar` with `import std.range;` as a top-level import. The `with` syntax could also be used internally in functions to narrow the functionality that has access to the imported symbols, but something like: void foo(T)(T input) { with (import some_module) { // only stuff in here has access // to some_module's symbols } // nothing here has access to // some_module's symbols } ... doesn't give you anything that you can't already get using: void foo(T)(T input) { { import some_module; // only stuff in here has access // to some_module's symbols } // nothing here has access to // some_module's symbols } In fact that's a good argument against the `with` syntax entirely, because in typical usage AFAIR, it creates a scope, whereas one wouldn't want the symbols defined in Hatem's proposal to be scoped relative to the module. It's a shame, because unlike Andrei I don't feel use of `with` is in principle superfluous: it has a value in clarifying intention, while allowing the use of traditional `import something` syntax. But special-casing of where `with` does and doesn't create a scope seems ... less good.
Dec 14 2016
next sibling parent reply Mathias Lang <mathias.lang sociomantic.com> writes:
On Wednesday, 14 December 2016 at 09:01:30 UTC, Joseph Rushton 
Wakeling wrote:
 On Wednesday, 14 December 2016 at 07:17:57 UTC, Jacob Carlborg 
 wrote:
 On 2016-12-14 03:23, Andrei Alexandrescu wrote:
 On 12/13/16 9:22 PM, Hatem Oraby wrote:
 with(import std.range)
 bool equal(R1, R2) if (isInputRange!R1 && isInputRange!R2)
 { ... }
I considered this, then figured with is superfluous. -- Andrei
It could allow to have a better control of the scope which the import affects, i.e.: with(import std.range) { void foo(T) if (isInputRange!T) void bar(T) if (isInputRange!T) }
Trouble is, there's no real difference between doing that, vs. creating a standalone module containing `foo` and `bar` with `import std.range;` as a top-level import.
That was my impression when reading this DIP. I'm very glad to see that decoupling made its way up in the growing list of things to do, my only concern is that this syntax sounds like a workaround for giant modules. Phobos is cited as a motivation for this enhancement. Dare I say that we have a problem of modules in phobos being too monolithic, and they should be split into more packages, like std.range and std.algorithms did ?
Dec 14 2016
next sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Wednesday, 14 December 2016 at 12:01:40 UTC, Mathias Lang 
wrote:
 That was my impression when reading this DIP. I'm very glad to 
 see that decoupling made its way up in the growing list of 
 things to do, my only concern is that this syntax sounds like a 
 workaround for giant modules.

 Phobos is cited as a motivation for this enhancement. Dare I 
 say that we have a problem of modules in phobos being too 
 monolithic, and they should be split into more packages, like 
 std.range and std.algorithms did ?
Yea, I think you put your finger on it: almost all of the stuff this feature could help achieve in Phobos could be just as well achieved by splitting stuff up better. Note, `std.range` and `std.algorithm` could still be much more modularized than they currently are.
Dec 14 2016
parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Wednesday, 14 December 2016 at 12:15:20 UTC, Joseph Rushton 
Wakeling wrote:
 Yea, I think you put your finger on it: almost all of the stuff 
 this feature could help achieve in Phobos could be just as well 
 achieved by splitting stuff up better.
no, it can't. why should i remember that i have to import "std.range.xyz" for something? of course, i will write `import std.range;` and forget about it. no kind of splitting will allow me to have self-contained "hygienic" declarations. finally, some DIP that addresses something i really think about almost every day.
Dec 14 2016
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/16 7:30 AM, ketmar wrote:
 finally, some DIP that addresses something i really think about almost
 every day.
Additional motivating examples are welcome. -- Andrei
Dec 14 2016
parent ketmar <ketmar ketmar.no-ip.org> writes:
On Wednesday, 14 December 2016 at 14:19:43 UTC, Andrei 
Alexandrescu wrote:
 On 12/14/16 7:30 AM, ketmar wrote:
 finally, some DIP that addresses something i really think 
 about almost
 every day.
Additional motivating examples are welcome. -- Andrei
it happened that DIP itself has all the samples i can think out. ;-) imports for function arguments and template constraints are the things i want. and i don't really have any favor for any particular syntax. thanks for making this DIP.
Dec 14 2016
prev sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday, 14 December 2016 at 12:30:33 UTC, ketmar wrote:
 On Wednesday, 14 December 2016 at 12:15:20 UTC, Joseph Rushton 
 Wakeling wrote:
 Yea, I think you put your finger on it: almost all of the 
 stuff this feature could help achieve in Phobos could be just 
 as well achieved by splitting stuff up better.
no, it can't. why should i remember that i have to import "std.range.xyz" for something? of course, i will write `import std.range;` and forget about it. no kind of splitting will allow me to have self-contained "hygienic" declarations.
Yeah, modularity from the standpoint of importing a set of functions and modularity from the standpoint of what those functions import are orthogonal concerns. Without this DIP, we're often unable to treat them orthogonally, but in principle, they are. Are this DIP (regardless of the exact syntax) would allow us to. - Jonathan M Davis
Dec 14 2016
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/16 7:01 AM, Mathias Lang wrote:
 Trouble is, there's no real difference between doing that, vs.
 creating a standalone module containing `foo` and `bar` with `import
 std.range;` as a top-level import.
That was my impression when reading this DIP. I'm very glad to see that decoupling made its way up in the growing list of things to do, my only concern is that this syntax sounds like a workaround for giant modules. Phobos is cited as a motivation for this enhancement. Dare I say that we have a problem of modules in phobos being too monolithic, and they should be split into more packages, like std.range and std.algorithms did ?
Creating many files indeed works around the problem that dependency granularity is file-level. I don't think that approach is desirable or scalable. -- Andrei
Dec 14 2016
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 14.12.2016 10:01, Joseph Rushton Wakeling wrote:
 On Wednesday, 14 December 2016 at 07:17:57 UTC, Jacob Carlborg wrote:
 On 2016-12-14 03:23, Andrei Alexandrescu wrote:
 On 12/13/16 9:22 PM, Hatem Oraby wrote:
 with(import std.range)
 bool equal(R1, R2) if (isInputRange!R1 && isInputRange!R2)
 { ... }
I considered this, then figured with is superfluous. -- Andrei
It could allow to have a better control of the scope which the import affects, i.e.: with(import std.range) { void foo(T) if (isInputRange!T) void bar(T) if (isInputRange!T) }
Trouble is, there's no real difference between doing that, vs. creating a standalone module containing `foo` and `bar` with `import std.range;` as a top-level import. ...
This point is independent of syntax.
 ...
 It's a shame, because unlike Andrei I don't feel use of `with` is in
 principle superfluous: it has a value in clarifying intention, while
 allowing the use of traditional `import something` syntax.  But
 special-casing of where `with` does and doesn't create a scope seems ...
 less good.
'static with'?
Dec 14 2016
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/16 2:17 AM, Jacob Carlborg wrote:
 On 2016-12-14 03:23, Andrei Alexandrescu wrote:
 On 12/13/16 9:22 PM, Hatem Oraby wrote:
 with(import std.range)
 bool equal(R1, R2) if (isInputRange!R1 && isInputRange!R2)
 { ... }
I considered this, then figured with is superfluous. -- Andrei
It could allow to have a better control of the scope which the import affects, i.e.: with(import std.range) { void foo(T) if (isInputRange!T) void bar(T) if (isInputRange!T) }
Thanks for this suggestion. * "with" is superfluous * The grouping is not indicating that the import is effected only if one of the names is looked up * I predict 90% of the time there will be one import per declaration, which takes it down to: with (import std.range) void foo(T) if (isInputRange!T) {} which underlines the redundancy of the "with". Andrei
Dec 14 2016
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday, 14 December 2016 at 14:11:47 UTC, Andrei 
Alexandrescu wrote:
 * I predict 90% of the time there will be one import per 
 declaration, which takes it down to:

 with (import std.range)
 void foo(T) if (isInputRange!T) {}

 which underlines the redundancy of the "with".
Honestly, while I can see some value in what Jacob is suggesting, because it allows you to restrict some imports without repeating them, I think that the reality of the matter is that in practice, it would result in the imports just being at the top of the module - particularly when you consider that function signtaures are actually pretty far apart in most cases, since you have the function body (which is often not small), the unittest block, and the documentation for the next function, making it pretty ugly to wrap all of those in an import block. - Jonathan M Davis
Dec 14 2016
prev sibling parent reply Jon Degenhardt <noreply noreply.com> writes:
On Wednesday, 14 December 2016 at 01:39:01 UTC, Andrei 
Alexandrescu wrote:
 I prefer the current form of the proposal:

 bool equal(R1, R2)
 import (std.range)
 if (isInputRange!R1 && isInputRange!R2)
 { ... }
I didn't see an example of the syntax when multiple imports are involved (apologies if I missed it). Would be good to spell it out. For example, the start to uninitializedArray (array.d): auto uninitializedArray(T, I...)(I sizes) nothrow system if (isDynamicArray!T && allSatisfy!(isIntegral, I) && hasIndirections!(ElementEncodingType!T)) { This uses imports from std.traits, std.meta, and std.range. Would there be one import statement or three? I'm guessing so the intent is one, so perhaps: auto uninitializedArray(T, I...)(I sizes) nothrow system import (std.traits; std.meta; std.range) if (isDynamicArray!T && allSatisfy!(isIntegral, I) && hasIndirections!(ElementEncodingType!T)) { Is that the idea? Similarly for importing individual symbols. In the above case: auto uninitializedArray(T, I...)(I sizes) nothrow system import (std.traits : hasIndirections, isDynamicArray, isIntegral; std.meta : allSatisfy; std.range : ElementEncodingType) if (isDynamicArray!T && allSatisfy!(isIntegral, I) && hasIndirections!(ElementEncodingType!T)) { --Jon
Dec 13 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/13/16 10:37 PM, Jon Degenhardt wrote:
 I didn't see an example of the syntax when multiple imports are involved
 (apologies if I missed it). Would be good to spell it out.
Thanks for the point and examples, will change the DIP accordingly. -- Andrei
Dec 14 2016
prev sibling next sibling parent reply Suliman <evermind live.ru> writes:
On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu 
wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files
Imho such syntaxis construction make language harder to learn. D is already pretty complex, but DIPs should simplify language, but do not make its harder
Dec 13 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/16 12:00 AM, Suliman wrote:
 On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files
Imho such syntaxis construction make language harder to learn. D is already pretty complex, but DIPs should simplify language, but do not make its harder
Thanks for the input. My take on this is DIPs should make the language _better_. A lot of the time simpler is indeed better, but not always. The motivation section should cover why we ought to take the hit of a complication in the language. Please let me know where it seems lacking. Andrei
Dec 14 2016
prev sibling next sibling parent reply Dicebot <public dicebot.lv> writes:
On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu 
wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei
Proposed functionality does feel useful but syntax options mentioned in the DIP all seem lacking to me. Most importantly, the one suggested as primary option puts import after the symbol in lexical order which looks baroque and may even be unprecedented in the language (can't think of another case like that immediately).
Dec 14 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/16 3:17 AM, Dicebot wrote:
 On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei
Proposed functionality does feel useful but syntax options mentioned in the DIP all seem lacking to me. Most importantly, the one suggested as primary option puts import after the symbol in lexical order which looks baroque and may even be unprecedented in the language (can't think of another case like that immediately).
I don't worry about that too much. We have a precedent with names used in the return type of a function, and nobody has ever complained about that: CommonType!(T1, T2) fun(T1, T2)(T1, T2); Andrei
Dec 14 2016
prev sibling next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 14/12/2016 11:33 AM, Andrei Alexandrescu wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei
Others have brought up similar syntax to this: import(my.mod).Type!argsT(7) import(Identifier).Identifier It shouldn't be confused with string imports or the like and would be more akin to a selective import. Pros: - Is an expression - Relatively clear and in line with string imports as an expression Cons: - Another language feature to learn (but if I was designing a new language, this would totally be in it) - More text that goes into template constraints - Personally I find if I need an import like this in one constraint, I want it in a lot more places - Much longer lines! ```D void func(T)(T x) if (import(std.traits).isBasicType!T || import(std.traits).isFloatingPoint!T) { } ```
Dec 14 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/16 6:06 AM, rikki cattermole wrote:
 On 14/12/2016 11:33 AM, Andrei Alexandrescu wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei
Others have brought up similar syntax to this: import(my.mod).Type!argsT(7) import(Identifier).Identifier It shouldn't be confused with string imports or the like and would be more akin to a selective import. Pros: - Is an expression
How does it then apply to declarations? -- Andrei
Dec 14 2016
parent rikki cattermole <rikki cattermole.co.nz> writes:
On 15/12/2016 3:16 AM, Andrei Alexandrescu wrote:
 On 12/14/16 6:06 AM, rikki cattermole wrote:
 On 14/12/2016 11:33 AM, Andrei Alexandrescu wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei
Others have brought up similar syntax to this: import(my.mod).Type!argsT(7) import(Identifier).Identifier It shouldn't be confused with string imports or the like and would be more akin to a selective import. Pros: - Is an expression
How does it then apply to declarations? -- Andrei
Like this? import(my.mod).Type!argsT value; Or? import(my.mod).Type!argsT func(import(my.mod).Type!argsT arg) { }
Dec 14 2016
prev sibling next sibling parent reply Dominikus Dittes Scherkl <Dominikus.Scherkl continental-corporation.com> writes:
On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu 
wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files
Why not leave it as it is and only change the compiler to perform inputs _within_ a function before evaluating the declaration, so that the symbols imported can be used in the declaration? e.g. fun(Range x) if(isInputRange!x) { import std.range; auto a = x.front(); } this is especially more clean if a function needs several imports and maybe some of them in the declaration but not all of them. This would otherwise split the imports to two different points, some within the (already much too long) declaration and some within the function. I consider this ugly.
Dec 14 2016
next sibling parent drug <drug2004 bk.ru> writes:
14.12.2016 16:26, Dominikus Dittes Scherkl пишет:
 On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files
Why not leave it as it is and only change the compiler to perform inputs _within_ a function before evaluating the declaration, so that the symbols imported can be used in the declaration? e.g. fun(Range x) if(isInputRange!x) { import std.range; auto a = x.front(); } this is especially more clean if a function needs several imports and maybe some of them in the declaration but not all of them. This would otherwise split the imports to two different points, some within the (already much too long) declaration and some within the function. I consider this ugly.
I think this is really smart way to solve the problem.
Dec 14 2016
prev sibling next sibling parent Robert burner Schadek <rburners gmail.com> writes:
I really like that solution. +1
Dec 14 2016
prev sibling next sibling parent Robert burner Schadek via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 2016-12-14 14:26, Dominikus Dittes Scherkl via Digitalmars-d wrote:
 On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files
Why not leave it as it is and only change the compiler to perform inputs _within_ a function before evaluating the declaration, so that the symbols imported can be used in the declaration? e.g. fun(Range x) if(isInputRange!x) { import std.range; auto a = x.front(); } this is especially more clean if a function needs several imports and maybe some of them in the declaration but not all of them. This would otherwise split the imports to two different points, some within the (already much too long) declaration and some within the function. I consider this ugly.
I like that, +1
Dec 14 2016
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/16 8:26 AM, Dominikus Dittes Scherkl wrote:
 On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files
Why not leave it as it is and only change the compiler to perform inputs _within_ a function before evaluating the declaration, so that the symbols imported can be used in the declaration?
Thanks. I considered this but it put pressure on the relationship between the name and where it's looked up. Consider that imports in a function don't need to be at the top. They may also be in nested scopes inside the function. It becomes quite tenuous to explain where the parameter type names are looked up. -- Andrei
Dec 14 2016
parent reply default0 <Kevin.Labschek gmx.de> writes:
On Wednesday, 14 December 2016 at 14:21:55 UTC, Andrei 
Alexandrescu wrote:
 On 12/14/16 8:26 AM, Dominikus Dittes Scherkl wrote:
 On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei 
 Alexandrescu wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files
Why not leave it as it is and only change the compiler to perform inputs _within_ a function before evaluating the declaration, so that the symbols imported can be used in the declaration?
Thanks. I considered this but it put pressure on the relationship between the name and where it's looked up. Consider that imports in a function don't need to be at the top. They may also be in nested scopes inside the function. It becomes quite tenuous to explain where the parameter type names are looked up. -- Andrei
Could restrict to imports that come before any other statements in the function body, ie: // valid, compiles when instantiated void fun(Range)(Range r) if(isInputRange!Range) { import std.range; } // invalid, import is not picked up because it is not the first statement in the method body void fun(Range)(Range r) if(isInputRange!Range) { doThings(); import std.range; } If this is desirable, the error message for failed symbol lookup of symbols in a function declaration should hint that imports in the function body are only considered if they stand at the beginning of the function, to help mitigate confusion. Seems simple enough to specify and explain, honestly.
Dec 14 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/2016 11:02 AM, default0 wrote:
 On Wednesday, 14 December 2016 at 14:21:55 UTC, Andrei Alexandrescu wrote:
 On 12/14/16 8:26 AM, Dominikus Dittes Scherkl wrote:
 On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files
Why not leave it as it is and only change the compiler to perform inputs _within_ a function before evaluating the declaration, so that the symbols imported can be used in the declaration?
Thanks. I considered this but it put pressure on the relationship between the name and where it's looked up. Consider that imports in a function don't need to be at the top. They may also be in nested scopes inside the function. It becomes quite tenuous to explain where the parameter type names are looked up. -- Andrei
Could restrict to imports that come before any other statements in the function body, ie: // valid, compiles when instantiated void fun(Range)(Range r) if(isInputRange!Range) { import std.range; } // invalid, import is not picked up because it is not the first statement in the method body void fun(Range)(Range r) if(isInputRange!Range) { doThings(); import std.range; } If this is desirable, the error message for failed symbol lookup of symbols in a function declaration should hint that imports in the function body are only considered if they stand at the beginning of the function, to help mitigate confusion. Seems simple enough to specify and explain, honestly.
To me it seems like adding additional rules to make up for a weakness that should be avoided in the first place. -- Andrei
Dec 14 2016
parent reply David Gileadi <gileadis NSPMgmail.com> writes:
On 12/14/16 9:27 AM, Andrei Alexandrescu wrote:
 On 12/14/2016 11:02 AM, default0 wrote:
 On Wednesday, 14 December 2016 at 14:21:55 UTC, Andrei Alexandrescu
 wrote:
 On 12/14/16 8:26 AM, Dominikus Dittes Scherkl wrote:
 On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu
 wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files
Why not leave it as it is and only change the compiler to perform inputs _within_ a function before evaluating the declaration, so that the symbols imported can be used in the declaration?
Thanks. I considered this but it put pressure on the relationship between the name and where it's looked up. Consider that imports in a function don't need to be at the top. They may also be in nested scopes inside the function. It becomes quite tenuous to explain where the parameter type names are looked up. -- Andrei
Could restrict to imports that come before any other statements in the function body, ie: // valid, compiles when instantiated void fun(Range)(Range r) if(isInputRange!Range) { import std.range; } // invalid, import is not picked up because it is not the first statement in the method body void fun(Range)(Range r) if(isInputRange!Range) { doThings(); import std.range; } If this is desirable, the error message for failed symbol lookup of symbols in a function declaration should hint that imports in the function body are only considered if they stand at the beginning of the function, to help mitigate confusion. Seems simple enough to specify and explain, honestly.
To me it seems like adding additional rules to make up for a weakness that should be avoided in the first place. -- Andrei
For the little that it's worth, my opinion is that it's easier to understand/teach the proposed rule above than the new syntax in DIP1005. This is true for me whether the proposed rule applies to all imports within the top scope of the template function/class/struct, or only to imports at the beginning of that scope. The above rule doesn't cover non-template function declarations like the `process` example in the DIP, however. Are they an important enough use case to justify new syntax?
Dec 14 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/2016 12:07 PM, David Gileadi wrote:
 The above rule doesn't cover non-template function declarations like the
 `process` example in the DIP, however. Are they an important enough use
 case to justify new syntax?
I suspect 90% of all uses will be straight definitions of template functions or template structs/classes. So by that estimate we should be in good shape. However, looking inside the definition in order to look up names in its declarations breaks the rule of least astonishment. Making the import part of the syntactical unit of the declaration seems to be the path of least resistance. Andrei
Dec 14 2016
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday, 14 December 2016 at 17:28:57 UTC, Andrei 
Alexandrescu wrote:
 On 12/14/2016 12:07 PM, David Gileadi wrote:
 The above rule doesn't cover non-template function 
 declarations like the
 `process` example in the DIP, however. Are they an important 
 enough use
 case to justify new syntax?
I suspect 90% of all uses will be straight definitions of template functions or template structs/classes. So by that estimate we should be in good shape. However, looking inside the definition in order to look up names in its declarations breaks the rule of least astonishment. Making the import part of the syntactical unit of the declaration seems to be the path of least resistance.
It also doesn't work with function prototypes. With the proposed syntax, you can do int foo(SysTime st) import(std.datetime); but if the compiler has to look into the function body to get at the import, then a prototype like this would be out of luck. - Jonathan M Davis
Dec 14 2016
parent reply Dominikus Dittes Scherkl <Dominikus.Scherkl continental-corporation.com> writes:
[about leaving the syntax as it is, only extend the import to 
include the previous (declaration) line]

On Wednesday, 14 December 2016 at 21:21:39 UTC, Jonathan M Davis 
wrote:
 It also doesn't work with function prototypes. With the 
 proposed syntax, you can do

     int foo(SysTime st) import(std.datetime);

 but if the compiler has to look into the function body to get 
 at the import, then a prototype like this would be out of luck.
You are right. I think this is the only shortcoming. But how does that work with imports within the function anyway? In this case you can't even today determine the dependencies of a function from it's prototype. Also I hate prototypes: .di-files are pretty useless - they make not clear what is needed to include and for templates you need the whole code anyway. I think we don't loose much if now we have even symbols within a .di-file which are not declared there. The only good thing about prototypes is: they can be auto-generated. We only need to modify this generation by adding all function-local includes to the top of the .di-file. This would also improve the usefulness of such files by having a whole list of dependencies of a library - but this has nothing to do with this change! This is necessary independently, because local imports are very common today.
Dec 14 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/15/2016 02:23 AM, Dominikus Dittes Scherkl wrote:
 Also I hate prototypes: .di-files are pretty useless - they make not
 clear what is needed to include and for templates you need the whole
 code anyway.
BTW are there outstanding issues with .di files? I recall some solid improvements were made a while ago. -- Andrei
Dec 19 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/14/2016 5:26 AM, Dominikus Dittes Scherkl wrote:
 On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files
Why not leave it as it is and only change the compiler to perform inputs _within_ a function before evaluating the declaration, so that the symbols imported can be used in the declaration? e.g. fun(Range x) if(isInputRange!x) { import std.range; auto a = x.front(); }
That would make it problematic to have function declarations.
Dec 15 2016
parent reply Dominikus Dittes Scherkl <Dominikus.Scherkl continental-corporation.com> writes:
On Thursday, 15 December 2016 at 16:16:51 UTC, Walter Bright 
wrote:
 On 12/14/2016 5:26 AM, Dominikus Dittes Scherkl wrote:
 Why not leave it as it is and only change the compiler to
 perform inputs _within_ a function before evaluating the 
 declaration, so that
 the symbols imported can be used in the declaration?

 e.g.

 fun(Range x) if(isInputRange!x)
 {
    import std.range;
    auto a = x.front();
 }
That would make it problematic to have function declarations.
Yeah, but declarations ARE already problematic, because local imports are not visible from them, so only having a declaration file (.di) at the moment already is not enough to determine the dependencies. And as I understand you, you don't plan to remove the local imports as they are now? (would be a really huge breaking change!) So, I no longer propose to change nothing except the internal compiler behaviour. Now I propose to additionally change the .di-file generation to also add all local imports to the start of the declaration file, so that having this file gives ALL dependencies of the declared stuff.
Dec 15 2016
next sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On Thursday, 15 December 2016 at 17:07:35 UTC, Dominikus Dittes 
Scherkl wrote:
 On Thursday, 15 December 2016 at 16:16:51 UTC, Walter Bright 
 wrote:
 On 12/14/2016 5:26 AM, Dominikus Dittes Scherkl wrote:
 Why not leave it as it is and only change the compiler to
 perform inputs _within_ a function before evaluating the 
 declaration, so that
 the symbols imported can be used in the declaration?

 e.g.

 fun(Range x) if(isInputRange!x)
 {
    import std.range;
    auto a = x.front();
 }
That would make it problematic to have function declarations.
So, I no longer propose to change nothing except the internal compiler behaviour. Now I propose to additionally change the .di-file generation to also add all local imports to the start of the declaration file, so that having this file gives ALL dependencies of the declared stuff.
This makes me think about the current .di generation. As far as I remember it only strips function bodies and leaves everything else intact. It could already try to remove unused imports. And in the future it could replace regular imports with auto-generated DCD-style imports.
Dec 15 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/15/2016 01:06 PM, Andrej Mitrovic wrote:
 On Thursday, 15 December 2016 at 17:07:35 UTC, Dominikus Dittes Scherkl
 wrote:
 On Thursday, 15 December 2016 at 16:16:51 UTC, Walter Bright wrote:
 On 12/14/2016 5:26 AM, Dominikus Dittes Scherkl wrote:
 Why not leave it as it is and only change the compiler to
 perform inputs _within_ a function before evaluating the
 declaration, so that
 the symbols imported can be used in the declaration?

 e.g.

 fun(Range x) if(isInputRange!x)
 {
    import std.range;
    auto a = x.front();
 }
That would make it problematic to have function declarations.
So, I no longer propose to change nothing except the internal compiler behaviour. Now I propose to additionally change the .di-file generation to also add all local imports to the start of the declaration file, so that having this file gives ALL dependencies of the declared stuff.
This makes me think about the current .di generation. As far as I remember it only strips function bodies and leaves everything else intact. It could already try to remove unused imports. And in the future it could replace regular imports with auto-generated DCD-style imports.
Please make this an issue. Thanks! -- Andrei
Dec 15 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/15/2016 9:07 AM, Dominikus Dittes Scherkl wrote:
 So, I no longer propose to change nothing except the internal compiler
behaviour.
 Now I propose to additionally change the .di-file generation to also add all
 local imports to the start of the declaration file, so that having this file
 gives ALL dependencies of the declared stuff.
That will subtly change the behavior of overloads across module lookups.
Dec 15 2016
parent Dominikus Dittes Scherkl <Dominikus.Scherkl continental-corporation.com> writes:
On Friday, 16 December 2016 at 07:15:51 UTC, Walter Bright wrote:
 On 12/15/2016 9:07 AM, Dominikus Dittes Scherkl wrote:
 So, I no longer propose to change nothing except the internal 
 compiler behaviour.
 Now I propose to additionally change the .di-file generation 
 to also add all
 local imports to the start of the declaration file, so that 
 having this file
 gives ALL dependencies of the declared stuff.
That will subtly change the behavior of overloads across module lookups.
Hm. Maybe if the .di-file would be used for code generation. But does that make sense? A .di-file is delivered together with an already compiled library to provide the sources of templates AND to show up all dependencies of the library, no? Both for the interested reader and the compiler. And the actual .di-files doesn't do that correct.
Dec 16 2016
prev sibling next sibling parent Iakh <iaktakh gmail.com> writes:
On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu 
wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei
How about something like Haskells "where" to allow any set of pre-declaration just for one declaration: with { import ...; alias Result = ...; } Result func(blah) {...}
Dec 14 2016
prev sibling next sibling parent Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu 
wrote:
 Destroy.
FWIW, the reason why I don't use inline imports in my code: http://d.puremagic.com/issues/show_bug.cgi?id=7016
Dec 14 2016
prev sibling next sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu 
wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei
How about: bool equal(R1, R2) : std.range if (isInputRange!R1 && isInputRange!R2) { ... } It's nice and concise, and you could in theory also allow multiple imports with a comma bool equal(R1, R2) : std.range, std.traits if (isInputRange!R1 && isInputRange!R2 && isArray!R2) { ... }
Dec 14 2016
parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Wednesday, 14 December 2016 at 17:01:50 UTC, Andrej Mitrovic 
wrote:
 On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei 
 Alexandrescu wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei
How about: bool equal(R1, R2) : std.range if (isInputRange!R1 && isInputRange!R2) { ... } It's nice and concise, and you could in theory also allow multiple imports with a comma bool equal(R1, R2) : std.range, std.traits if (isInputRange!R1 && isInputRange!R2 && isArray!R2) { ... }
breaks possible selective import. if both modules exports some symbol, we will need to selectively import and/or rename it.
Dec 14 2016
next sibling parent reply Anonymouse <asdf asdf.net> writes:
On Wednesday, 14 December 2016 at 17:09:44 UTC, ketmar wrote:
 On Wednesday, 14 December 2016 at 17:01:50 UTC, Andrej Mitrovic
 How about:

 bool equal(R1, R2) : std.range
 if (isInputRange!R1 && isInputRange!R2)
 { ... }

 It's nice and concise, and you could in theory also allow 
 multiple imports with a comma

 bool equal(R1, R2) : std.range, std.traits
 if (isInputRange!R1 && isInputRange!R2 && isArray!R2)
 { ... }
breaks possible selective import. if both modules exports some symbol, we will need to selectively import and/or rename it.
The with keyword then? bool equal(R1, R2) with (std.range : isInputRange, isOutputRange) && (std.stdio : writeln) && (std.algorithm) if (isInputRange!R1 && isInputRange!R2) { ... } A problem with that is that it reads "bool equal with std.stdio if something", suggesting something decides whether or not to import std.stdio.
Dec 14 2016
next sibling parent reply Anonymouse <asdf asdf.net> writes:
On Wednesday, 14 December 2016 at 17:24:29 UTC, Anonymouse wrote:
 The with keyword then?

 bool equal(R1, R2)
 with (std.range : isInputRange, isOutputRange) && (std.stdio : 
 writeln) && (std.algorithm)
 if (isInputRange!R1 && isInputRange!R2)
 { ... }
I guess the &&s there don't make sense. Maybe with a semicolon delimeter instead. bool equal(R1, R2) with (std.range : isInputRange, isOutputRange; std.stdio : writeln; std.algorithm) if (isInputRange!R1 && isInputRange!R2) { ... }
Dec 14 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/2016 12:28 PM, Anonymouse wrote:
 On Wednesday, 14 December 2016 at 17:24:29 UTC, Anonymouse wrote:
 The with keyword then?

 bool equal(R1, R2)
 with (std.range : isInputRange, isOutputRange) && (std.stdio :
 writeln) && (std.algorithm)
 if (isInputRange!R1 && isInputRange!R2)
 { ... }
I guess the &&s there don't make sense. Maybe with a semicolon delimeter instead. bool equal(R1, R2) with (std.range : isInputRange, isOutputRange; std.stdio : writeln; std.algorithm) if (isInputRange!R1 && isInputRange!R2) { ... }
This would not pass either. -- Andrei
Dec 14 2016
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/2016 12:24 PM, Anonymouse wrote:
 On Wednesday, 14 December 2016 at 17:09:44 UTC, ketmar wrote:
 On Wednesday, 14 December 2016 at 17:01:50 UTC, Andrej Mitrovic
 How about:

 bool equal(R1, R2) : std.range
 if (isInputRange!R1 && isInputRange!R2)
 { ... }

 It's nice and concise, and you could in theory also allow multiple
 imports with a comma

 bool equal(R1, R2) : std.range, std.traits
 if (isInputRange!R1 && isInputRange!R2 && isArray!R2)
 { ... }
breaks possible selective import. if both modules exports some symbol, we will need to selectively import and/or rename it.
The with keyword then? bool equal(R1, R2) with (std.range : isInputRange, isOutputRange) && (std.stdio : writeln) && (std.algorithm) if (isInputRange!R1 && isInputRange!R2) { ... }
The "import" keyword must be used. -- Andrei
Dec 14 2016
prev sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On Wednesday, 14 December 2016 at 17:09:44 UTC, ketmar wrote:
 bool equal(R1, R2) : std.range, std.traits
 if (isInputRange!R1 && isInputRange!R2 && isArray!R2)
 { ... }
breaks possible selective import. if both modules exports some symbol, we will need to selectively import and/or rename it.
Not necessarily. The first colon is the only special case, afterwards you can use all forms of imports:
 bool equal(R1, R2) : std.range : isInputRange, Trait = 
 std.traits
 if (isInputRange!R1 && isInputRange!R2 && Trait.isArray!R2)
 { ... }
Dec 14 2016
next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Wed, Dec 14, 2016 at 05:24:42PM +0000, Andrej Mitrovic via Digitalmars-d
wrote:
 On Wednesday, 14 December 2016 at 17:09:44 UTC, ketmar wrote:
 bool equal(R1, R2) : std.range, std.traits
 if (isInputRange!R1 && isInputRange!R2 && isArray!R2)
 { ... }
breaks possible selective import. if both modules exports some symbol, we will need to selectively import and/or rename it.
Not necessarily. The first colon is the only special case, afterwards you can use all forms of imports:
 bool equal(R1, R2) : std.range : isInputRange, Trait = std.traits
 if (isInputRange!R1 && isInputRange!R2 && Trait.isArray!R2)
 { ... }
What about: /* Showing full declaration just for context */ bool myFunc(R1, R2)(R1 r1, R2 r2) import { std.range : isInputRange, std.traits : isNum = isNumeric } if (isInputRange!R1 && isInputRange!R2 && isNum!(ElementType!R1)) in { assert(someCondition!R1); } out(result) { assert(result == expectedResult(...)); } body { ... } T -- EMACS = Extremely Massive And Cumbersome System
Dec 14 2016
next sibling parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Wednesday, 14 December 2016 at 17:32:10 UTC, H. S. Teoh wrote:
 What about:

 	/* Showing full declaration just for context */
 	bool myFunc(R1, R2)(R1 r1, R2 r2)
 	import {
 		std.range : isInputRange,
 		std.traits : isNum = isNumeric
 	}
 	if (isInputRange!R1 && isInputRange!R2 &&
 	    isNum!(ElementType!R1))
 	in { assert(someCondition!R1); }
 	out(result) { assert(result == expectedResult(...)); }
 	body
 	{
 		...
 	}


 T
you won't stop me trying to write some code in import section then. hey, there are curly brackets, so it SHOULD accept code there! ;-)
Dec 14 2016
parent reply Timothee Cour via Digitalmars-d <digitalmars-d puremagic.com> writes:
What about simply this:


```
module foo;

{
import std.stdio;
void fun(File foo){}
}

{
import sd.range;
void foo(T) if(isInputRange!T){}
}

```



On Wed, Dec 14, 2016 at 9:50 AM, ketmar via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Wednesday, 14 December 2016 at 17:32:10 UTC, H. S. Teoh wrote:

 What about:

         /* Showing full declaration just for context */
         bool myFunc(R1, R2)(R1 r1, R2 r2)
         import {
                 std.range : isInputRange,
                 std.traits : isNum = isNumeric
         }
         if (isInputRange!R1 && isInputRange!R2 &&
             isNum!(ElementType!R1))
         in { assert(someCondition!R1); }
         out(result) { assert(result == expectedResult(...)); }
         body
         {
                 ...
         }


 T
you won't stop me trying to write some code in import section then. hey, there are curly brackets, so it SHOULD accept code there! ;-)
Dec 14 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/2016 02:22 PM, Timothee Cour via Digitalmars-d wrote:
 What about simply this:


 ```
 module foo;

 {
 import std.stdio;
 void fun(File foo){}
 }

 {
 import sd.range;
 void foo(T) if(isInputRange!T){}
 }

 ```
Walter proposed this as well. We found two problems with it: (a) it is not clear that the import is effected only if the name is looked up; (b) the common case is one import clause for one declaration, which makes the code awkward. Either you have an indirection level to start with: { import std.range; void foo(T) if(isInputRange!T) { ... one extra indent already ... } } or the horror of C++ namespaces: {import std.range; void foo(T) if(isInputRange!T) { ... } } The fact that C++ namespaces introduce scopes has been an unpleasant mistake. Virtually all coding standards I've seen make namespace-introduced scopes special in that they don't participate in the usual indentation rules. Even special emacs and vi rules have been devised for that. Something we can learn from. Andrei
Dec 14 2016
prev sibling next sibling parent reply Meta <jared771 gmail.com> writes:
On Wednesday, 14 December 2016 at 17:32:10 UTC, H. S. Teoh wrote:
 What about:

 	/* Showing full declaration just for context */
 	bool myFunc(R1, R2)(R1 r1, R2 r2)
 	import {
 		std.range : isInputRange,
 		std.traits : isNum = isNumeric
 	}
 	if (isInputRange!R1 && isInputRange!R2 &&
 	    isNum!(ElementType!R1))
 	in { assert(someCondition!R1); }
 	out(result) { assert(result == expectedResult(...)); }
 	body
 	{
 		...
 	}


 T
The fact that declarations like this WILL show up in Phobos or elsewhere if this proposal is implemented triggers my gag reflex. There's got to be a better way than tacking yet another pre-ambulatory clause to the function body.
Dec 14 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/2016 02:04 PM, Meta wrote:
 On Wednesday, 14 December 2016 at 17:32:10 UTC, H. S. Teoh wrote:
 What about:

     /* Showing full declaration just for context */
     bool myFunc(R1, R2)(R1 r1, R2 r2)
     import {
         std.range : isInputRange,
         std.traits : isNum = isNumeric
     }
     if (isInputRange!R1 && isInputRange!R2 &&
         isNum!(ElementType!R1))
     in { assert(someCondition!R1); }
     out(result) { assert(result == expectedResult(...)); }
     body
     {
         ...
     }


 T
The fact that declarations like this WILL show up in Phobos or elsewhere if this proposal is implemented triggers my gag reflex. There's got to be a better way than tacking yet another pre-ambulatory clause to the function body.
What I see here is a function definition that has: * parameters and return types * dependencies on other modules * the static condition under which arguments are acceptable * the dynamic condition under which arguments are acceptable * the guarantee made upon successful completion * the implementation ... each of which is: * easily distinguishable * opt-in Andrei
Dec 14 2016
next sibling parent reply ArturG <var.spool.mail700 gmail.com> writes:
On Wednesday, 14 December 2016 at 19:39:55 UTC, Andrei 
Alexandrescu wrote:
 On 12/14/2016 02:04 PM, Meta wrote:
 On Wednesday, 14 December 2016 at 17:32:10 UTC, H. S. Teoh 
 wrote:
 What about:

     /* Showing full declaration just for context */
     bool myFunc(R1, R2)(R1 r1, R2 r2)
     import {
         std.range : isInputRange,
         std.traits : isNum = isNumeric
     }
     if (isInputRange!R1 && isInputRange!R2 &&
         isNum!(ElementType!R1))
     in { assert(someCondition!R1); }
     out(result) { assert(result == expectedResult(...)); }
     body
     {
         ...
     }
as ketmar said, would the import block have any restrictions what code could be used inside, would this work? T1 fun(T1, T2)(T1 t1, T2 t2) import { version(A) { someMod: T1, T2; } else { someOtherMod: T1, T2; } // static if or any other code? } { T1 ret = t1 + t2; return ret; }
Dec 14 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/2016 04:47 PM, ArturG wrote:
 On Wednesday, 14 December 2016 at 19:39:55 UTC, Andrei Alexandrescu wrote:
 On 12/14/2016 02:04 PM, Meta wrote:
 On Wednesday, 14 December 2016 at 17:32:10 UTC, H. S. Teoh wrote:
 What about:

     /* Showing full declaration just for context */
     bool myFunc(R1, R2)(R1 r1, R2 r2)
     import {
         std.range : isInputRange,
         std.traits : isNum = isNumeric
     }
     if (isInputRange!R1 && isInputRange!R2 &&
         isNum!(ElementType!R1))
     in { assert(someCondition!R1); }
     out(result) { assert(result == expectedResult(...)); }
     body
     {
         ...
     }
as ketmar said, would the import block have any restrictions what code could be used inside, would this work? T1 fun(T1, T2)(T1 t1, T2 t2) import { version(A) { someMod: T1, T2; } else { someOtherMod: T1, T2; } // static if or any other code? } { T1 ret = t1 + t2; return ret; }
The acceptability of the proposal decays exponentially with its deviation from existing import syntax. -- Andrei
Dec 14 2016
parent reply arturg <var.spool.mail700 gmail.com> writes:
On Wednesday, 14 December 2016 at 22:56:54 UTC, Andrei 
Alexandrescu wrote:
 The acceptability of the proposal decays exponentially with its 
 deviation from existing import syntax. -- Andrei
sorry, i missed the import keyword :/ T1 fun(T1, T2)(T1 t1, T2 t2) import { version(A) { import someMod: T1, T2; } else { import someOtherMod: T1, T2;} import std.stdio; // this import block would support only import statemens, version and static if etc... } { t2.writeln; // use symbols imported by the import block T1 ret = t1 + t2; return ret; }
Dec 14 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/16 6:46 PM, arturg wrote:
 On Wednesday, 14 December 2016 at 22:56:54 UTC, Andrei Alexandrescu wrote:
 The acceptability of the proposal decays exponentially with its
 deviation from existing import syntax. -- Andrei
sorry, i missed the import keyword :/ T1 fun(T1, T2)(T1 t1, T2 t2) import { version(A) { import someMod: T1, T2; } else { import someOtherMod: T1, T2;} import std.stdio; // this import block would support only import statemens, version and static if etc... }
I wrote the proposal and I have difficulty understanding what's meant here. Imagine us inflicting this upon our users. -- Andrei
Dec 14 2016
prev sibling parent Meta <jared771 gmail.com> writes:
On Wednesday, 14 December 2016 at 19:39:55 UTC, Andrei 
Alexandrescu wrote:
 On 12/14/2016 02:04 PM, Meta wrote:
 On Wednesday, 14 December 2016 at 17:32:10 UTC, H. S. Teoh 
 wrote:
 What about:

     /* Showing full declaration just for context */
     bool myFunc(R1, R2)(R1 r1, R2 r2)
     import {
         std.range : isInputRange,
         std.traits : isNum = isNumeric
     }
     if (isInputRange!R1 && isInputRange!R2 &&
         isNum!(ElementType!R1))
     in { assert(someCondition!R1); }
     out(result) { assert(result == expectedResult(...)); }
     body
     {
         ...
     }


 T
The fact that declarations like this WILL show up in Phobos or elsewhere if this proposal is implemented triggers my gag reflex. There's got to be a better way than tacking yet another pre-ambulatory clause to the function body.
What I see here is a function definition that has: * parameters and return types * dependencies on other modules * the static condition under which arguments are acceptable * the dynamic condition under which arguments are acceptable * the guarantee made upon successful completion * the implementation ... each of which is: * easily distinguishable * opt-in Andrei
Don't forget "looks awful and cluttered as hell".
Dec 14 2016
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/2016 12:32 PM, H. S. Teoh via Digitalmars-d wrote:
 	import {
 		std.range : isInputRange,
 		std.traits : isNum = isNumeric
 	}
This seems ambiguous. In import { a : b, c.d } did you meant to import c.d from a, or a different module? Andrei
Dec 14 2016
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Wed, Dec 14, 2016 at 02:30:29PM -0500, Andrei Alexandrescu via Digitalmars-d
wrote:
 On 12/14/2016 12:32 PM, H. S. Teoh via Digitalmars-d wrote:
 	import {
 		std.range : isInputRange,
 		std.traits : isNum = isNumeric
 	}
This seems ambiguous. In import { a : b, c.d } did you meant to import c.d from a, or a different module?
[...] That's easy to fix: import { a : b, c; d: e, f; } T -- There is no gravity. The earth sucks.
Dec 14 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/2016 03:33 PM, H. S. Teoh via Digitalmars-d wrote:
 That's easy to fix:

 	import { a : b, c;  d: e, f; }
The cost being adding new syntax. -- Andrei
Dec 14 2016
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Wed, Dec 14, 2016 at 04:08:56PM -0500, Andrei Alexandrescu via Digitalmars-d
wrote:
 On 12/14/2016 03:33 PM, H. S. Teoh via Digitalmars-d wrote:
 That's easy to fix:
 
 	import { a : b, c;  d: e, f; }
The cost being adding new syntax. -- Andrei
Aren't we already adding new syntax? On thinking about this again, though, isn't the whole point of this DIP to add a way of attaching imports X, Y, Z to some declaration such that the imports are in effect for the scope of the declaration? The specific syntax is really secondary. I think we should rather be more concerned about the semantics of it, such as: (1) The scope of the imports: I assume the import would be in effect until the end of the function body, correct? Or does it only last until the beginning of the function body? (a) Are DCD-imported symbols accessible in in- and out-contracts? (b) Are DCD imports permitted in the definition of inner functions? What is their effect, if allowed? Will symbols thus imported be restricted to the body of the inner function, or will it be visible throughout the outer function body? (2) How overloads are handled: (a) If the current module declares symbol X, and a DCD import pulls in another module that declares X, which X will it resolve to when the declaration references X? Are there cases of possible symbol hijacking here? What if the function body also declares a local symbol X? (b) If there are multiple DCD imports that declare symbol X, how would overload resolution work? (c) How are parameter symbols figured into overload resolution? E.g., if the DCD import pulls in module M that declares a module global xyz, and one of the function parameter names is also xyz. (d) Will symbols in a DCD imported module that aren't specifically imported shadow module globals, parameters, or local variables? E.g., you import my.module : X, but my.module also declares Y, and there's a {module global (in the current module), parameter, local variable} called Y. How would overload resolution work in this case? (4) Are DCD imports restricted to function declarations, or template declarations, or all declarations in general? (a) I.e., is it possible to declare a module global of type X, where X is defined by another module pulled in via a DCD import? (b) What about the other eponymous template shorthands, like `enum E(X) = ...`: will it be possible to attach DCD imports to that? E.g., if we want to say isInputRange!X in the enum definition. (c) Is it possible to attach DCD imports to delegate / lambda definitions? If so, what is their scope / interaction with the containing scope? (5) Is it possible to attach DCD imports to individual parameters? E.g.: auto myFunc(import.std.stdio.File f) { ... } or: auto myFunc(scope void delegate(import.std.stdio.File f) dg) { ... } Or do such imports have to be attached to the top-level declaration? T -- Computerese Irregular Verb Conjugation: I have preferences. You have biases. He/She has prejudices. -- Gene Wirchenko
Dec 14 2016
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/16 6:11 PM, H. S. Teoh via Digitalmars-d wrote:
 On Wed, Dec 14, 2016 at 04:08:56PM -0500, Andrei Alexandrescu via
Digitalmars-d wrote:
 On 12/14/2016 03:33 PM, H. S. Teoh via Digitalmars-d wrote:
 That's easy to fix:

 	import { a : b, c;  d: e, f; }
The cost being adding new syntax. -- Andrei
Aren't we already adding new syntax?
That's not a Boolean. One may be adding a little or a lot.
 On thinking about this again, though, isn't the whole point of this DIP
 to add a way of attaching imports X, Y, Z to some declaration such that
 the imports are in effect for the scope of the declaration?
That is correct.
 The
 specific syntax is really secondary.
That is also correct, and thanks for bringing it up; discussion of syntax eclipse other aspects. I'll make a pass to integrate your points. Andrei
Dec 14 2016
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/2016 06:11 PM, H. S. Teoh via Digitalmars-d wrote:
 On thinking about this again, though, isn't the whole point of this DIP
 to add a way of attaching imports X, Y, Z to some declaration such that
 the imports are in effect for the scope of the declaration?  The
 specific syntax is really secondary.  I think we should rather be more
 concerned about the semantics of it, such as:
[snip] Thanks very much for your review. I just pushed a commit addressing it. -- Andrei
Dec 19 2016
prev sibling parent ketmar <ketmar ketmar.no-ip.org> writes:
On Wednesday, 14 December 2016 at 17:24:42 UTC, Andrej Mitrovic 
wrote:
 On Wednesday, 14 December 2016 at 17:09:44 UTC, ketmar wrote:
 bool equal(R1, R2) : std.range, std.traits
 if (isInputRange!R1 && isInputRange!R2 && isArray!R2)
 { ... }
breaks possible selective import. if both modules exports some symbol, we will need to selectively import and/or rename it.
Not necessarily. The first colon is the only special case, afterwards you can use all forms of imports:
 bool equal(R1, R2) : std.range : isInputRange, Trait = 
 std.traits
 if (isInputRange!R1 && isInputRange!R2 && Trait.isArray!R2)
 { ... }
i know that C school likes to add punctuation everywhere ;-), but i can't see why that is better than: bool equal(R1, R2) import(std.range : isInputRange, Trait = std.traits) if ... i think that the version with explicit keyword is easier to read.
Dec 14 2016
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/13/2016 05:33 PM, Andrei Alexandrescu wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files
I just realized there is a ddoc consequence to this: ddoc could generate the import list and make that available to the generated documentation. I'll add that to the DIP. -- Andrei
Dec 14 2016
prev sibling next sibling parent reply bitwise <bitwise.pvt gmail.com> writes:
On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu 
wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei
I think the DIP indicated you may have considered this, but why not just remove the requirement of 'static import ..' for using fully qualified names? struct Buffered(Range) if (std.range.isInputRange!Range) { import std.range; // ... } You could then simply qualify any parameters or constraints and use local imports for the body. Bit
Dec 14 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/2016 03:13 PM, bitwise wrote:
 On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei
I think the DIP indicated you may have considered this, but why not just remove the requirement of 'static import ..' for using fully qualified names? struct Buffered(Range) if (std.range.isInputRange!Range) { import std.range; // ... } You could then simply qualify any parameters or constraints and use local imports for the body.
We'd like people and simplistic tools such as grep to indicate quickly where imports are present. -- Andrei
Dec 14 2016
parent reply Seb <seb wilzba.ch> writes:
On Wednesday, 14 December 2016 at 20:16:29 UTC, Andrei 
Alexandrescu wrote:
 On 12/14/2016 03:13 PM, bitwise wrote:
 I think the DIP indicated you may have considered this, but 
 why not just
 remove the requirement of 'static import ..' for using fully 
 qualified
 names?

 struct Buffered(Range) if (std.range.isInputRange!Range)
 {
     import std.range;
     // ...
 }

 You could then simply qualify any parameters or constraints 
 and use
 local imports for the body.
We'd like people and simplistic tools such as grep to indicate quickly where imports are present. -- Andrei
From all ideas I read so far, this is my favorite. It's a really concise syntax. What is the use case for using "grep" to find all imports? Even for something simplistic as to find all unittest blocks, one already needs to use Dscanner, so I don't see why this is a problem. Just image this case for "grep": import std.algorithm, std.range, std.traits; Moreover for a list of all imports, there is already a dedicated flag in DMD (-deps).
Dec 14 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/2016 09:21 PM, Seb wrote:
 What is the use case for using "grep" to find all imports?
The main idea here is to use "import" for importing things. -- Andrei
Dec 14 2016
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Made a pass through the document integrating a lot of feedback and 
fleshing the proposal better:

https://github.com/andralex/DIPs/blob/155ff59984b26749af7830aeb172d3af2dae8cd7/DIPs/DIP1005.md

https://github.com/dlang/DIPs/pull/51/files


Andrei
Dec 14 2016
next sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On Wednesday, 14 December 2016 at 22:55:26 UTC, Andrei 
Alexandrescu wrote:
 Made a pass through the document integrating a lot of feedback 
 and fleshing the proposal better:

 https://github.com/andralex/DIPs/blob/155ff59984b26749af7830aeb172d3af2dae8cd7/DIPs/DIP1005.md

 https://github.com/dlang/DIPs/pull/51/files


 Andrei
I'd also mention that the advantage of this feature is for the human reader (and not just ddoc generators) as you can instantly tell where all the symbols are located which a function uses. It could save a lot of time for the reader when analyzing a function. For example, if we have a function like the following: ubyte[] readSomeBytes ( ) { return read(1024); } It's a non-trivial exercise for the reader to understand where the `read` symbol is coming from. It usually involves looking up the import list at the top of the module which is typically very long and contains unrelated imports as the DIP mentions. Sometimes you can use a symbol lookup shortcut in your editor, but with a name like `read` you'll likely get a dozen modules listed in the pop-up and you still couldn't immediately tell which one of the symbols this `read` call refers to. Someone will mention IDEs, but remember when you're looking at the code in another environment (say reviewing code on Github) you may not have quick point & click functionality to find the symbol declaration. That's another win for this DIP.
Dec 14 2016
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 15.12.2016 00:17, Andrej Mitrovic wrote:
 ubyte[] readSomeBytes ( )
 {
     return read(1024);
 }

 It's a non-trivial exercise for the reader to understand where the
 `read` symbol is coming from.
pragma(msg,fullyQualifiedName!read);
Dec 14 2016
next sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On Wednesday, 14 December 2016 at 23:38:19 UTC, Timon Gehr wrote:
 On 15.12.2016 00:17, Andrej Mitrovic wrote:
 ubyte[] readSomeBytes ( )
 {
     return read(1024);
 }

 It's a non-trivial exercise for the reader to understand where 
 the
 `read` symbol is coming from.
pragma(msg,fullyQualifiedName!read);
Right, which requires building!
Dec 14 2016
parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On Thursday, 15 December 2016 at 00:44:02 UTC, Andrej Mitrovic 
wrote:
 On Wednesday, 14 December 2016 at 23:38:19 UTC, Timon Gehr 
 wrote:
 On 15.12.2016 00:17, Andrej Mitrovic wrote:
 ubyte[] readSomeBytes ( )
 {
     return read(1024);
 }

 It's a non-trivial exercise for the reader to understand 
 where the
 `read` symbol is coming from.
pragma(msg,fullyQualifiedName!read);
Right, which requires building!
Also, this wouldn't work with overloads would it? :)
Dec 14 2016
prev sibling parent David Gileadi <gileadis NSPMgmail.com> writes:
On 12/14/16 4:38 PM, Timon Gehr wrote:
 On 15.12.2016 00:17, Andrej Mitrovic wrote:
 ubyte[] readSomeBytes ( )
 {
     return read(1024);
 }

 It's a non-trivial exercise for the reader to understand where the
 `read` symbol is coming from.
pragma(msg,fullyQualifiedName!read);
Or, for the IDE-addict, hover or control-click on `read`.
Dec 14 2016
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/2016 06:17 PM, Andrej Mitrovic wrote:
 On Wednesday, 14 December 2016 at 22:55:26 UTC, Andrei Alexandrescu wrote:
 Made a pass through the document integrating a lot of feedback and
 fleshing the proposal better:

 https://github.com/andralex/DIPs/blob/155ff59984b26749af7830aeb172d3af2dae8cd7/DIPs/DIP1005.md


 https://github.com/dlang/DIPs/pull/51/files


 Andrei
I'd also mention that the advantage of this feature is for the human reader (and not just ddoc generators) as you can instantly tell where all the symbols are located which a function uses. It could save a lot of time for the reader when analyzing a function. For example, if we have a function like the following: ubyte[] readSomeBytes ( ) { return read(1024); } It's a non-trivial exercise for the reader to understand where the `read` symbol is coming from. It usually involves looking up the import list at the top of the module which is typically very long and contains unrelated imports as the DIP mentions. Sometimes you can use a symbol lookup shortcut in your editor, but with a name like `read` you'll likely get a dozen modules listed in the pop-up and you still couldn't immediately tell which one of the symbols this `read` call refers to. Someone will mention IDEs, but remember when you're looking at the code in another environment (say reviewing code on Github) you may not have quick point & click functionality to find the symbol declaration. That's another win for this DIP.
Cool, added that. -- Andrei
Dec 19 2016
prev sibling parent reply Chris M <chrismohrfeld comcast.net> writes:
On Wednesday, 14 December 2016 at 22:55:26 UTC, Andrei 
Alexandrescu wrote:
 Made a pass through the document integrating a lot of feedback 
 and fleshing the proposal better:

 https://github.com/andralex/DIPs/blob/155ff59984b26749af7830aeb172d3af2dae8cd7/DIPs/DIP1005.md

 https://github.com/dlang/DIPs/pull/51/files


 Andrei
For the "with" keyword, why is the import statement kept? Seems unnecessary to keep it. void process(File input) import std.stdio; struct Buffered(Range) if (isInputRange!Range) with (std.range) { ... }
Dec 14 2016
parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Thursday, 15 December 2016 at 00:29:17 UTC, Chris M wrote:
 For the "with" keyword, why is the import statement kept? Seems 
 unnecessary to keep it.

 void process(File input) import std.stdio;
 struct Buffered(Range) if (isInputRange!Range)
 with (std.range)
 {
     ...
 }
to not block future uses of such `with` syntax for different purposes. and (which is more important for me, for example) to make greping easier. and to keep `with` semantics: it should introduce scope over *existing* things. explicit `import` brings the module to the existance. choose any options you like. ;-)
Dec 14 2016
parent reply Timothee Cour via Digitalmars-d <digitalmars-d puremagic.com> writes:
how about this:

variant 1 // currently legal D; just need to attach semantics

```
// applies to all below
 deps!({import std.stdio; pragma(lib, "curl"); }):

// applies to 1 below
 deps!({import std.range})
void fun(T)(isInputRange!T){} // depends on both deps

void fun2(File file){}  // depends on 1st deps ending with ':'

```

variant 2: //currently legal if not ending with ':' as above; less noisy
syntax
 deps{import std.stdio; pragma(lib, "curl"); }
 deps{import std.stdio; pragma(lib, "curl"); }: // applies to all below

Advantages:

* no new syntax (just new semantics)
* no nesting inside {}
* allows grouping multiple declarations under a single set of imports
* `grep import` will still work unlike many of the existing proposals
* can use existing traits to query for such imports (eg getSymbolsByUDA)
* allows a more general solution (eg also handles `pragma(lib, "curl");` if
we want that as well)




On Wed, Dec 14, 2016 at 4:54 PM, ketmar via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Thursday, 15 December 2016 at 00:29:17 UTC, Chris M wrote:

 For the "with" keyword, why is the import statement kept? Seems
 unnecessary to keep it.

 void process(File input) import std.stdio;
 struct Buffered(Range) if (isInputRange!Range)
 with (std.range)
 {
     ...
 }
to not block future uses of such `with` syntax for different purposes. and (which is more important for me, for example) to make greping easier. and to keep `with` semantics: it should introduce scope over *existing* things. explicit `import` brings the module to the existance. choose any options you like. ;-)
Dec 14 2016
next sibling parent Yuxuan Shui <yshuiv7 gmail.com> writes:
On Thursday, 15 December 2016 at 01:59:33 UTC, Timothee Cour 
wrote:
 how about this:

 variant 1 // currently legal D; just need to attach semantics

 ```
 // applies to all below
  deps!({import std.stdio; pragma(lib, "curl"); }):

 // applies to 1 below
  deps!({import std.range})
 void fun(T)(isInputRange!T){} // depends on both deps

 void fun2(File file){}  // depends on 1st deps ending with ':'

 ```

 variant 2: //currently legal if not ending with ':' as above; 
 less noisy
 syntax
  deps{import std.stdio; pragma(lib, "curl"); }
  deps{import std.stdio; pragma(lib, "curl"); }: // applies to 
 all below

 Advantages:

 * no new syntax (just new semantics)
 * no nesting inside {}
 * allows grouping multiple declarations under a single set of 
 imports
 * `grep import` will still work unlike many of the existing 
 proposals
 * can use existing traits to query for such imports (eg 
 getSymbolsByUDA)
 * allows a more general solution (eg also handles `pragma(lib, 
 "curl");` if
 we want that as well)
Really nice idea! Except it's still a little noisy. I would be really happy if this gets accepted...
Dec 14 2016
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/2016 08:59 PM, Timothee Cour via Digitalmars-d wrote:
 how about this:

 variant 1 // currently legal D; just need to attach semantics

 ```
 // applies to all below
  deps!({import std.stdio; pragma(lib, "curl"); }):
Thanks, added. -- Andrei
Dec 19 2016
prev sibling next sibling parent reply Meta <jared771 gmail.com> writes:
On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu 
wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei
What's the main goal with this proposal? Is it to introduce DCDs into the language or is it to clean up some warts of the local import pattern, i.e.: int doSomething(T)(T t) //std.range is lazily imported but { //can't be used in the signature import std.range; //... } struct Test(R) //Ditto { import std.algorithm; //etc... } I'm assuming it's the former, or there wouldn't be any talk about introducing any new syntax. Imports would just be made completely lazy and we wouldn't have to change the language at all otherwise. So other than the fact that you can move the declaration anywhere without breaking it due to missing imports, are the advantages that DCDs provide really worth it enough to introduce yet another clause that can be put on any declaration? Especially when we have another solution (lazy imports) that does exactly what you want but excludes the DCD aspect?
Dec 14 2016
next sibling parent deadalnix <deadalnix gmail.com> writes:
On Thursday, 15 December 2016 at 03:26:24 UTC, Meta wrote:
 On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei 
 Alexandrescu wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei
What's the main goal with this proposal? Is it to introduce DCDs into the language or is it to clean up some warts of the local import pattern, i.e.: int doSomething(T)(T t) //std.range is lazily imported but { //can't be used in the signature import std.range; //... } struct Test(R) //Ditto { import std.algorithm; //etc... } I'm assuming it's the former, or there wouldn't be any talk about introducing any new syntax. Imports would just be made completely lazy and we wouldn't have to change the language at all otherwise. So other than the fact that you can move the declaration anywhere without breaking it due to missing imports, are the advantages that DCDs provide really worth it enough to introduce yet another clause that can be put on any declaration? Especially when we have another solution (lazy imports) that does exactly what you want but excludes the DCD aspect?
I think there is a case to be made to use the import in the symbol's signature, which is currently not possible. Not sure it pays for itself to add a feature in the language just for this. The tooling case is moot IMO. The languages keeps being fubared with obtuse complexities, which makes tooling harder to write, and then this is used an an excuse to fubar it even more. This isn't leading to a good place. Now there are some ways I see this happen leveraging the 'with' creating a "with import" construct. void foo(R)(R r) if (isInputRange!R) with import std.range { // ... } The good thing here is that the import syntax doesn't need to be altered in any way. You can still do "with import foo.bar : buzz".
Dec 14 2016
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/2016 10:26 PM, Meta wrote:
 On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei
What's the main goal with this proposal?
If the document does not clarify that, please let me know of specifics. Thanks. -- Andrei
Dec 15 2016
parent reply Meta <jared771 gmail.com> writes:
On Thursday, 15 December 2016 at 13:49:26 UTC, Andrei 
Alexandrescu wrote:
 On 12/14/2016 10:26 PM, Meta wrote:
 On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei 
 Alexandrescu wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei
What's the main goal with this proposal?
If the document does not clarify that, please let me know of specifics. Thanks. -- Andrei
Well of course it is about DCDs, but I am talking more about the underlying motivation. The listed benefits of DCDs from the DIP: 1. Specifies dependencies at declaration level, not at module level. This allows reasoning about the dependency cost of declarations in separation instead of aggregated at module level. 2. Dependency-Carrying Declarations are easier to move around, making for simpler and faster refactorings. 3. Dependency-Carrying Declarations allow scalable template libraries... Distributing a template library in the form of Dependency-Carrying Declarations creates a scalable, pay-as-you-go setup. It seems to me that making all imports lazy accomplishes 3, which is the real prize. 1 and 2 are nice to have, but it doesn't seem to me that the added complexity of the inline import feature is worth it when the biggest bang for our buck is not 1 or 2, but 3. So my question is, what do you want? Is your primary goal to introduce DCDs into the language to allow for easier refactoring and having it made more clear which imports a declaration uses, or is it to make imports "pay as you go" such that a user only pays for what they actually use and it's more or less transparent to them? You argue in the DIP that DCDs give us all 3 so we don't have to choose, but as I said previously, DCDs will not give us the biggest ROI compared to the cost of the feature.
Dec 15 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/15/2016 09:31 AM, Meta wrote:
 On Thursday, 15 December 2016 at 13:49:26 UTC, Andrei Alexandrescu wrote:
 On 12/14/2016 10:26 PM, Meta wrote:
 On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei
What's the main goal with this proposal?
If the document does not clarify that, please let me know of specifics. Thanks. -- Andrei
Well of course it is about DCDs, but I am talking more about the underlying motivation. The listed benefits of DCDs from the DIP: 1. Specifies dependencies at declaration level, not at module level. This allows reasoning about the dependency cost of declarations in separation instead of aggregated at module level. 2. Dependency-Carrying Declarations are easier to move around, making for simpler and faster refactorings. 3. Dependency-Carrying Declarations allow scalable template libraries... Distributing a template library in the form of Dependency-Carrying Declarations creates a scalable, pay-as-you-go setup. It seems to me that making all imports lazy accomplishes 3, which is the real prize. 1 and 2 are nice to have, but it doesn't seem to me that the added complexity of the inline import feature is worth it when the biggest bang for our buck is not 1 or 2, but 3. So my question is, what do you want? Is your primary goal to introduce DCDs into the language to allow for easier refactoring and having it made more clear which imports a declaration uses, or is it to make imports "pay as you go" such that a user only pays for what they actually use and it's more or less transparent to them? You argue in the DIP that DCDs give us all 3 so we don't have to choose, but as I said previously, DCDs will not give us the biggest ROI compared to the cost of the feature.
Thanks for sharing your opinion. Mine (and the argument made by DIP1005) is that on balance packing dependencies together with declarations is worth the language addition. The document does specify the advantages and disadvantages of lazy imports, as follows: === * Full lazy `import`s. Assume all `import`s are lazy without any change in the language. That would allow an idiom by which libraries use fully qualified names everywhere, and the `import`s would be effected only when names are looked up. This would allow scalability but not the dependency-carrying aspect. === If that is misrepresenting things, please let me know so I can improve the DIP. Andrei
Dec 15 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/15/2016 6:53 AM, Andrei Alexandrescu wrote:
 The document does specify the advantages and disadvantages of lazy
 imports, as follows:

 ===
 * Full lazy `import`s. Assume all `import`s are lazy without any change in the
 language. That would allow an idiom by which libraries use fully qualified
names
 everywhere, and the `import`s would be effected only when names are looked up.
 This would allow scalability but not the dependency-carrying aspect.
 ===
That would be a massive breaking change.
Dec 15 2016
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/15/2016 11:11 AM, Walter Bright wrote:
 On 12/15/2016 6:53 AM, Andrei Alexandrescu wrote:
 The document does specify the advantages and disadvantages of lazy
 imports, as follows:

 ===
 * Full lazy `import`s. Assume all `import`s are lazy without any
 change in the
 language. That would allow an idiom by which libraries use fully
 qualified names
 everywhere, and the `import`s would be effected only when names are
 looked up.
 This would allow scalability but not the dependency-carrying aspect.
 ===
That would be a massive breaking change.
This may be a misunderstanding. The idiom would be opt-in so existing behavior will be preserved (albeit it won't benefit of the improvements). Andrei
Dec 15 2016
next sibling parent reply Timothee Cour via Digitalmars-d <digitalmars-d puremagic.com> writes:
Some more details on my proposa based on UDA:

// applies to next decl
 deps!({import std.algorithm;})
void test1(){}

// applies to a set of decls
 deps!({import std.stdio;}){
  void test2(){}
  void test3(){}
}

// applies to all following decls (':')
 deps!({import std.array;}):

// can specify other dependencies beyond imports and have arbitrary complex
logic:
 deps!({
  import std.range;
  static int[100] data2;
  version(linux){
    enum data1=import("foo");//string import
    pragma(lib, "curl");
  }
}):
void test4(){}

// Can alias some dependencies:
alias deps1=deps!({import std.algorithm;});

 deps1
void test4(){}

NOTE: the above code compiles if we add
`struct deps(T...){}`, but that logic would be implemented in the compiler.

Advantages:

* no new syntax (just new semantics)
* no nesting inside {} required, so no additional indentation
* allows grouping multiple declarations under a single set of imports
* `grep import` will still work unlike many of the existing proposals
* can use existing traits to query for such imports (eg getSymbolsByUDA)
* allows a more general solution (eg also handles `pragma(lib, "curl");` if
we want that as well)

(Note: ignore the `variant 2` i had described above)

On Thu, Dec 15, 2016 at 8:32 AM, Andrei Alexandrescu via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 12/15/2016 11:11 AM, Walter Bright wrote:

 On 12/15/2016 6:53 AM, Andrei Alexandrescu wrote:

 The document does specify the advantages and disadvantages of lazy
 imports, as follows:

 ===
 * Full lazy `import`s. Assume all `import`s are lazy without any
 change in the
 language. That would allow an idiom by which libraries use fully
 qualified names
 everywhere, and the `import`s would be effected only when names are
 looked up.
 This would allow scalability but not the dependency-carrying aspect.
 ===
That would be a massive breaking change.
This may be a misunderstanding. The idiom would be opt-in so existing behavior will be preserved (albeit it won't benefit of the improvements). Andrei
Dec 15 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/15/2016 02:22 PM, Timothee Cour via Digitalmars-d wrote:
 Some more details on my proposa based on UDA:

 // applies to next decl
  deps!({import std.algorithm;})
 void test1(){}

 // applies to a set of decls
  deps!({import std.stdio;}){
   void test2(){}
   void test3(){}
 }

 // applies to all following decls (':')
  deps!({import std.array;}):

 // can specify other dependencies beyond imports and have arbitrary
 complex logic:
  deps!({
   import std.range;
   static int[100] data2;
   version(linux){
     enum data1=import("foo");//string import
     pragma(lib, "curl");
   }
 }):
 void test4(){}

 // Can alias some dependencies:
 alias deps1=deps!({import std.algorithm;});

  deps1
 void test4(){}

 NOTE: the above code compiles if we add
 `struct deps(T...){}`, but that logic would be implemented in the compiler.
I now understand the idea, thank you. My question is, doesn't this take things too far? Earlier I wrote:
 The acceptability of the proposal decays exponentially with its
 deviation from existing import syntax.
Indeed adding less syntax is better, but that's not an absolute; the optimum isn't necessarily at the "zero syntax added" point. This is because there are several things to harmonize in language design, which naturally are in tension. Andrei
Dec 15 2016
parent reply ArturG <var.spool.mail700 gmail.com> writes:
On Thursday, 15 December 2016 at 19:52:50 UTC, Andrei 
Alexandrescu wrote:
 On 12/15/2016 02:22 PM, Timothee Cour via Digitalmars-d wrote:
 Some more details on my proposa based on UDA:

 ...
I now understand the idea, thank you. My question is, doesn't this take things too far? Earlier I wrote:
 The acceptability of the proposal decays exponentially with its
 deviation from existing import syntax.
Indeed adding less syntax is better, but that's not an absolute; the optimum isn't necessarily at the "zero syntax added" point. This is because there are several things to harmonize in language design, which naturally are in tension. Andrei
Something like Timothee Cour's deps proposal is interesting, because it adds the same options to DCD's as module level imports have. They can be applied to any symbol that supports udas which the current syntax from the dip doesnt. As he displayed, they can be used to group symbols together which makes it more dry, but that can also be a drawback when refactoring depending which version you use. And you dont have to support all shown features as that syntax is easier extendable. The initial version could support only import statements.
Dec 15 2016
parent reply Timothee Cour via Digitalmars-d <digitalmars-d puremagic.com> writes:
one more thing:

we can simplify further (while still having formatted looking code) with
!q{} instead of !() :

```
// applies to next decl
 deps!q{import std.algorithm;}
void test1(){}

// applies to a set of decls
 deps!q{import std.stdio;}{
  void test2(){}
  void test3(){}
}

// applies to all following decls (':')
 deps!q{import std.array;}:

// can specify other dependencies beyond imports and have arbitrary complex
logic:
 deps!q{
  import std.range;
  static int[100] data2;
  version(linux){
    enum data1=import("foo");//string import
    pragma(lib, "curl");
  }
}:
void test4(){}

// Can alias some dependencies:
alias deps1=deps!q{import std.algorithm;};

 deps1
void test4(){}
```



On Thu, Dec 15, 2016 at 3:46 PM, ArturG via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Thursday, 15 December 2016 at 19:52:50 UTC, Andrei Alexandrescu wrote:

 On 12/15/2016 02:22 PM, Timothee Cour via Digitalmars-d wrote:

 Some more details on my proposa based on UDA:

 ...

 I now understand the idea, thank you.
My question is, doesn't this take things too far? Earlier I wrote: The acceptability of the proposal decays exponentially with its
 deviation from existing import syntax.
Indeed adding less syntax is better, but that's not an absolute; the optimum isn't necessarily at the "zero syntax added" point. This is because there are several things to harmonize in language design, which naturally are in tension. Andrei
Something like Timothee Cour's deps proposal is interesting, because it adds the same options to DCD's as module level imports have. They can be applied to any symbol that supports udas which the current syntax from the dip doesnt. As he displayed, they can be used to group symbols together which makes it more dry, but that can also be a drawback when refactoring depending which version you use. And you dont have to support all shown features as that syntax is easier extendable. The initial version could support only import statements.
Dec 15 2016
parent Yuxuan Shui <yshuiv7 gmail.com> writes:
On Friday, 16 December 2016 at 00:53:14 UTC, Timothee Cour wrote:
 one more thing:

 we can simplify further (while still having formatted looking 
 code) with
 !q{} instead of !() :

 ```
 // applies to next decl
  deps!q{import std.algorithm;}
 void test1(){}

 // applies to a set of decls
  deps!q{import std.stdio;}{
   void test2(){}
   void test3(){}
 }

 // applies to all following decls (':')
  deps!q{import std.array;}:
q{} would suggest the argument is a string, right? Plus, why does deps have to be a template?
 // can specify other dependencies beyond imports and have 
 arbitrary complex
 logic:
  deps!q{
   import std.range;
   static int[100] data2;
   version(linux){
     enum data1=import("foo");//string import
     pragma(lib, "curl");
   }
 }:
This is unnecessarily powerful.
 void test4(){}

 // Can alias some dependencies:
 alias deps1=deps!q{import std.algorithm;};

  deps1
 void test4(){}
I would prefer my import alternative. If we make use of q{}, it would look fine too: import(q{std.file: File}) import(q{std.range: isInputRange}) void foo(T)(File x) if (isInputRange!T);
 ```



 On Thu, Dec 15, 2016 at 3:46 PM, ArturG via Digitalmars-d < 
 digitalmars-d puremagic.com> wrote:

 [...]
Dec 15 2016
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 12/15/2016 8:32 AM, Andrei Alexandrescu wrote:
 On 12/15/2016 11:11 AM, Walter Bright wrote:
 On 12/15/2016 6:53 AM, Andrei Alexandrescu wrote:
 The document does specify the advantages and disadvantages of lazy
 imports, as follows:

 ===
 * Full lazy `import`s. Assume all `import`s are lazy without any
 change in the
 language. That would allow an idiom by which libraries use fully
 qualified names
 everywhere, and the `import`s would be effected only when names are
 looked up.
 This would allow scalability but not the dependency-carrying aspect.
 ===
That would be a massive breaking change.
This may be a misunderstanding. The idiom would be opt-in so existing behavior will be preserved (albeit it won't benefit of the improvements).
If existing behavior is preserved, how can it lazily parse the imports?
Dec 15 2016
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Thursday, 15 December 2016 at 16:11:56 UTC, Walter Bright 
wrote:
 That would be a massive breaking change.
SDC do parse the module only when an identifier resolution reach top level, and then populate the module's top level symbol table without running any semantic analysis on any of its symbols. Symbol are analyzed on demand when they are used. I think this achieve something close enough to what Andrei was proposing and is not breaking. You may also want to look at https://www.youtube.com/watch?v=b_T-eCToX1I to see what clang's up to.
Dec 15 2016
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/15/16 4:48 PM, deadalnix wrote:
 On Thursday, 15 December 2016 at 16:11:56 UTC, Walter Bright wrote:
 That would be a massive breaking change.
SDC do parse the module only when an identifier resolution reach top level, and then populate the module's top level symbol table without running any semantic analysis on any of its symbols. Symbol are analyzed on demand when they are used. I think this achieve something close enough to what Andrei was proposing and is not breaking.
Yah, that would work (note we are not talking about breakage; perhaps you meant no language addition?). That would (a) miss the dependency carrying aspect and (b) would be suboptimal. The moment any unqualified symbol is looked up in a module, all modules it imports need to be loaded. This could be avoided by using consistently fully qualified symbols. The comment about alternatives in the DIP does mention that but without going into details. Let me know if you think they should be added. Andrei
Dec 15 2016
prev sibling next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 12/15/2016 1:48 PM, deadalnix wrote:
 On Thursday, 15 December 2016 at 16:11:56 UTC, Walter Bright wrote:
 That would be a massive breaking change.
SDC do parse the module only when an identifier resolution reach top level, and then populate the module's top level symbol table without running any semantic analysis on any of its symbols.
I have a hard time seeing in practice where one would ever find it not necessary to parse for symbols. Note that 'object' is implicitly imported, so just using 'size_t' means every top level import has to be parsed.
 Symbol are analyzed on demand when they are used.
I agree that is not breaking and actually is something I've planned to implement in dmd for a while.
Dec 15 2016
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 12/15/2016 1:48 PM, deadalnix wrote:
 You may also want to look at https://www.youtube.com/watch?v=b_T-eCToX1I to see
 what clang's up to.
Thanks. I see they've done much that Zortech C++ did to be fast, as well as Warp and dmd. dmd eliminates the need for lib and ar by building in the code for that, so object files do not have to be written then read. I've often thought that this should be done with the linker, too. I.e. have dmd generate an exe file directly.
Dec 15 2016
prev sibling next sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 12/13/16 11:33 PM, Andrei Alexandrescu wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei
On first it seems like an awesome idea. That solves ... but wait what? Thinking more about the problem at hand - I fail to see what this DIP accomplishes that can't be done better today. 1. The benefit of placing import to each function is based on the untold assumption that we have: a) huge modules with many functions b) that constraints of these functions require different (large) modules The reality is far from this picture - if anything 99% of template constraints are dependent on std.range.primitives and std.traits with a bit of std.meta from time to time. So we'd have a boilerplate of auto foo(R)(R range) (import std.range.primitives) if(isInputRange!R){ ... } everywhere for no noticeable benefit - touch one of functions and you get full set of imports of these _small_ modules. 2. By itself the mechanism for delaying import even for constraint until the function is touched is moot as long as the module in question is huge and is not split in pieces. In other words: auto foo(R)(R range) (import std.range) if(isInputRange!R){ ... } Pulls in full std.range the moment foo is touched, compared to import std.range.primitives; ... auto foo(R)(R range) if(isInputRange!R){ ... } which is because it actually isolates the whole mess of complete std.range from the user of a template. All in all my practical response is split the modules at least in 2 parts: constraints + full functionality, then import the one with constraints at the top level. Works today and solves the practical issues unlike the proposal. --- Dmitry Olshansky
Dec 15 2016
next sibling parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Thursday, 15 December 2016 at 22:56:42 UTC, Dmitry Olshansky 
wrote:
 On 12/13/16 11:33 PM, Andrei Alexandrescu wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei
On first it seems like an awesome idea. That solves ... but wait what? Thinking more about the problem at hand - I fail to see what this DIP accomplishes that can't be done better today. 1. The benefit of placing import to each function is based on the untold assumption that we have: a) huge modules with many functions b) that constraints of these functions require different (large) modules The reality is far from this picture - if anything 99% of template constraints are dependent on std.range.primitives and std.traits with a bit of std.meta from time to time. So we'd have a boilerplate of auto foo(R)(R range) (import std.range.primitives) if(isInputRange!R){ ... } everywhere for no noticeable benefit - touch one of functions and you get full set of imports of these _small_ modules. 2. By itself the mechanism for delaying import even for constraint until the function is touched is moot as long as the module in question is huge and is not split in pieces. In other words: auto foo(R)(R range) (import std.range) if(isInputRange!R){ ... } Pulls in full std.range the moment foo is touched, compared to import std.range.primitives; ... auto foo(R)(R range) if(isInputRange!R){ ... } which is because it actually isolates the whole mess of complete std.range from the user of a template. All in all my practical response is split the modules at least in 2 parts: constraints + full functionality, then import the one with constraints at the top level. Works today and solves the practical issues unlike the proposal. --- Dmitry Olshansky
+1 I understand the motivation for the proposal but I think that good practice with package/module organisation renders the benefits marginal. The flipside of that is that introducing this new syntax reduces the pressure on people to organise their code sensibly. Is it a feature I'd like to be able to use when I'm writing: yes Is it a feature I actually want to see in D and in the D codebases I read daily: probably no, meh Is it worth the time and effort of core developers and community reviewers, both now and as an ongoing feature-maintenance burden: absolutely not. I have no idea why this proposal is a priority right now.
Dec 16 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/16/2016 04:58 AM, John Colvin wrote:
 Is it worth the time and effort of core developers and community
 reviewers, both now and as an ongoing feature-maintenance burden:
 absolutely not. I have no idea why this proposal is a priority right now.
Walter and I wanted to have a positive example of a good DIP that is relatively simple, is noncontroversial, and marks a definite move forward. We believe the feature's benefits are significant, obvious, and immediate. With this feature, any large project would be able to radically clarify and improve its dependency structure with only little refactoring work. I interpret the negative feedback as failings of the DIP to explain and argue matters appropriately, so I'm working hard on improving it. Please keep feedback coming. Thanks, Andrei
Dec 16 2016
parent reply Jon Degenhardt <noreply noreply.com> writes:
On Friday, 16 December 2016 at 17:19:56 UTC, Andrei Alexandrescu 
wrote:
 Walter and I wanted to have a positive example of a good DIP 
 that is relatively simple, is noncontroversial, and marks a 
 definite move forward.

 We believe the feature's benefits are significant, obvious, and 
 immediate. With this feature, any large project would be able 
 to radically clarify and improve its dependency structure with 
 only little refactoring work. I interpret the negative feedback 
 as failings of the DIP to explain and argue matters 
 appropriately, so I'm working hard on improving it. Please keep 
 feedback coming.
A suggestion: Now that the proposal details are largely fleshed out, take a file from phobos and rewrite it in the new style. Should give insight into aesthetics and identify remaining issues to resolve. --Jon
Dec 16 2016
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/16/2016 01:24 PM, Jon Degenhardt wrote:
 A suggestion: Now that the proposal details are largely fleshed out,
 take a file from phobos and rewrite it in the new style. Should give
 insight into aesthetics and identify remaining issues to resolve.
What a great idea. For starters, I attempted the simple exercise of converting all module-level imports of std.array into static imports. I stopped here: https://github.com/dlang/phobos/pull/4962 Soon the naive approach "let's replace this import with a static import and rebuild" ran into the midst of a comedy of errors. Code in various other modules fails to compile. Sometimes (happy case) the error indicates the symbol that is not seen, which is easy to fix. Other times (as you may see if you try to touch the PR above) there is zero indication on where the problem originates. I invite anyone who believes DIP1005 is solving just a minor problem and/or that we can rein in by using lazy imports and no improvement to the language, to try their hand at continuing the PR above or starting a similar effort. Currently it is exhaustingly difficult to figure where names are looked up in a large D project. Andrei
Dec 16 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 12/16/2016 11:59 AM, Andrei Alexandrescu wrote:
 Soon the naive approach "let's replace this import with a static import and
 rebuild" ran into the midst of a comedy of errors. Code in various other
modules
 fails to compile. Sometimes (happy case) the error indicates the symbol that is
 not seen, which is easy to fix. Other times (as you may see if you try to touch
 the PR above) there is zero indication on where the problem originates.
The switch: -verrors=spec is very useful here. It prints the errors otherwise gagged when inside __traits(compiles,...).
Dec 17 2016
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/16/2016 01:24 PM, Jon Degenhardt wrote:
 On Friday, 16 December 2016 at 17:19:56 UTC, Andrei Alexandrescu wrote:
 Walter and I wanted to have a positive example of a good DIP that is
 relatively simple, is noncontroversial, and marks a definite move
 forward.

 We believe the feature's benefits are significant, obvious, and
 immediate. With this feature, any large project would be able to
 radically clarify and improve its dependency structure with only
 little refactoring work. I interpret the negative feedback as failings
 of the DIP to explain and argue matters appropriately, so I'm working
 hard on improving it. Please keep feedback coming.
A suggestion: Now that the proposal details are largely fleshed out, take a file from phobos and rewrite it in the new style. Should give insight into aesthetics and identify remaining issues to resolve.
I also submit this as evidence: https://github.com/dlang/phobos/pull/4963 This was a successful attempt to convert all imports to selective imports in std.array. It was a very difficult process that took a long time, as shown in the comment. Andrei
Dec 16 2016
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/15/2016 05:56 PM, Dmitry Olshansky wrote:
 On 12/13/16 11:33 PM, Andrei Alexandrescu wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei
On first it seems like an awesome idea. That solves ... but wait what? Thinking more about the problem at hand - I fail to see what this DIP accomplishes that can't be done better today.
[snip] Thanks for this great feedback. I have improved the DIP to integrate it: Commit: https://github.com/dlang/DIPs/pull/51/commits/8f86c5add037c2bfb519dbcd646d1946b0e96571 Current file: https://github.com/dlang/DIPs/pull/51/files View: https://github.com/dlang/DIPs/blob/8f86c5add037c2bfb519dbcd646d1946b0e96571/DIPs/DIP1005.md Andrei
Dec 16 2016
parent reply Chris Wright <dhasenan gmail.com> writes:
Just looking at this again:

 The obvious workaround to the problem that dependencies must be module-
 level is to simply define many small modules---in the extreme, one per
 declaration.
Andrei works in phobos a lot. Phobos has a lot of large modules. For instance, std.datetime is 35,000 lines. It's not unusual for a phobos module to have over 6,000 lines (std.math, std.typecons, std.traits, std.format, std.conv). I'd normally recommend breaking up modules at one fifth that size. The standard library benefits from low granularity modules. It needs to implement a variety of related tools for working with particular things. For the hunting-for-definitions case, you also need: * a module with more than a few imports, from different libraries or packages * ambiguous names, or functions that are widely used * the user can't use an IDE / ctags / dcd * the user can't use ddox / dpldocs.info, which turns type references into links; or the user is using that and needs to find the definition of a template constraint * the maintainer cannot use selective imports * the maintainer cannot break the module up to reduce the number of dependencies * the maintainer is willing to spend the effort to convert top-level imports into tightly scoped imports For the compilation-speed case, you need: * large dependencies that this allows you to skip (the module combines several types of functionality with different dependencies) * the imported module must be in another compilation unit (incremental compilation or a separate library) * the dependencies can't be used by any other module in the compilation unit * no selective imports * the module being compiled depends on something in the same scope That's a pretty marginal use case.
Dec 17 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/17/2016 02:34 PM, Chris Wright wrote:
 Just looking at this again:

 The obvious workaround to the problem that dependencies must be module-
 level is to simply define many small modules---in the extreme, one per
 declaration.
Andrei works in phobos a lot. Phobos has a lot of large modules. For instance, std.datetime is 35,000 lines. It's not unusual for a phobos module to have over 6,000 lines (std.math, std.typecons, std.traits, std.format, std.conv).
Let's take a look at that hypothesis. The example I chose randomly (and which turned to be a rat's nest of fuzzy dependencies) was std/array.d, clocking at 3585 lines. Then looking at the entire project: wc -l std/*.d std/{algorithm,container,digest,experimental,internal,net,range,regex}/**/*.d | sort --key=1 -n | cat -n This outputs the modules in the standard library (excluding those that are simple header translations), sorted by LoC, numbered. See result in http://paste.ofcode.org/Lc5xfcs8GqpT2cabApSSgk. That shows 137 modules, median length 903, average length 2055 --- including full documentation, unittests, and examples. These numbers seem quite reasonable and if anything compare favorably against other projects I've been on.
 I'd normally recommend breaking up modules at one fifth that size.
Yeah, std/datetime.d is a monster, from what I can tell owing to a rote and redundant way of handling unittesting. I didn't look at its dependencies, but I doubt they are special. I was quite vocal about breaking it up, but I got mellower with time since (a) someone measured its size without unittests and it was something like one order of magnitude smaller, and (b) there was really no more trouble using or maintaining it than with anything else in Phobos. I should also add that each large project has a couple of outliers like that. I even recall a switch of a couple thousand lines once :o).
 The
 standard library benefits from low granularity modules. It needs to
 implement a variety of related tools for working with particular things.

 For the hunting-for-definitions case, you also need:

 * a module with more than a few imports, from different libraries or
 packages
 * ambiguous names, or functions that are widely used
 * the user can't use an IDE / ctags / dcd
 * the user can't use ddox / dpldocs.info, which turns type references
 into links; or the user is using that and needs to find the definition of
 a template constraint
 * the maintainer cannot use selective imports
 * the maintainer cannot break the module up to reduce the number of
 dependencies
 * the maintainer is willing to spend the effort to convert top-level
 imports into tightly scoped imports

 For the compilation-speed case, you need:

 * large dependencies that this allows you to skip (the module combines
 several types of functionality with different dependencies)
 * the imported module must be in another compilation unit (incremental
 compilation or a separate library)
 * the dependencies can't be used by any other module in the compilation
 unit
 * no selective imports
 * the module being compiled depends on something in the same scope

 That's a pretty marginal use case.
Most of these have been the case with all C++ and D projects I've been involved with at Facebook. Please let me know what of this information I should include in the DIP to make it better. Thanks. Andrei
Dec 17 2016
next sibling parent reply Chris Wright <dhasenan gmail.com> writes:
On Sat, 17 Dec 2016 19:34:12 -0500, Andrei Alexandrescu wrote:
 Most of these have been the case with all C++ and D projects I've been
 involved with at Facebook.
I've had similar frustrations when using C# without an IDE (which is a problem because it encourages larger namespaces), and somewhat with Python (though much worse because of the lack of static typing). D doesn't have either of those pitfalls, so I haven't seen it cause problems. I'm also a bit skeptical that this will see much use outside phobos. This isn't really an argument against it. I just don't see any argument for it, not that's supported by my own experience.
 Please let me know what of this information I should include in the DIP
 to make it better. Thanks.
You don't mention improved documentation as an option for readability. As a general comment, the Rationale section takes a while to get to the benefits (and with it the associated problems that the DIP is trying to address). My advisor in college exhorted us to be *prefix-competitive* in our papers and talks: your audience is present at the start but will get bored and wander off at some random point, so make sure that you maximize the impact of the part of the message they hear. Every prefix of your talk should include as much value as you can pack into that many words.
Dec 17 2016
parent reply pineapple <meapineapple gmail.com> writes:
On Sunday, 18 December 2016 at 02:40:59 UTC, Chris Wright wrote:
 D doesn't have either of those pitfalls, so I haven't seen it 
 cause problems. I'm also a bit skeptical that this will see 
 much use outside phobos.

 This isn't really an argument against it. I just don't see any 
 argument for it, not that's supported by my own experience.
I would like to echo this sentiment. I am developing a general-use library for D that is currently resting at around 50,000 lines. I have never felt a need for a feature like this, and I can't imagine a reason to begin using it. Dependency management has just never presented an issue. Very nearly all modules in the library are fewer than 1,000 lines long and very nearly all symbols are selectively imported, and the approach has proven to be completely manageable. If it can be added without interfering with the existing patterns, I don't really have an argument against this feature. But I do think that what this DIP is meant to address is not really a problem experienced by all or even most who are working with D. It's a problem being experienced with Phobos, but there are very valid solutions to that problem that don't involve an addition to the language - only some refactoring. I think that makes the argument in favor somewhat weak.
Dec 17 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/17/16 10:21 PM, pineapple wrote:
 On Sunday, 18 December 2016 at 02:40:59 UTC, Chris Wright wrote:
 D doesn't have either of those pitfalls, so I haven't seen it cause
 problems. I'm also a bit skeptical that this will see much use outside
 phobos.

 This isn't really an argument against it. I just don't see any
 argument for it, not that's supported by my own experience.
I would like to echo this sentiment. I am developing a general-use library for D that is currently resting at around 50,000 lines.
Is the source code publicly available?
 I have never felt a need for a feature like this,
 and I can't imagine a reason to begin using it. Dependency management
 has just never presented an issue. Very nearly all modules in the
 library are fewer than 1,000 lines long and very nearly all symbols are
 selectively imported, and the approach has proven to be completely
 manageable.

 If it can be added without interfering with the existing patterns, I
 don't really have an argument against this feature. But I do think that
 what this DIP is meant to address is not really a problem experienced by
 all or even most who are working with D.
I recall Liran Zvibel repeatedly mentioned this as a big pain point at Weka.io. I will reach out to him. On what basis do you extrapolate from personal experience to most or all who are working with D?
 It's a problem being
 experienced with Phobos, but there are very valid solutions to that
 problem that don't involve an addition to the language - only some
 refactoring. I think that makes the argument in favor somewhat weak.
How is Phobos special? What kind of refactoring do you envision would improve dependency management and build times? Thanks, Andrei
Dec 18 2016
parent reply pineapple <meapineapple gmail.com> writes:
On Sunday, 18 December 2016 at 13:31:48 UTC, Andrei Alexandrescu 
wrote:
 On 12/17/16 10:21 PM, pineapple wrote:
 I am developing a general-use library for D that is currently 
 resting at
 around 50,000 lines.
Is the source code publicly available?
https://github.com/pineapplemachine/mach.d
Dec 18 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/18/2016 10:01 AM, pineapple wrote:
 On Sunday, 18 December 2016 at 13:31:48 UTC, Andrei Alexandrescu wrote:
 On 12/17/16 10:21 PM, pineapple wrote:
 I am developing a general-use library for D that is currently resting at
 around 50,000 lines.
Is the source code publicly available?
https://github.com/pineapplemachine/mach.d
The code looks clean, congrats. With your permission I'd like to give this library as an example in the "Workaround: Increasing Granularity of Modules" section. Please advise, thanks. -- Andrei
Dec 18 2016
parent reply pineapple <meapineapple gmail.com> writes:
On Sunday, 18 December 2016 at 21:09:46 UTC, Andrei Alexandrescu 
wrote:
 On 12/18/2016 10:01 AM, pineapple wrote:
 On Sunday, 18 December 2016 at 13:31:48 UTC, Andrei 
 Alexandrescu wrote:
 Is the source code publicly available?
https://github.com/pineapplemachine/mach.d
The code looks clean, congrats. With your permission I'd like to give this library as an example in the "Workaround: Increasing Granularity of Modules" section. Please advise, thanks. -- Andrei
No problem, feel free. Thank you for asking.
Dec 18 2016
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/18/2016 05:18 PM, pineapple wrote:
 On Sunday, 18 December 2016 at 21:09:46 UTC, Andrei Alexandrescu wrote:
 On 12/18/2016 10:01 AM, pineapple wrote:
 On Sunday, 18 December 2016 at 13:31:48 UTC, Andrei Alexandrescu wrote:
 Is the source code publicly available?
https://github.com/pineapplemachine/mach.d
The code looks clean, congrats. With your permission I'd like to give this library as an example in the "Workaround: Increasing Granularity of Modules" section. Please advise, thanks. -- Andrei
No problem, feel free. Thank you for asking.
Is there a simple command to e.g. unittest everything in the project? Also, is there a build process or it's all templated? -- Andrei
Dec 18 2016
parent pineapple <meapineapple gmail.com> writes:
On Sunday, 18 December 2016 at 22:31:34 UTC, Andrei Alexandrescu 
wrote:
 Is there a simple command to e.g. unittest everything in the 
 project? Also, is there a build process or it's all templated? 
 -- Andrei
There's no build process. To run tests, I compile the root `package.d` file with rdmd, including the -debug, -unittest, and --main flags, and -I"path/to/repo/mach.d". To compile the mach.sdl package, which is not currently imported by the root `package.d`, you would have to include a directory containing the requisite Derelict dependencies for the -I option.
Dec 18 2016
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/18/2016 05:18 PM, pineapple wrote:
 On Sunday, 18 December 2016 at 21:09:46 UTC, Andrei Alexandrescu wrote:
 On 12/18/2016 10:01 AM, pineapple wrote:
 On Sunday, 18 December 2016 at 13:31:48 UTC, Andrei Alexandrescu wrote:
 Is the source code publicly available?
https://github.com/pineapplemachine/mach.d
The code looks clean, congrats. With your permission I'd like to give this library as an example in the "Workaround: Increasing Granularity of Modules" section. Please advise, thanks. -- Andrei
No problem, feel free. Thank you for asking.
Great, thanks. Please take a look at the accuracy of the discussion. I expanded the "Workaround" section and moved it near the top. https://github.com/dlang/DIPs/pull/51 https://github.com/dlang/DIPs/blob/dd46252e820dce66df746540d7ab94e0b00a6505/DIPs/DIP1005.md Andrei
Dec 18 2016
next sibling parent pineapple <meapineapple gmail.com> writes:
On Sunday, 18 December 2016 at 23:18:27 UTC, Andrei Alexandrescu 
wrote:
 Great, thanks. Please take a look at the accuracy of the 
 discussion. I expanded the "Workaround" section and moved it 
 near the top.

 https://github.com/dlang/DIPs/pull/51

 https://github.com/dlang/DIPs/blob/dd46252e820dce66df746540d7ab94e0b00a6505/DIPs/DIP1005.md


 Andrei
It's reasonably accurate. The only thing I would point out is that few modules define only one public symbol; the median probably lies around 4 or 5. A few modules, most or all being in the traits package, define considerably more. The mach.traits.primitives module defines 28 public symbols, by my count. (Though that's still small stuff relative to Phobos.) I think it would be most accurate to word the sentence as:
 It is organized as such that each module contains a small 
 number of related declarations (such as canAdjoin, Adjoin, and 
 AdjoinFlat in module adjoin.d) along with documentation and 
 unit tests.
Dec 18 2016
prev sibling next sibling parent reply pineapple <meapineapple gmail.com> writes:
On Sunday, 18 December 2016 at 23:18:27 UTC, Andrei Alexandrescu 
wrote:
 Great, thanks. Please take a look at the accuracy of the 
 discussion. I expanded the "Workaround" section and moved it 
 near the top.
I would also like to register that while I respect your argument regarding scalability, I have personally found that a greater number of smaller files is easier to manage than a smaller number of larger files. Including for the 580,000+ line project I work on for a living.
Dec 18 2016
next sibling parent pineapple <meapineapple gmail.com> writes:
On Monday, 19 December 2016 at 00:44:14 UTC, pineapple wrote:
 On Sunday, 18 December 2016 at 23:18:27 UTC, Andrei 
 Alexandrescu wrote:
 Great, thanks. Please take a look at the accuracy of the 
 discussion. I expanded the "Workaround" section and moved it 
 near the top.
I would also like to register that while I respect your argument regarding scalability, I have personally found that a greater number of smaller files is easier to manage than a smaller number of larger files. Including for the 580,000+ line project I work on for a living.
(Granted, that old ugly codebase has a lot of problems of its own, and smaller files does not always mean small on the same scale that mach.d's files are small, but I can tell you from experience that once a single ~4,000 line module was broken down into many ~200 line modules, that code became a great deal easier to understand and to reason about and to maintain.)
Dec 18 2016
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/18/16 7:44 PM, pineapple wrote:
 On Sunday, 18 December 2016 at 23:18:27 UTC, Andrei Alexandrescu wrote:
 Great, thanks. Please take a look at the accuracy of the discussion. I
 expanded the "Workaround" section and moved it near the top.
I would also like to register that while I respect your argument regarding scalability, I have personally found that a greater number of smaller files is easier to manage than a smaller number of larger files. Including for the 580,000+ line project I work on for a living.
Is it publicly available? -- Andrei
Dec 18 2016
parent pineapple <meapineapple gmail.com> writes:
On Monday, 19 December 2016 at 00:54:13 UTC, Andrei Alexandrescu 
wrote:
 On 12/18/16 7:44 PM, pineapple wrote:
 On Sunday, 18 December 2016 at 23:18:27 UTC, Andrei 
 Alexandrescu wrote:
 Great, thanks. Please take a look at the accuracy of the 
 discussion. I
 expanded the "Workaround" section and moved it near the top.
I would also like to register that while I respect your argument regarding scalability, I have personally found that a greater number of smaller files is easier to manage than a smaller number of larger files. Including for the 580,000+ line project I work on for a living.
Is it publicly available? -- Andrei
Sadly not.
Dec 18 2016
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Sunday, 18 December 2016 at 23:18:27 UTC, Andrei Alexandrescu 
wrote:
 Great, thanks. Please take a look at the accuracy of the 
 discussion. I expanded the "Workaround" section and moved it 
 near the top.

 https://github.com/dlang/DIPs/pull/51

 https://github.com/dlang/DIPs/blob/dd46252e820dce66df746540d7ab94e0b00a6505/DIPs/DIP1005.md


 Andrei
What's wrong with the parentheseless version ? The DIP says it looks "out of place" but that doesn't strike me as a very good argument. IMO the version without ";" is the way to go, as it doesn't require to add a new syntax for imports. Identical things looking identical is valuable.
Dec 18 2016
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 19.12.2016 06:31, deadalnix wrote:
 On Sunday, 18 December 2016 at 23:18:27 UTC, Andrei Alexandrescu wrote:
 Great, thanks. Please take a look at the accuracy of the discussion. I
 expanded the "Workaround" section and moved it near the top.

 https://github.com/dlang/DIPs/pull/51

 https://github.com/dlang/DIPs/blob/dd46252e820dce66df746540d7ab94e0b00a6505/DIPs/DIP1005.md



 Andrei
What's wrong with the parentheseless version ? The DIP says it looks "out of place" but that doesn't strike me as a very good argument. IMO the version without ";" is the way to go, as it doesn't require to add a new syntax for imports. Identical things looking identical is valuable.
+1.
Dec 20 2016
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2016-12-18 01:34, Andrei Alexandrescu wrote:

 Yeah, std/datetime.d is a monster, from what I can tell owing to a rote
 and redundant way of handling unittesting. I didn't look at its
 dependencies, but I doubt they are special. I was quite vocal about
 breaking it up, but I got mellower with time since (a) someone measured
 its size without unittests and it was something like one order of
 magnitude smaller, and (b) there was really no more trouble using or
 maintaining it than with anything else in Phobos.
Most other languages don't have inline unit tests, which saves a lot of lines of code. Not sure if this is the case. But if we have unit tests that are more on the functional/integration side perhaps those should be moved to a separate file structure.
 I should also add that each large project has a couple of outliers like
 that. I even recall a switch of a couple thousand lines once :o).
Just because another project is worse doesn't mean we're in a good position. Rubocop, the major linter in the Ruby world, will complain if a class is more than 100 lines of code. I think that is on the extreme side but I think any module with more than 2000 lines of code is too big. -- /Jacob Carlborg
Dec 18 2016
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 18 December 2016 at 11:47:06 UTC, Jacob Carlborg wrote:
 I think any module with more than 2000 lines of code is too big.
newCTFE has more than 2000 lines of code and is still growing, There is no way it could be split up.
Dec 18 2016
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/18/16 6:47 AM, Jacob Carlborg wrote:
 On 2016-12-18 01:34, Andrei Alexandrescu wrote:

 Yeah, std/datetime.d is a monster, from what I can tell owing to a rote
 and redundant way of handling unittesting. I didn't look at its
 dependencies, but I doubt they are special. I was quite vocal about
 breaking it up, but I got mellower with time since (a) someone measured
 its size without unittests and it was something like one order of
 magnitude smaller, and (b) there was really no more trouble using or
 maintaining it than with anything else in Phobos.
Most other languages don't have inline unit tests, which saves a lot of lines of code.
How does the lack of the feature save lines? Doesn't it just move them elsewhere?
 Not sure if this is the case. But if we have unit tests that are more on
 the functional/integration side perhaps those should be moved to a
 separate file structure.

 I should also add that each large project has a couple of outliers like
 that. I even recall a switch of a couple thousand lines once :o).
Just because another project is worse doesn't mean we're in a good position.
The point was just that it's par for the course.
 Rubocop, the major linter in the Ruby world, will complain if a class is
 more than 100 lines of code.
Does that include full documentation and unittests?
 I think that is on the extreme side but I
 think any module with more than 2000 lines of code is too big.
Phobos' average file size 2055 is lines and median file size is 903 lines. Do you find these appropriate for your preferences, considering this size includes documentation and unittests? Would you find it helpful to use a tool that collapses these? Andrei
Dec 18 2016
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 12/18/2016 5:38 AM, Andrei Alexandrescu wrote:
 On 12/18/16 6:47 AM, Jacob Carlborg wrote:
 Rubocop, the major linter in the Ruby world, will complain if a class is
 more than 100 lines of code.
Does that include full documentation and unittests?
My complaint is more along the lines of aggregates that seem to have scores of member functions. It brings to mind Scott Meyers' article about kitchen sink aggregates: http://www.drdobbs.com/cpp/how-non-member-functions-improve-encapsu/184401197
Dec 18 2016
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2016-12-18 14:38, Andrei Alexandrescu wrote:

 How does the lack of the feature save lines? Doesn't it just move them
 elsewhere?
Yes, I was referring to the file containing the implementation.
 Does that include full documentation and unittests?
It does not count the documentation in those lines. It would count the unit tests because there's no special syntax for that, just regular Ruby code. But one would most likely not have the implementation and the unit tests in the same file.
 Phobos' average file size 2055 is lines and median file size is 903
 lines. Do you find these appropriate for your preferences, considering
 this size includes documentation and unittests?
I would prefer smaller, but I would also prefer to have the unit tests separately.
 Would you find it helpful to use a tool that collapses these?
It's more helpful, yes. I do have that but since signatures in D can be arbitrary complex I'm doubtful that there's a tool/editor can fold all signatures that are valid in D. I know that TextMate that I use cannot. Take this for example: typeof({ class Foo { void bar() { } } }) foo (); I know that that's highly unlikely to happen in the real world but it shows how problematic it is to fold D code. -- /Jacob Carlborg
Dec 19 2016
prev sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Thursday, 15 December 2016 at 22:56:42 UTC, Dmitry Olshansky 
wrote:
 On 12/13/16 11:33 PM, Andrei Alexandrescu wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei
On first it seems like an awesome idea. That solves ... but wait what? Thinking more about the problem at hand - I fail to see what this DIP accomplishes that can't be done better today. 1. The benefit of placing import to each function is based on the untold assumption that we have: a) huge modules with many functions b) that constraints of these functions require different (large) modules The reality is far from this picture - if anything 99% of template constraints are dependent on std.range.primitives and std.traits with a bit of std.meta from time to time. So we'd have a boilerplate of auto foo(R)(R range) (import std.range.primitives) if(isInputRange!R){ ... } everywhere for no noticeable benefit - touch one of functions and you get full set of imports of these _small_ modules. 2. By itself the mechanism for delaying import even for constraint until the function is touched is moot as long as the module in question is huge and is not split in pieces. In other words: auto foo(R)(R range) (import std.range) if(isInputRange!R){ ... } Pulls in full std.range the moment foo is touched, compared to import std.range.primitives; ... auto foo(R)(R range) if(isInputRange!R){ ... } which is because it actually isolates the whole mess of complete std.range from the user of a template. All in all my practical response is split the modules at least in 2 parts: constraints + full functionality, then import the one with constraints at the top level. Works today and solves the practical issues unlike the proposal. --- Dmitry Olshansky
I largely agree with Dmitry. Ilya refactored several Phobos modules to use scoped, selective imports much more, and I pitched in for some remaining imports in the largest modules, so that only these module-level imports remain, ie those necessary for symbols imported in template constraints: std.datetime - https://github.com/dlang/phobos/pull/4373/files std.uni - https://github.com/dlang/phobos/pull/4365/files std.string and std.traits - https://github.com/dlang/phobos/pull/4370/files When I first saw this DIP, like Dmitry I was happy that we could get rid of those too, but the more I see these horrible syntax suggestions for what is really a minor convenience, I changed my mind. std.datetime, the 35k line (17 kloc according to Dscanner) beast of phobos, only needs 20 or so symbols at module-scope. std.uni- 10k lines, 4.2 kloc- only needs 17 symbols, all from the three modules Dmitry mentioned. I don't think his workaround of splitting up modules is even needed for such a low amount of module-level imports. Maybe there are other issues having to do with symbol resolution and dependency encapsulation that are addressed by this DIP, ie the technical performance of the compiler rather than refactoring or code clarity, that I don't fully grasp, but from the first two points of the claimed benefits of DCDs, ie ease of reasoning about dependencies and refactoring code, I don't think this feature will come anywhere close to carrying its own weight. As for the third benefit having to do with scalable template libraries, I'm not sure I completely understand all the details there, but I wonder if those problems aren't an artifact of the way dmd works now rather than something that can't be fixed without this DIP.
Dec 18 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/18/16 1:03 PM, Joakim wrote:
 I largely agree with Dmitry. Ilya refactored several Phobos modules to
 use scoped, selective imports much more, and I pitched in for some
 remaining imports in the largest modules, so that only these
 module-level imports remain, ie those necessary for symbols imported in
 template constraints:

 std.datetime - https://github.com/dlang/phobos/pull/4373/files
 std.uni - https://github.com/dlang/phobos/pull/4365/files
 std.string and std.traits - https://github.com/dlang/phobos/pull/4370/files
Yah, there's been a lot of good work (Jack Stouffer did a lot as well IIRC) on pushing imports inside. The following searches should be relevant: git grep '^\(private \)\?import' | wc -l yields about 426 top-level import declarations. The number of indented imports is 4605: git grep ' *import\W' | wc -l So we're looking at 10% of imports being problematic. Sadly, they turn out to make things quite difficult for Phobos. Last time I looked at the module dependency graph it wasn't a lot better than it used to before scoped imports. (I don't have it handy... could anyone please produce it?)
 When I first saw this DIP, like Dmitry I was happy that we could get rid
 of those too, but the more I see these horrible syntax suggestions for
 what is really a minor convenience, I changed my mind.  std.datetime,
 the 35k line (17 kloc according to Dscanner) beast of phobos, only needs
 20 or so symbols at module-scope. std.uni- 10k lines, 4.2 kloc- only
 needs 17 symbols, all from the three modules Dmitry mentioned.  I don't
 think his workaround of splitting up modules is even needed for such a
 low amount of module-level imports.
This paragraph is a good example of a couple of counterarguments that I think point directly to flaws in the DIP: (1) The DIP uses Phobos as an example, so it is applicable mostly to Phobos. In fact, Phobos is among the systems that would benefit least from the DIP: it has only druntime as dependency, and is distributed in its entirety. Many projects out there list multiple dependencies and may have various building and distribution policies. The converse is to believe that working around a problem in Phobos would render the DIP less useful in general. (2) "I don't like the syntax, hence I don't like the feature." I see this as a good opportunity for tasteful design, not a downside of the feature.
 Maybe there are other issues having to do with symbol resolution and
 dependency encapsulation that are addressed by this DIP, ie the
 technical performance of the compiler rather than refactoring or code
 clarity, that I don't fully grasp, but from the first two points of the
 claimed benefits of DCDs, ie ease of reasoning about dependencies and
 refactoring code, I don't think this feature will come anywhere close to
 carrying its own weight.
Does the refactoring in https://github.com/dlang/phobos/pull/4962 make dependencies clearer? Are you e.g. clear on what you need in order to use Appender? Would you want to take this (or another) experiment to another module and see how it improves its dependency structure?
 As for the third benefit having to do with scalable template libraries,
 I'm not sure I completely understand all the details there, but I wonder
 if those problems aren't an artifact of the way dmd works now rather
 than something that can't be fixed without this DIP.
The DIP now dedicates an entire section to the pluses and minuses of lazy imports, and concludes that lazy imports would address scalability if carefully used. Andrei
Dec 18 2016
parent reply Joakim <dlang joakim.fea.st> writes:
On Sunday, 18 December 2016 at 18:42:36 UTC, Andrei Alexandrescu 
wrote:
 On 12/18/16 1:03 PM, Joakim wrote:
 I largely agree with Dmitry. Ilya refactored several Phobos 
 modules to
 use scoped, selective imports much more, and I pitched in for 
 some
 remaining imports in the largest modules, so that only these
 module-level imports remain, ie those necessary for symbols 
 imported in
 template constraints:

 std.datetime - https://github.com/dlang/phobos/pull/4373/files
 std.uni - https://github.com/dlang/phobos/pull/4365/files
 std.string and std.traits - 
 https://github.com/dlang/phobos/pull/4370/files
Yah, there's been a lot of good work (Jack Stouffer did a lot as well IIRC) on pushing imports inside. The following searches should be relevant: git grep '^\(private \)\?import' | wc -l yields about 426 top-level import declarations. The number of indented imports is 4605: git grep ' *import\W' | wc -l So we're looking at 10% of imports being problematic. Sadly, they turn out to make things quite difficult for Phobos. Last time I looked at the module dependency graph it wasn't a lot better than it used to before scoped imports. (I don't have it handy... could anyone please produce it?)
Why do you care _so_ much about the module dependency graph? To make this question concrete, let's look at an example, the std.array module you keep mentioning. This is what it looked like before Ilya scoped as many imports as he could: https://github.com/dlang/phobos/commit/3fcf723aa498b96de165361b5abb9d3450fdc069#diff-54cf8402b22024ae667d4048a5126f0e That was a mess, similar to opaque C/C++ code, 13 modules imported at module-scope were whittled down to 4. You just made those more specific in a commit related to this DIP, by listing the actual symbols selectively imported from those four modules: https://github.com/dlang/phobos/commit/e064d5664f92c4b2f0866c08f6d0290ba66825ed#diff-54cf8402b22024ae667d4048a5126f0e If I'm looking at the template constraints for any particular function and see a couple symbols I don't recognize, I don't think it's a big deal to find the symbols in that list at the top. In other words, D already allows you to scope most imports. I don't consider the dozen or two remaining symbols from templaint constraints and function arguments to provide much overhead. Rather, I consider the weight of this additional syntax, ie the cognitive overhead from having to remember and parse more syntax in my head, to be worse than the remaining dependency reasoning problem you're trying to solve: the cost outweights the benefit. Perhaps that's subjective and others may disagree. Now, there's also the question of purely technical benefits, like compilation speed or executable bloat. I looked at the latter a little last summer, after Ilya had cleaned up a lot of the standard library: http://forum.dlang.org/thread/gmjqfjoemwtvgqrtdsdr forum.dlang.org I found that commenting out a single scoped, selective import of "std.string: format" in std.utf led to a 5% decrease in executable size for "hello world." This is a problem with how dmd compiles or appends these module dependencies and would presumably still be there after this DIP, as you would not remove the dependency. I think scoped, selective imports have been great at hacking away at the module dependency graph, as you lay out. It is not clear what technical costs you see from the remaining few dependencies and if this DIP is the best way to remove them. I think you should explain why you want to untangle the remaining dependency graph, and consider if this DIP is really doing that much.
 When I first saw this DIP, like Dmitry I was happy that we 
 could get rid
 of those too, but the more I see these horrible syntax 
 suggestions for
 what is really a minor convenience, I changed my mind.  
 std.datetime,
 the 35k line (17 kloc according to Dscanner) beast of phobos, 
 only needs
 20 or so symbols at module-scope. std.uni- 10k lines, 4.2 
 kloc- only
 needs 17 symbols, all from the three modules Dmitry mentioned.
  I don't
 think his workaround of splitting up modules is even needed 
 for such a
 low amount of module-level imports.
This paragraph is a good example of a couple of counterarguments that I think point directly to flaws in the DIP: (1) The DIP uses Phobos as an example, so it is applicable mostly to Phobos. In fact, Phobos is among the systems that would benefit least from the DIP: it has only druntime as dependency, and is distributed in its entirety. Many projects out there list multiple dependencies and may have various building and distribution policies.
It is not clear how those alternate dependency, building and distribution policies change the picture. Perhaps you should cite one of those as an example.
 The converse is to believe that working around a problem in 
 Phobos would render the DIP less useful in general.
The argument is not that Phobos has "worked around" the problem, but that once you scope everything you can other than template constraints (which you have said should be standard operating procedure in the DIP), the problem is minimal.
 (2) "I don't like the syntax, hence I don't like the feature." 
 I see this as a good opportunity for tasteful design, not a 
 downside of the feature.
The syntax is bad, but even if were beautiful, I don't consider the addition itself to be worth it. Again, could be subjective.
 Maybe there are other issues having to do with symbol 
 resolution and
 dependency encapsulation that are addressed by this DIP, ie the
 technical performance of the compiler rather than refactoring 
 or code
 clarity, that I don't fully grasp, but from the first two 
 points of the
 claimed benefits of DCDs, ie ease of reasoning about 
 dependencies and
 refactoring code, I don't think this feature will come 
 anywhere close to
 carrying its own weight.
Does the refactoring in https://github.com/dlang/phobos/pull/4962 make dependencies clearer?
I'm fine with your selective import refactoring, which I linked above: https://github.com/dlang/phobos/pull/4963
 Are you e.g. clear on what you need in order to use Appender?
It is easy to see that it requires isDynamicArray and that is found in std.traits. It may internally rely on other symbols that are currently imported from module scope, ie symbols from template constraints inside the struct Appender that happen to also be used outside that struct and so are imported at the top of the module. If you really wanted to make that clear, you could scope those imports inside the struct too.
 Would you want to take this (or another) experiment to another 
 module and see how it improves its dependency structure?
Ilya and I have already done this, in the PRs I linked above. I was mildly disappointed back then that I could not get rid of the remaining module-level imports, because of template constraints. I was predisposed to favor doing something about it, but I don't think this DIP is worth it.
 As for the third benefit having to do with scalable template 
 libraries,
 I'm not sure I completely understand all the details there, 
 but I wonder
 if those problems aren't an artifact of the way dmd works now 
 rather
 than something that can't be fixed without this DIP.
The DIP now dedicates an entire section to the pluses and minuses of lazy imports, and concludes that lazy imports would address scalability if carefully used.
Yes, there might be other ways to lower the technical costs of the dependency graph too, such as the size issue I mentioned above.
Dec 18 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/19/2016 01:47 AM, Joakim wrote:
 Why do you care _so_ much about the module dependency graph?  To make
 this question concrete, let's look at an example, the std.array module
 you keep mentioning.
 This is what it looked like before Ilya scoped as
 many imports as he could:

 https://github.com/dlang/phobos/commit/3fcf723aa498b96de165361b5abb9d3450fdc069#diff-54cf8402b22024ae667d4048a5126f0e

 That was a mess, similar to opaque C/C++ code, 13 modules imported at
 module-scope were whittled down to 4.  You just made those more specific
 in a commit related to this DIP, by listing the actual symbols
 selectively imported from those four modules:

 https://github.com/dlang/phobos/commit/e064d5664f92c4b2f0866c08f6d0290ba66825ed#diff-54cf8402b22024ae667d4048a5126f0e


 If I'm looking at the template constraints for any particular function
 and see a couple symbols I don't recognize, I don't think it's a big
 deal to find the symbols in that list at the top.
It actually is. Some symbols in e.g. std.range are used in constraints, some others used in definitions; some in a UFCS manner. Some code runs during compilation, with errors gagged (as in __traits(compiles)) or not gagged. I don't know of an easy manual method to figure out who's who (which makes Vladimir's idea of tooling it with brute force awesome and scary at the same time). At any way, I don't see how I can use this tidbit to improve the DIP. Anything I could add to it to make it more compelling?
 In other words, D already allows you to scope most imports.  I don't
 consider the dozen or two remaining symbols from templaint constraints
 and function arguments to provide much overhead.  Rather, I consider the
 weight of this additional syntax, ie the cognitive overhead from having
 to remember and parse more syntax in my head, to be worse than the
 remaining dependency reasoning problem you're trying to solve: the cost
 outweights the benefit.  Perhaps that's subjective and others may disagree.
It is subjective and difficult to convert into action. Online review among folks who know next to nothing about each other does have its challenges. What happens here is I post a proposal for a problem that I believe is important for large D projects. And I get back "eh, that's not as important to me." At a traditional work place we'd know a lot about one another's strengths, specialization areas, and design sensibilities. Here, all I can do is look at your past PRs (that's why I emailed you asking for your github handle; I figure it's joakim-noah). This makes it difficult to act on "I don't buy it" feedback.
 Now, there's also the question of purely technical benefits, like
 compilation speed or executable bloat.  I looked at the latter a little
 last summer, after Ilya had cleaned up a lot of the standard library:

 http://forum.dlang.org/thread/gmjqfjoemwtvgqrtdsdr forum.dlang.org

 I found that commenting out a single scoped, selective import of
 "std.string: format" in std.utf led to a 5% decrease in executable size
 for "hello world."  This is a problem with how dmd compiles or appends
 these module dependencies and would presumably still be there after this
 DIP, as you would not remove the dependency.
That might be a related but distinct issue. Can you reproduce that experiment?
 I think scoped, selective imports have been great at hacking away at the
 module dependency graph, as you lay out.  It is not clear what technical
 costs you see from the remaining few dependencies and if this DIP is the
 best way to remove them.  I think you should explain why you want to
 untangle the remaining dependency graph, and consider if this DIP is
 really doing that much.
I've made a few more passes through the DIP, but throughout I assume a couple of things without stressing them too much - dependencies are important, and associating dependencies with declarations makes the dependency graph as fine and as precise it could get. I don't see a reasonable way to make it better or clearer. Should I add an introductory section on why dependencies are important? Andrei
Dec 19 2016
next sibling parent Timothee Cour via Digitalmars-d <digitalmars-d puremagic.com> writes:
Andrei: you can use blockquotes to make long lines wrap when quoting text:

https://github.com/adam-p/markdown-here/wiki/Markdown-Here-Cheatsheet#blockquotes

that way, we don't have to scroll everytime there's a quote, eg:

replace:
<tab>Such scaffolding is of course
with:
 Such scaffolding is of course
On Mon, Dec 19, 2016 at 8:04 PM, Andrei Alexandrescu via Digitalmars-d < digitalmars-d puremagic.com> wrote:
 On 12/19/2016 01:47 AM, Joakim wrote:

 Why do you care _so_ much about the module dependency graph?  To make
 this question concrete, let's look at an example, the std.array module
 you keep mentioning.
 This is what it looked like before Ilya scoped as
 many imports as he could:

 https://github.com/dlang/phobos/commit/3fcf723aa498b96de1653
 61b5abb9d3450fdc069#diff-54cf8402b22024ae667d4048a5126f0e

 That was a mess, similar to opaque C/C++ code, 13 modules imported at
 module-scope were whittled down to 4.  You just made those more specific
 in a commit related to this DIP, by listing the actual symbols
 selectively imported from those four modules:

 https://github.com/dlang/phobos/commit/e064d5664f92c4b2f0866
 c08f6d0290ba66825ed#diff-54cf8402b22024ae667d4048a5126f0e


 If I'm looking at the template constraints for any particular function
 and see a couple symbols I don't recognize, I don't think it's a big
 deal to find the symbols in that list at the top.
It actually is. Some symbols in e.g. std.range are used in constraints, some others used in definitions; some in a UFCS manner. Some code runs during compilation, with errors gagged (as in __traits(compiles)) or not gagged. I don't know of an easy manual method to figure out who's who (which makes Vladimir's idea of tooling it with brute force awesome and scary at the same time). At any way, I don't see how I can use this tidbit to improve the DIP. Anything I could add to it to make it more compelling? In other words, D already allows you to scope most imports. I don't
 consider the dozen or two remaining symbols from templaint constraints
 and function arguments to provide much overhead.  Rather, I consider the
 weight of this additional syntax, ie the cognitive overhead from having
 to remember and parse more syntax in my head, to be worse than the
 remaining dependency reasoning problem you're trying to solve: the cost
 outweights the benefit.  Perhaps that's subjective and others may
 disagree.
It is subjective and difficult to convert into action. Online review among folks who know next to nothing about each other does have its challenges. What happens here is I post a proposal for a problem that I believe is important for large D projects. And I get back "eh, that's not as important to me." At a traditional work place we'd know a lot about one another's strengths, specialization areas, and design sensibilities. Here, all I can do is look at your past PRs (that's why I emailed you asking for your github handle; I figure it's joakim-noah). This makes it difficult to act on "I don't buy it" feedback. Now, there's also the question of purely technical benefits, like
 compilation speed or executable bloat.  I looked at the latter a little
 last summer, after Ilya had cleaned up a lot of the standard library:

 http://forum.dlang.org/thread/gmjqfjoemwtvgqrtdsdr forum.dlang.org

 I found that commenting out a single scoped, selective import of
 "std.string: format" in std.utf led to a 5% decrease in executable size
 for "hello world."  This is a problem with how dmd compiles or appends
 these module dependencies and would presumably still be there after this
 DIP, as you would not remove the dependency.
That might be a related but distinct issue. Can you reproduce that experiment? I think scoped, selective imports have been great at hacking away at the
 module dependency graph, as you lay out.  It is not clear what technical
 costs you see from the remaining few dependencies and if this DIP is the
 best way to remove them.  I think you should explain why you want to
 untangle the remaining dependency graph, and consider if this DIP is
 really doing that much.
I've made a few more passes through the DIP, but throughout I assume a couple of things without stressing them too much - dependencies are important, and associating dependencies with declarations makes the dependency graph as fine and as precise it could get. I don't see a reasonable way to make it better or clearer. Should I add an introductory section on why dependencies are important? Andrei
Dec 19 2016
prev sibling next sibling parent Timothee Cour via Digitalmars-d <digitalmars-d puremagic.com> writes:
what about using `lazy` instead of `with`:

`with(import foo)`
=>
`lazy(import foo)`

advantages:
* avoids confusion regarding usual scoping rules of `with` ;
* conveys that the import is indeed lazy

Furthermore (regardless of which keyword is used), what about allowing `:`
```
// case 1
lazy(import foo)
void fun(){}

// case 2
lazy(import foo) {
  void fun(){}
}

// case 3 : this is new
lazy(import foo):
void fun1(){}
void fun2(){}
```

advantages:

* same behavior as other constructs which don't introduce a scope:
```
// case 1, 2 3 are allowed:
version(A):
static if(true):
private:
void fun(){}
```

* avoids nesting when case 3 is used (compared to when using `{}`)

* I would argue that grouping lazy imports is actually a common case;
without case 3, the indentation will increase.






On Mon, Dec 19, 2016 at 8:56 PM, Timothee Cour <thelastmammoth gmail.com>
wrote:

 Andrei: you can use blockquotes to make long lines wrap when quoting text:

 https://github.com/adam-p/markdown-here/wiki/Markdown-
 Here-Cheatsheet#blockquotes

 that way, we don't have to scroll everytime there's a quote, eg:

 replace:
 <tab>Such scaffolding is of course
 with:
 Such scaffolding is of course
On Mon, Dec 19, 2016 at 8:04 PM, Andrei Alexandrescu via Digitalmars-d < digitalmars-d puremagic.com> wrote:
 On 12/19/2016 01:47 AM, Joakim wrote:

 Why do you care _so_ much about the module dependency graph?  To make
 this question concrete, let's look at an example, the std.array module
 you keep mentioning.
 This is what it looked like before Ilya scoped as
 many imports as he could:

 https://github.com/dlang/phobos/commit/3fcf723aa498b96de1653
 61b5abb9d3450fdc069#diff-54cf8402b22024ae667d4048a5126f0e

 That was a mess, similar to opaque C/C++ code, 13 modules imported at
 module-scope were whittled down to 4.  You just made those more specific
 in a commit related to this DIP, by listing the actual symbols
 selectively imported from those four modules:

 https://github.com/dlang/phobos/commit/e064d5664f92c4b2f0866
 c08f6d0290ba66825ed#diff-54cf8402b22024ae667d4048a5126f0e


 If I'm looking at the template constraints for any particular function
 and see a couple symbols I don't recognize, I don't think it's a big
 deal to find the symbols in that list at the top.
It actually is. Some symbols in e.g. std.range are used in constraints, some others used in definitions; some in a UFCS manner. Some code runs during compilation, with errors gagged (as in __traits(compiles)) or not gagged. I don't know of an easy manual method to figure out who's who (which makes Vladimir's idea of tooling it with brute force awesome and scary at the same time). At any way, I don't see how I can use this tidbit to improve the DIP. Anything I could add to it to make it more compelling? In other words, D already allows you to scope most imports. I don't
 consider the dozen or two remaining symbols from templaint constraints
 and function arguments to provide much overhead.  Rather, I consider the
 weight of this additional syntax, ie the cognitive overhead from having
 to remember and parse more syntax in my head, to be worse than the
 remaining dependency reasoning problem you're trying to solve: the cost
 outweights the benefit.  Perhaps that's subjective and others may
 disagree.
It is subjective and difficult to convert into action. Online review among folks who know next to nothing about each other does have its challenges. What happens here is I post a proposal for a problem that I believe is important for large D projects. And I get back "eh, that's not as important to me." At a traditional work place we'd know a lot about one another's strengths, specialization areas, and design sensibilities. Here, all I can do is look at your past PRs (that's why I emailed you asking for your github handle; I figure it's joakim-noah). This makes it difficult to act on "I don't buy it" feedback. Now, there's also the question of purely technical benefits, like
 compilation speed or executable bloat.  I looked at the latter a little
 last summer, after Ilya had cleaned up a lot of the standard library:

 http://forum.dlang.org/thread/gmjqfjoemwtvgqrtdsdr forum.dlang.org

 I found that commenting out a single scoped, selective import of
 "std.string: format" in std.utf led to a 5% decrease in executable size
 for "hello world."  This is a problem with how dmd compiles or appends
 these module dependencies and would presumably still be there after this
 DIP, as you would not remove the dependency.
That might be a related but distinct issue. Can you reproduce that experiment? I think scoped, selective imports have been great at hacking away at the
 module dependency graph, as you lay out.  It is not clear what technical
 costs you see from the remaining few dependencies and if this DIP is the
 best way to remove them.  I think you should explain why you want to
 untangle the remaining dependency graph, and consider if this DIP is
 really doing that much.
I've made a few more passes through the DIP, but throughout I assume a couple of things without stressing them too much - dependencies are important, and associating dependencies with declarations makes the dependency graph as fine and as precise it could get. I don't see a reasonable way to make it better or clearer. Should I add an introductory section on why dependencies are important? Andrei
Dec 19 2016
prev sibling next sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Tuesday, 20 December 2016 at 04:04:14 UTC, Andrei Alexandrescu 
wrote:
 On 12/19/2016 01:47 AM, Joakim wrote:
 Why do you care _so_ much about the module dependency graph?  
 To make
 this question concrete, let's look at an example, the 
 std.array module
 you keep mentioning.
 This is what it looked like before Ilya scoped as
 many imports as he could:

 https://github.com/dlang/phobos/commit/3fcf723aa498b96de165361b5abb9d3450fdc069#diff-54cf8402b22024ae667d4048a5126f0e

 That was a mess, similar to opaque C/C++ code, 13 modules 
 imported at
 module-scope were whittled down to 4.  You just made those 
 more specific
 in a commit related to this DIP, by listing the actual symbols
 selectively imported from those four modules:

 https://github.com/dlang/phobos/commit/e064d5664f92c4b2f0866c08f6d0290ba66825ed#diff-54cf8402b22024ae667d4048a5126f0e


 If I'm looking at the template constraints for any particular 
 function
 and see a couple symbols I don't recognize, I don't think it's 
 a big
 deal to find the symbols in that list at the top.
It actually is. Some symbols in e.g. std.range are used in constraints, some others used in definitions; some in a UFCS manner. Some code runs during compilation, with errors gagged (as in __traits(compiles)) or not gagged. I don't know of an easy manual method to figure out who's who (which makes Vladimir's idea of tooling it with brute force awesome and scary at the same time).
I'm not sure why you care who's who. I noted that if I'm looking at the constraints for a function, it's easy to check the selective imports at the top of the file for any symbols. That's the common scenario. I don't know why anyone would be checking the selective imports at the top to see where each symbol is actually used, which is the uncommon scenario you now present.
 At any way, I don't see how I can use this tidbit to improve 
 the DIP. Anything I could add to it to make it more compelling?
You could answer my question above. We have already scoped 90% of the dependencies, why is it so important to remove the remaining 10% that we need to add new syntax?
 In other words, D already allows you to scope most imports.  I 
 don't
 consider the dozen or two remaining symbols from templaint 
 constraints
 and function arguments to provide much overhead.  Rather, I 
 consider the
 weight of this additional syntax, ie the cognitive overhead 
 from having
 to remember and parse more syntax in my head, to be worse than 
 the
 remaining dependency reasoning problem you're trying to solve: 
 the cost
 outweights the benefit.  Perhaps that's subjective and others 
 may disagree.
It is subjective and difficult to convert into action. Online review among folks who know next to nothing about each other does have its challenges. What happens here is I post a proposal for a problem that I believe is important for large D projects. And I get back "eh, that's not as important to me." At a traditional work place we'd know a lot about one another's strengths, specialization areas, and design sensibilities. Here, all I can do is look at your past PRs (that's why I emailed you asking for your github handle; I figure it's joakim-noah). This makes it difficult to act on "I don't buy it" feedback.
I didn't just say "eh:" I gave evidence for why I think the problem is minimal and asked why it's so important to scope those last 3-4 imported modules too, which you didn't answer. As for looking at my PRs, there were some links above, which show that is my handle, but I don't think you'll get much from that, as I haven't designed anything in github.com/dlang, only cleaning up and porting.
 Now, there's also the question of purely technical benefits, 
 like
 compilation speed or executable bloat.  I looked at the latter 
 a little
 last summer, after Ilya had cleaned up a lot of the standard 
 library:

 http://forum.dlang.org/thread/gmjqfjoemwtvgqrtdsdr forum.dlang.org

 I found that commenting out a single scoped, selective import 
 of
 "std.string: format" in std.utf led to a 5% decrease in 
 executable size
 for "hello world."  This is a problem with how dmd compiles or 
 appends
 these module dependencies and would presumably still be there 
 after this
 DIP, as you would not remove the dependency.
That might be a related but distinct issue. Can you reproduce that experiment?
I went back and looked at that particular import and it appears Walter subsequently removed it: https://github.com/dlang/phobos/pull/3455 I will experiment a bit more with some sample code and see what I find.
 I think scoped, selective imports have been great at hacking 
 away at the
 module dependency graph, as you lay out.  It is not clear what 
 technical
 costs you see from the remaining few dependencies and if this 
 DIP is the
 best way to remove them.  I think you should explain why you 
 want to
 untangle the remaining dependency graph, and consider if this 
 DIP is
 really doing that much.
I've made a few more passes through the DIP, but throughout I assume a couple of things without stressing them too much - dependencies are important, and associating dependencies with declarations makes the dependency graph as fine and as precise it could get. I don't see a reasonable way to make it better or clearer. Should I add an introductory section on why dependencies are important?
If it deals with technical costs of dependencies that you're aiming to alleviate with this DIP, yes. I don't see why the module constructor reason you just gave can't be automated. We appear to disagree on the value of localizing the remaining selective imports needed for constraints/declarations, from the standpoint of reasoning and refactoring. Let me note that I'm not the only one in this thread saying that this isn't a problem worth adding syntax for and wondering why this DIP is worth doing. If we're going to do it, I think Dominik's suggestion is the best: just change the semantics of scoped imports to apply to the constraints and function arguments too. I don't think it's as surprising as you think, as the constraints are part of the function/struct. If somebody buries the import deep in the function and far away from the constraint, that's just bad style on their part, and is already possible (and annoying) now. We can automate the imports for declarations or leave that case alone. Finally, while all this dependency-localizing support is good, we really need tools to help us do it. I just happened to be thinking again about implementing such a tool a couple days before you posted this DIP. It'll have to wait till I push out the latest Android beta in the next week or two.
Dec 20 2016
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
I've asked Joakim about this via email just now, likely other folks also 
know the answer:

1. I found these PRs related to local imports:

https://github.com/dlang/phobos/pull/4361
https://github.com/dlang/phobos/pull/4365
https://github.com/dlang/phobos/pull/4370
https://github.com/dlang/phobos/pull/4373
https://github.com/dlang/phobos/pull/4379
https://github.com/dlang/phobos/pull/4392
https://github.com/dlang/phobos/pull/4467

Are there more? Is there a central point for them (such as a bugzilla 
issue)?

2. I see you've done a bunch of work in the area. Where, in your
estimate, are we on the spectrum of making imports local without a
major reorganization? Any low-hanging fruit to look at, or have Joakim,
Ilya, Jack and others made a pass through most/all modules already?


Thanks!
Dec 20 2016
next sibling parent Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Tuesday, 20 December 2016 at 20:51:54 UTC, Andrei Alexandrescu 
wrote:
 I've asked Joakim about this via email just now, likely other 
 folks also know the answer:

 1. I found these PRs related to local imports:

 https://github.com/dlang/phobos/pull/4361
 https://github.com/dlang/phobos/pull/4365
 https://github.com/dlang/phobos/pull/4370
 https://github.com/dlang/phobos/pull/4373
 https://github.com/dlang/phobos/pull/4379
 https://github.com/dlang/phobos/pull/4392
 https://github.com/dlang/phobos/pull/4467

 Are there more? Is there a central point for them (such as a 
 bugzilla issue)?

 2. I see you've done a bunch of work in the area. Where, in your
 estimate, are we on the spectrum of making imports local 
 without a
 major reorganization? Any low-hanging fruit to look at, or have 
 Joakim,
 Ilya, Jack and others made a pass through most/all modules 
 already?


 Thanks!
This is 95% of my PRs for local imports. Few of them moves codes between files. https://github.com/dlang/phobos/pull/2658 https://github.com/dlang/phobos/pull/2659 https://github.com/dlang/phobos/pull/2661 https://github.com/dlang/phobos/pull/2665 https://github.com/dlang/phobos/pull/2666 https://github.com/dlang/phobos/pull/2667 https://github.com/dlang/phobos/pull/2669 https://github.com/dlang/phobos/pull/2670 https://github.com/dlang/phobos/pull/2671 https://github.com/dlang/phobos/pull/2672 https://github.com/dlang/phobos/pull/2673 https://github.com/dlang/phobos/pull/2675 https://github.com/dlang/phobos/pull/2678 https://github.com/dlang/phobos/pull/2679 https://github.com/dlang/phobos/pull/2680 https://github.com/dlang/phobos/pull/2681 https://github.com/dlang/phobos/pull/2686 https://github.com/dlang/phobos/pull/2691 https://github.com/dlang/phobos/pull/2694 https://github.com/dlang/phobos/pull/2695 https://github.com/dlang/phobos/pull/2696 https://github.com/dlang/phobos/pull/2705 https://github.com/dlang/phobos/pull/2706 https://github.com/dlang/phobos/pull/2707 https://github.com/dlang/phobos/pull/2708 https://github.com/dlang/phobos/pull/2709 https://github.com/dlang/phobos/pull/2710 https://github.com/dlang/phobos/pull/2711 https://github.com/dlang/phobos/pull/2712 https://github.com/dlang/phobos/pull/2713 https://github.com/dlang/phobos/pull/2714 https://github.com/dlang/phobos/pull/2715 https://github.com/dlang/phobos/pull/2716 https://github.com/dlang/phobos/pull/2717 https://github.com/dlang/phobos/pull/2718 https://github.com/dlang/phobos/pull/2719 https://github.com/dlang/phobos/pull/2720 https://github.com/dlang/phobos/pull/2721 https://github.com/dlang/phobos/pull/2722 https://github.com/dlang/phobos/pull/2725 https://github.com/dlang/phobos/pull/2726 https://github.com/dlang/phobos/pull/2727 https://github.com/dlang/phobos/pull/2728 https://github.com/dlang/phobos/pull/2729 https://github.com/dlang/phobos/pull/2754 https://github.com/dlang/phobos/pull/2755 Ilya
Dec 20 2016
prev sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Tuesday, 20 December 2016 at 20:51:54 UTC, Andrei Alexandrescu 
wrote:
 I've asked Joakim about this via email just now, likely other 
 folks also know the answer:

 1. I found these PRs related to local imports:

 https://github.com/dlang/phobos/pull/4361
 https://github.com/dlang/phobos/pull/4365
 https://github.com/dlang/phobos/pull/4370
 https://github.com/dlang/phobos/pull/4373
 https://github.com/dlang/phobos/pull/4379
 https://github.com/dlang/phobos/pull/4392
 https://github.com/dlang/phobos/pull/4467

 Are there more? Is there a central point for them (such as a 
 bugzilla issue)?
Ilya lists a lot more above, he did most of the work. No central point that I know of.
 2. I see you've done a bunch of work in the area. Where, in your
 estimate, are we on the spectrum of making imports local 
 without a
 major reorganization? Any low-hanging fruit to look at, or have 
 Joakim,
 Ilya, Jack and others made a pass through most/all modules 
 already?
There is more to be done, but my guess would be 80-90% done for Phobos. Ilya scoped a lot, but usually left druntime imports alone. Top-level module imports mostly don't use selective imports yet, because Martin wanted to hold off till he was sure the symbol leak bug was fixed. On Tuesday, 20 December 2016 at 22:08:38 UTC, Andrei Alexandrescu wrote:
 On 12/20/2016 03:46 AM, Joakim wrote:
 I didn't just say "eh:" I gave evidence for why I think the 
 problem is
 minimal and asked why it's so important to scope those last 
 3-4 imported
 modules too, which you didn't answer.
You have asked for a smoking gun, and one has been found. I have just uploaded a major update that carefully analyzes the improvements brought about by switching to local imports in the D Standard Library. Please refer to the section "Workaround: Are Local Imports Good Enough?" and Appendix A: https://github.com/dlang/DIPs/pull/51/files https://github.com/dlang/DIPs/blob/91baecedcfe7cb75ac22e66478722ec797ebb901/DIPs/DIP1005.md
Thanks for this analysis of the remaining dependency graph, it is worth looking at. Allow me to poke some holes in it. To begin with, the amount of scoping that has been done is overstated, if you simply count scoped imports and compare it to module-level imports. Each module-level import has to be replicated multiple times for each local scope, especially in unittest blocks. A better number is more like 20-30%, as I pointed out 4 out of 13 modules remain at top-level in std.array. Using that metric, a 3-4X reduction in top-level imports has led to at least a 2.2x improvement in imported files, so the effort has been more meaningful than you conclude. Second, as I noted above, most top-level imports have not been made selective yet, because of the symbol leak bug that was recently fixed by Martin. You will see in my PRs that I only list those symbols as a comment, because I could not turn those into selective imports yet. If the compiler is doing its job right, selective imports should greatly reduce the cost of importing a module, even if your metric would still show the module being imported. Third, checking some of the output from the commands you ran in your script shows that up to half of the imported modules are from druntime. I noted earlier that Ilya usually didn't bother scoping top-level druntime imports, because he perceived their cost to be low (I scoped them too in the handful I cleaned up, just for completeness). As far as I know, nobody has bothered to spend any time scoping druntime, so it would be better if you filtered out druntime imports from your analysis. Finally, while it's nice to know the extent of the dependency graph, what really matters is the _cost_ of each link of the graph, which is what I keep hammering on. If the cost of links is small, it doesn't matter how entangled it is. If minimizing the dependency graph through scoping alone, ie without implementing this DIP, removes most of the cost, that's all I care about. I have noted one example above, where _a single DCD in phobos_, ie a scoped, selective import, had gigantic costs in terms of executable size, where entire modules were included because of it. If that's the case more generally, then _no_ amount of dependency disentangling will matter, because the cost of single DCDs is still huge. Perhaps that's just an isolated issue however, it needs to be investigated. My point is that the dependency graph matters, but now that we're getting down to the last entanglements, we need to know the cost of those last links. Your dependency analysis gives us some quantitative idea of the size of the remaining graph, but tells us nothing about the cost of those links. That's what I'm looking for. I will spend some time now investigating those costs with sample code. My request all along has been that you give us some idea of those costs, if you know the answer already.
Dec 20 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/20/2016 11:32 PM, Joakim wrote:
 On Tuesday, 20 December 2016 at 20:51:54 UTC, Andrei Alexandrescu wrote:
 Thanks for this analysis of the remaining dependency graph, it is worth
 looking at.  Allow me to poke some holes in it.

 To begin with, the amount of scoping that has been done is overstated,
 if you simply count scoped imports and compare it to module-level
 imports.  Each module-level import has to be replicated multiple times
 for each local scope, especially in unittest blocks.  A better number is
 more like 20-30%, as I pointed out 4 out of 13 modules remain at
 top-level in std.array. Using that metric, a 3-4X reduction in top-level
 imports has led to at least a 2.2x improvement in imported files, so the
 effort has been more meaningful than you conclude.
Fixed. I also made a note about the need to duplicate imports as they are pushed down.
 Second, as I noted above, most top-level imports have not been made
 selective yet, because of the symbol leak bug that was recently fixed by
 Martin.  You will see in my PRs that I only list those symbols as a
 comment, because I could not turn those into selective imports yet.  If
 the compiler is doing its job right, selective imports should greatly
 reduce the cost of importing a module, even if your metric would still
 show the module being imported.
That is not relevant to this section, which discusses the effectiveness of using local imports with the current compilation technology. Per the section's opening sentence:
 A legitimate question to ask is whether consistent use of local
 imports wherever possible would be an appropriate approximation of
 the Dependency-Carrying Declarations goal with no change in the
 language at all.
The section "Alternative: Lazy Imports" discusses how static or local imports could be used in conjunction with new compilation technologies. If there are improvements to be made there, please advise.
 Third, checking some of the output from the commands you ran in your
 script shows that up to half of the imported modules are from druntime.
 I noted earlier that Ilya usually didn't bother scoping top-level
 druntime imports, because he perceived their cost to be low (I scoped
 them too in the handful I cleaned up, just for completeness).  As far as
 I know, nobody has bothered to spend any time scoping druntime, so it
 would be better if you filtered out druntime imports from your analysis.
Fixed to only count imports from std.
 Finally, while it's nice to know the extent of the dependency graph,
 what really matters is the _cost_ of each link of the graph, which is
 what I keep hammering on.  If the cost of links is small, it doesn't
 matter how entangled it is.  If minimizing the dependency graph through
 scoping alone, ie without implementing this DIP, removes most of the
 cost, that's all I care about.
In first approximation, whether a file gets opened or not makes a difference (filesystem operations (possibly including networking), necessity to rebuild if dependent code is changed. The analysis shows there is significant overhead remaining, on average 10.5 additional files per unused import. If the current document could be clearer in explaining costs, please let me know.
 I have noted one example above, where _a single DCD in phobos_, ie a
 scoped, selective import, had gigantic costs in terms of executable
 size, where entire modules were included because of it.  If that's the
 case more generally, then _no_ amount of dependency disentangling will
 matter, because the cost of single DCDs is still huge.  Perhaps that's
 just an isolated issue however, it needs to be investigated.
That seems an unrelated matter. Yes, you could pull a large dependency in one shot with old or new technology.
 My point is that the dependency graph matters, but now that we're
 getting down to the last entanglements, we need to know the cost of
 those last links.  Your dependency analysis gives us some quantitative
 idea of the size of the remaining graph, but tells us nothing about the
 cost of those links. That's what I'm looking for.

 I will spend some time now investigating those costs with sample code.
 My request all along has been that you give us some idea of those costs,
 if you know the answer already.
I don't know how to make matters much clearer than the current document. Any suggestions are welcome. The section "Workaround: Are Local Imports Good Enough?" discusses the material cost in terms of extra files that need to be opened and parsed (some unnecessarily) in order to complete a compilation. The "Rationale" part of the document discusses the costs in terms of maintainability, clarity, and documentation. Thanks, Andrei
Dec 21 2016
next sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Wednesday, 21 December 2016 at 15:16:34 UTC, Andrei 
Alexandrescu wrote:
 On 12/20/2016 11:32 PM, Joakim wrote:
 Second, as I noted above, most top-level imports have not been 
 made
 selective yet, because of the symbol leak bug that was 
 recently fixed by
 Martin.  You will see in my PRs that I only list those symbols 
 as a
 comment, because I could not turn those into selective imports 
 yet.  If
 the compiler is doing its job right, selective imports should 
 greatly
 reduce the cost of importing a module, even if your metric 
 would still
 show the module being imported.
That is not relevant to this section, which discusses the effectiveness of using local imports with the current compilation technology. Per the section's opening sentence:
 A legitimate question to ask is whether consistent use of local
 imports wherever possible would be an appropriate 
 approximation of
 the Dependency-Carrying Declarations goal with no change in the
 language at all.
It is relevant because it could further reduce the cost from module-scope imports. Whether your section only chooses to focus on locally scoped imports and ignore the impact of selective imports is irrelevant to me.
 Finally, while it's nice to know the extent of the dependency 
 graph,
 what really matters is the _cost_ of each link of the graph, 
 which is
 what I keep hammering on.  If the cost of links is small, it 
 doesn't
 matter how entangled it is.  If minimizing the dependency 
 graph through
 scoping alone, ie without implementing this DIP, removes most 
 of the
 cost, that's all I care about.
In first approximation, whether a file gets opened or not makes a difference (filesystem operations (possibly including networking), necessity to rebuild if dependent code is changed. The analysis shows there is significant overhead remaining, on average 10.5 additional files per unused import.
Opening a file or 10 is extremely cheap compared to all the other costs of the compiler. Purely from a technical cost perspective, I'm not sure even scoped imports were worth it, as my simple investigation below suggests.
 If the current document could be clearer in explaining costs, 
 please let me know.
Yes, please explain what significant "overhead" was reduced by scoping imports and would be futher reduced by this DIP. Opening files doesn't cut it.
 My point is that the dependency graph matters, but now that 
 we're
 getting down to the last entanglements, we need to know the 
 cost of
 those last links.  Your dependency analysis gives us some 
 quantitative
 idea of the size of the remaining graph, but tells us nothing 
 about the
 cost of those links. That's what I'm looking for.

 I will spend some time now investigating those costs with 
 sample code.
 My request all along has been that you give us some idea of 
 those costs,
 if you know the answer already.
I don't know how to make matters much clearer than the current document. Any suggestions are welcome. The section "Workaround: Are Local Imports Good Enough?" discusses the material cost in terms of extra files that need to be opened and parsed (some unnecessarily) in order to complete a compilation. The "Rationale" part of the document discusses the costs in terms of maintainability, clarity, and documentation.
I don't know how to make it clearer that that's not good enough. You seem to understand that I want more justification than hand-waving about "scalable" and "overhead," which is why you now give the cost of opening files as justification, but you don't seem to have anything more substantive than that flimsy claim. I just tried compiling phobos and its unittests for dmd 2.066.1 and 2.067.1, the dmd releases from right before and right after Ilya's PRs linked above (compiling phobos from the older release with the newer dmd didn't work and I wasn't interested in porting it). I found they took about the same amount of time to compile their respective phobos and the later version consistently took 15 seconds longer to compile the unittests on a single core. This suggests that there has been essentially no technical benefit to scoped imports, that it is largely superficial. That's fine, I think it's worth it just from the standpoint of understanding where most dependencies are coming from. I don't think an additional syntax change is necessary for the remaining few module-scope dependencies for template constraints. Note on investigation: Of course, some other relevant factors could have changed in dmd and phobos between those two releases, so the result is certainly not conclusive. But the fact that there was no decrease in compile times while phobos took around the same amount of time is highly suggestive. I'm uninterested in investigating further given the consistent hand-waving justifications for this DIP. If somebody else had submitted this DIP, it would have been quickly shot down, and rightly so.
Dec 22 2016
next sibling parent Timothee Cour via Digitalmars-d <digitalmars-d puremagic.com> writes:
 I just tried compiling phobos and its unittests for dmd 2.066.1 and
2.067.1 I think it's worth considering compile time for partial recompilation as opposed to full compilation. The benifit of this DIP should be more pronounced there since there'll be more opportunities to skip parsing modules in that case. Partial recompilation is what matters most during `edit compile debug cycle` anyways On Thu, Dec 22, 2016 at 2:17 AM, Joakim via Digitalmars-d < digitalmars-d puremagic.com> wrote:
 On Wednesday, 21 December 2016 at 15:16:34 UTC, Andrei Alexandrescu wrote:

 On 12/20/2016 11:32 PM, Joakim wrote:

 Second, as I noted above, most top-level imports have not been made
 selective yet, because of the symbol leak bug that was recently fixed by
 Martin.  You will see in my PRs that I only list those symbols as a
 comment, because I could not turn those into selective imports yet.  If
 the compiler is doing its job right, selective imports should greatly
 reduce the cost of importing a module, even if your metric would still
 show the module being imported.
That is not relevant to this section, which discusses the effectiveness of using local imports with the current compilation technology. Per the section's opening sentence: A legitimate question to ask is whether consistent use of local
 imports wherever possible would be an appropriate approximation of
 the Dependency-Carrying Declarations goal with no change in the
 language at all.
It is relevant because it could further reduce the cost from module-scope imports. Whether your section only chooses to focus on locally scoped imports and ignore the impact of selective imports is irrelevant to me. Finally, while it's nice to know the extent of the dependency graph,
 what really matters is the _cost_ of each link of the graph, which is
 what I keep hammering on.  If the cost of links is small, it doesn't
 matter how entangled it is.  If minimizing the dependency graph through
 scoping alone, ie without implementing this DIP, removes most of the
 cost, that's all I care about.
In first approximation, whether a file gets opened or not makes a difference (filesystem operations (possibly including networking), necessity to rebuild if dependent code is changed. The analysis shows there is significant overhead remaining, on average 10.5 additional files per unused import.
Opening a file or 10 is extremely cheap compared to all the other costs of the compiler. Purely from a technical cost perspective, I'm not sure even scoped imports were worth it, as my simple investigation below suggests. If the current document could be clearer in explaining costs, please let
 me know.
Yes, please explain what significant "overhead" was reduced by scoping imports and would be futher reduced by this DIP. Opening files doesn't cut it. My point is that the dependency graph matters, but now that we're
 getting down to the last entanglements, we need to know the cost of
 those last links.  Your dependency analysis gives us some quantitative
 idea of the size of the remaining graph, but tells us nothing about the
 cost of those links. That's what I'm looking for.

 I will spend some time now investigating those costs with sample code.
 My request all along has been that you give us some idea of those costs,
 if you know the answer already.
I don't know how to make matters much clearer than the current document. Any suggestions are welcome. The section "Workaround: Are Local Imports Good Enough?" discusses the material cost in terms of extra files that need to be opened and parsed (some unnecessarily) in order to complete a compilation. The "Rationale" part of the document discusses the costs in terms of maintainability, clarity, and documentation.
I don't know how to make it clearer that that's not good enough. You seem to understand that I want more justification than hand-waving about "scalable" and "overhead," which is why you now give the cost of opening files as justification, but you don't seem to have anything more substantive than that flimsy claim. I just tried compiling phobos and its unittests for dmd 2.066.1 and 2.067.1, the dmd releases from right before and right after Ilya's PRs linked above (compiling phobos from the older release with the newer dmd didn't work and I wasn't interested in porting it). I found they took about the same amount of time to compile their respective phobos and the later version consistently took 15 seconds longer to compile the unittests on a single core. This suggests that there has been essentially no technical benefit to scoped imports, that it is largely superficial. That's fine, I think it's worth it just from the standpoint of understanding where most dependencies are coming from. I don't think an additional syntax change is necessary for the remaining few module-scope dependencies for template constraints. Note on investigation: Of course, some other relevant factors could have changed in dmd and phobos between those two releases, so the result is certainly not conclusive. But the fact that there was no decrease in compile times while phobos took around the same amount of time is highly suggestive. I'm uninterested in investigating further given the consistent hand-waving justifications for this DIP. If somebody else had submitted this DIP, it would have been quickly shot down, and rightly so.
Dec 22 2016
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/22/2016 05:17 AM, Joakim wrote:
 On Wednesday, 21 December 2016 at 15:16:34 UTC, Andrei Alexandrescu wrote:
 In first approximation, whether a file gets opened or not makes a
 difference (filesystem operations (possibly including networking),
 necessity to rebuild if dependent code is changed. The analysis shows
 there is significant overhead remaining, on average 10.5 additional
 files per unused import.
Opening a file or 10 is extremely cheap compared to all the other costs of the compiler. Purely from a technical cost perspective, I'm not sure even scoped imports were worth it, as my simple investigation below suggests.
This is a misunderstanding. (I'll make the DIP clearer.) It's not about the cost of opening the file per se; I'm using the number of files opened as a proxy for all work involved in processing a file. Meaning, if you import 10 files you're likely to do roughly 10x the work of importing one file.
 I just tried compiling phobos and its unittests for dmd 2.066.1 and
 2.067.1, the dmd releases from right before and right after Ilya's PRs
 linked above (compiling phobos from the older release with the newer dmd
 didn't work and I wasn't interested in porting it).  I found they took
 about the same amount of time to compile their respective phobos and the
 later version consistently took 15 seconds longer to compile the
 unittests on a single core.
Unittesting Phobos (whether in separation or together) will instantiate everything so they are the case when local imports do _not_ make any difference. That's why DIP1005 uses that case as an estimate of everything in an imported module being used. Compiling Phobos in its entirety (one command) and measuring that time is also of tenuous relevance; I need to think of it but at first sight the cost of unnecessary imports is collapsed, and I suspect (and will measure) that either way most modules are in fact imported. One measurement of interest is: write a module that imports exactly one stdlib module, compile it, and measure time. Looking at the generated object and executable size would be also of interest. I'll do that.
 This suggests that there has been essentially no technical benefit to
 scoped imports, that it is largely superficial.  That's fine, I think
 it's worth it just from the standpoint of understanding where most
 dependencies are coming from.  I don't think an additional syntax change
 is necessary for the remaining few module-scope dependencies for
 template constraints.

 Note on investigation: Of course, some other relevant factors could have
 changed in dmd and phobos between those two releases, so the result is
 certainly not conclusive.  But the fact that there was no decrease in
 compile times while phobos took around the same amount of time is highly
 suggestive.

 I'm uninterested in investigating further given the consistent
 hand-waving justifications for this DIP.  If somebody else had submitted
 this DIP, it would have been quickly shot down, and rightly so.
Walter and I have the role of scrutinizing every addition to the language (and reject most). It is natural that our own work is met with increased scrutiny. Picking to death anything and everything we say or do is a staple in this community, and a rite of passage on github. It is of course impossible to know what would have happened if the proposal were made by someone else. All I can say is Walter knew nothing about it and said it is good (except for the initial syntax; he's on board with "with"). Anyhow, not to worry. The burden of proof is on the DIP. I'll take a look at making some more measurements. Andrei
Dec 22 2016
prev sibling parent reply Chris Wright <dhasenan gmail.com> writes:
On Thu, 22 Dec 2016 10:17:33 +0000, Joakim wrote:
 Opening a file or 10 is extremely cheap compared to all the other costs
 of the compiler.  Purely from a technical cost perspective,
 I'm not sure even scoped imports were worth it, as my simple
 investigation below suggests.
The compiler doesn't merely have to open the file. It has to run at least partial semantic analysis on it in order to locate symbols. When templates are involved, this can become quite expensive. With static and selective imports, this step can be entirely avoided -- the current file tells you which files contain which declarations. With either, the compiler can tell exactly which file contains which declarations, so it can avoid pulling in anything beyond what's strictly necessary. With declaration-scoped imports, you avoid the same step. However, it requires the maintainer's discipline to reduce the declaration's imports to the minimal possible set of imports. For instance, I have a declaration: import myapp.users, std.socket; bool isUserOnline(User user, Socket userSocket); I decide that this needs compilation time optimizations. The current way: static import myapp.users, std.socket; bool isUserOnline(myapp.users.User user, std.socket.Socket socket); And Andrei's way: bool isUserOnline(User user, Socket userSocket) import myapp.users, std.socket; I refactor things so that this check finds the user socket on its own. The current way: static import myapp.users, std.socket; bool isUserOnline(myapp.users.User user); Eh, I forgot I don't need std.socket anymore, but this costs a few microseconds of compiler time to add it to the current symbol table. It has to allocate a lazily expanded module stub. Shouldn't be a huge deal. Andrei's way: bool isUserOnline(User user) import myapp.users, std.socket; Again, I forgot to update the imports, but this time the compiler has to read std.socket from disk, parse it, run semantic on it, and import all its top-level symbols into the scope's symbol table. Because there's nothing here that says the 'User' type is in myapp.users instead of std.socket. --- In point of fact, selective and static imports should be *faster* than Andrei's way. Consider: static import myapp.users, std.socket; bool isUserOnline(myapp.users.User user, std.socket.Socket socket); This has to locate a declaration named `User` in myapp.users, and it has to locate a declaration named `Socket` in std.socket. But let's look at Andrei's way: bool isUserOnline(User user, Socket userSocket) import myapp.users, std.socket; Here, the compiler has to search *both* myapp.users and std.socket for a declaration named `User`, then it has to search both for `Socket`. (Even if it finds `User` in the first, it still needs to search the second in case both define that symbol.) You go from O(distinct number of types referenced) lookups to O(types * imports). Granted, you'll usually have between one and three types, between one and three imports, so the point is a bit less salient.
Dec 22 2016
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/22/16 9:53 PM, Chris Wright wrote:
   bool isUserOnline(User user, Socket userSocket)
   import myapp.users, std.socket;
Has changed to with (import myapp.users, std.socket) bool isUserOnline(User user, Socket userSocket); Andrei
Dec 23 2016
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/22/16 9:53 PM, Chris Wright wrote:
 In point of fact, selective and static imports should be *faster* than
 Andrei's way. Consider:

   static import myapp.users, std.socket;
   bool isUserOnline(myapp.users.User user, std.socket.Socket socket);

 This has to locate a declaration named `User` in myapp.users, and it has
 to locate a declaration named `Socket` in std.socket.

 But let's look at Andrei's way:

   bool isUserOnline(User user, Socket userSocket)
   import myapp.users, std.socket;
with (static import myapp.users, std.socket) bool isUserOnline(myapp.users.User user, std.socket.Socket socket); Andrei
Dec 23 2016
prev sibling parent piotrklos <poliklosio happypizza.com> writes:
On Wednesday, 21 December 2016 at 15:16:34 UTC, Andrei 
Alexandrescu wrote:
 On 12/20/2016 11:32 PM, Joakim wrote:
 (...)
I don't know how to make matters much clearer than the current document. Any suggestions are welcome. The section "Workaround: Are Local Imports Good Enough?" discusses the material cost in terms of extra files that need to be opened and parsed (some unnecessarily) in order to complete a compilation. The "Rationale" part of the document discusses the costs in terms of maintainability, clarity, and documentation. Thanks, Andrei
Stipulation: I think the difference of opinion may be caused by working on different sizes of projects in ones career (tens of thousands vs millions of LoC). Suggestion 1: Maybe the DIP should point out that the cost of redundant imports (however small) tends to grow quadratically with code size (size of import tree times the number of compilations). If this is not the case then maybe the DIP is really in the wrong direction. Suggestion 2: Implement the DIP, autogenerate millions of lines of D code (2 versions: with and without DCDs) and see which version of DMD compiles them faster. This may also expose other ways to improve scalability without implementing this DIP.
Dec 22 2016
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/20/2016 03:46 AM, Joakim wrote:
 I didn't just say "eh:" I gave evidence for why I think the problem is
 minimal and asked why it's so important to scope those last 3-4 imported
 modules too, which you didn't answer.
You have asked for a smoking gun, and one has been found. I have just uploaded a major update that carefully analyzes the improvements brought about by switching to local imports in the D Standard Library. Please refer to the section "Workaround: Are Local Imports Good Enough?" and Appendix A: https://github.com/dlang/DIPs/pull/51/files https://github.com/dlang/DIPs/blob/91baecedcfe7cb75ac22e66478722ec797ebb901/DIPs/DIP1005.md Andrei
Dec 20 2016
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/20/2016 05:08 PM, Andrei Alexandrescu wrote:
 On 12/20/2016 03:46 AM, Joakim wrote:
 I didn't just say "eh:" I gave evidence for why I think the problem is
 minimal and asked why it's so important to scope those last 3-4 imported
 modules too, which you didn't answer.
You have asked for a smoking gun, and one has been found. I have just uploaded a major update that carefully analyzes the improvements brought about by switching to local imports in the D Standard Library. Please refer to the section "Workaround: Are Local Imports Good Enough?" and Appendix A: https://github.com/dlang/DIPs/pull/51/files https://github.com/dlang/DIPs/blob/91baecedcfe7cb75ac22e66478722ec797ebb901/DIPs/DIP1005.md
I've also added a supporting script that creates the table in Appendix A: https://github.com/andralex/DIPs/blob/9e9ebc8738ce04b2d85a92feafacc7ef81e59811/DIPs/DI 1005-countlines.zsh -- Andrei
Dec 20 2016
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/20/2016 05:08 PM, Andrei Alexandrescu wrote:
 On 12/20/2016 03:46 AM, Joakim wrote:
 I didn't just say "eh:" I gave evidence for why I think the problem is
 minimal and asked why it's so important to scope those last 3-4 imported
 modules too, which you didn't answer.
You have asked for a smoking gun, and one has been found. I have just uploaded a major update that carefully analyzes the improvements brought about by switching to local imports in the D Standard Library. Please refer to the section "Workaround: Are Local Imports Good Enough?" and Appendix A: https://github.com/dlang/DIPs/pull/51/files https://github.com/dlang/DIPs/blob/91baecedcfe7cb75ac22e66478722ec797ebb901/DIPs/DIP1005.md
Eh, looks like http://dillinger.io/ and github don't agree on table rendering... Fixed URL for nice viewing: https://github.com/dlang/DIPs/blob/249b28ddd784220e44e343e78e5ea7a65c4c7bde/DIPs/DIP1005.md Andrei
Dec 20 2016
prev sibling parent reply Timothee Cour via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Mon, Dec 19, 2016 at 9:33 PM, Timothee Cour <thelastmammoth gmail.com>
wrote:

 what about using `lazy` instead of `with`:

 `with(import foo)`
 =>
 `lazy(import foo)`

 advantages:
 * avoids confusion regarding usual scoping rules of `with` ;
 * conveys that the import is indeed lazy

 Furthermore (regardless of which keyword is used), what about allowing `:`
 ```
 // case 1
 lazy(import foo)
 void fun(){}

 // case 2
 lazy(import foo) {
   void fun(){}
 }

 // case 3 : this is new
 lazy(import foo):
 void fun1(){}
 void fun2(){}
 ```

 advantages:

 * same behavior as other constructs which don't introduce a scope:
 ```
 // case 1, 2 3 are allowed:
 version(A):
 static if(true):
 private:
 void fun(){}
 ```

 * avoids nesting when case 3 is used (compared to when using `{}`)

 * I would argue that grouping lazy imports is actually a common case;
 without case 3, the indentation will increase.
Andrei: ping on this? (especially regarding allowing `:`)
Dec 21 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/21/16 6:40 PM, Timothee Cour via Digitalmars-d wrote:
 Andrei: ping on this? (especially regarding allowing `:`)
I think "lazy" is a bit too cute. "with" is so close to what's actually needed, it would be a waste to not use it. Generally I'm weary of the use of ":" (never liked it - it makes code dependent on long-distance context) so I'd rather snatch the opportunity to avoid it. Andrei
Dec 21 2016
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/13/2016 05:33 PM, Andrei Alexandrescu wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files
Major changes effected to the DIP, which affect it. In-depth discussion of lazy imports, a change in lookup rules, and a few minor things: https://github.com/dlang/DIPs/blob/d228f48a12c1d976735412fd5b7543cb1febda7b/DIPs/DIP1005.md Andrei
Dec 16 2016
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2016-12-13 23:33, Andrei Alexandrescu wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files
A couple of questions. 1. The text says: "void process(File input) import (std.stdio); With this syntax, the import is executed only if the declared name is actually looked up." Is "declared name" referring to the function "process"? To me it sounds like it could be misinterpreted to refer to "File", which would indicate the import is lazily executed. 2. The text says: "Currently inline imports do not apply to alias declarations, enumerated types declarations, or variable declarations" What's the reasoning behind this? 3. Is it required to place the import after the declaration or can it be placed in front of it? 4. Related to the third question. Would it be better (if not already) to implement the import as an attribute, which would allow for a more flexible syntax: void process(File input) import (std.stdio); import (std.stdio) void process(File input); import (std.stdio) { void process(File input); void process2(File input); } import (std.stdio): void process(File input); void process2(File input); 5. About the original syntax that was used, i.e. "import.std.range.isInputRange!Range". That syntax could require to always use a colon to separate the module name from the imported symbol. It's the same syntax as a selective import, which I think fits since with this syntax it will always be selective imports, as far as I understand. Example: void process(import.std.stdio:File input); This syntax could also be extended to all symbols: void process(import.std.stdio:File input) { import.std.stdio:File a = input; } 6. Not sure if it's a requirement for a DIP but I don't see any formal description of the necessary grammar changes. -- /Jacob Carlborg
Dec 17 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/17/2016 10:39 AM, Jacob Carlborg wrote:
 On 2016-12-13 23:33, Andrei Alexandrescu wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files
A couple of questions. 1. The text says: "void process(File input) import (std.stdio); With this syntax, the import is executed only if the declared name is actually looked up." Is "declared name" referring to the function "process"? To me it sounds like it could be misinterpreted to refer to "File", which would indicate the import is lazily executed.
[snip] Thanks, made a change. -- Andrei
Dec 19 2016
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Pushed again, now with the syntax using "with" proposed by Hatem Oraby 
and others.

https://github.com/dlang/DIPs/pull/51

https://github.com/dlang/DIPs/blob/71bde077488b566fba7603de6095b45984d9294a/DIPs/DIP1005.md


Andrei
Dec 19 2016
parent Meta <jared771 gmail.com> writes:
On Tuesday, 20 December 2016 at 01:06:01 UTC, Andrei Alexandrescu 
wrote:
 Pushed again, now with the syntax using "with" proposed by 
 Hatem Oraby and others.

 https://github.com/dlang/DIPs/pull/51

 https://github.com/dlang/DIPs/blob/71bde077488b566fba7603de6095b45984d9294a/DIPs/DIP1005.md


 Andrei
"In addition, we propose the statement and declaration with (import ImportList). ImportList is any syntactical construct currently accepted by the import declaration. The with (import ImportList) obeys the following rules: - Inside any function, with (Import ImportList) is a statement that introduces a scope. Inside the with, lookup considers the import local to the declaration (similar to the current handling of nested imports). - Everywhere else, with (Import ImportList) is always a declaration and does not introduce a new scope. Lookup of symbols is the same as for the statement case." I must've somehow missed this when reading through for the first time after your changes as I thought this was a glaring omission. Glad to see that this was covered as this rightly makes it turtles all the way down. Actually, you could completely replace import statements with this new construct, having the `import std.range` at the top level replaced with a module-wide attribute `with (import std.range):`. In that light my only real criticism is that it's kind of redundant having both this new form *and* local imports, and it's not clear which is better and why to a new user.
Dec 20 2016
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Added this note about module constructors:

## Future Possibilities and Directions

Inline and scoped imports offer the option of better handling of static 
module constructors. Currently, modules that mutually `import` one 
another (either directly or through a longer chain) cannot 
simultaneously define `shared static this()` constructors. The reason is 
that, again, dependencies are computed at module level.

If instead modules have no top-level dependencies, then the compiler is 
able to compute the narrow set of dependencies needed for executing the 
static module constructor. The static constructor may be (a) a part of a 
`with` declaration, (b) use local imports within, and (c) call other 
functions within the module that have their own dependencies. For example:

```d
// assume no top-level import
with (module_a) void fun(T)()
{
     import module_b;
     return gun();
}
with (module_c)
static shared this()
{
     import module_d;
     fun!int;
}
```

In this case, the module constructor depends (only) on `module_a`, 
`module_b`, `module_c`, and `module_d`. The full information is confined 
within the current module so it is inferrable during separate compilation.



Andrei
Dec 19 2016
prev sibling next sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 12/13/16 11:33 PM, Andrei Alexandrescu wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei
Just a thought but with all of proliferation of imports down to each declaration comes the pain that e.g. renaming a module cascades to countless instances of import statements. This is true of local imports as well but the problem gets bigger. ---- Dmitry Olshansky
Dec 20 2016
next sibling parent Meta <jared771 gmail.com> writes:
On Tuesday, 20 December 2016 at 14:31:38 UTC, Dmitry Olshansky 
wrote:
 On 12/13/16 11:33 PM, Andrei Alexandrescu wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei
Just a thought but with all of proliferation of imports down to each declaration comes the pain that e.g. renaming a module cascades to countless instances of import statements. This is true of local imports as well but the problem gets bigger. ---- Dmitry Olshansky
Could you not have the old module just be empty and publicly import the new one, and also deprecate it so people have time to change their code? Also having special syntax makes finding every occurrence of an inline import a fairly simple search.
Dec 20 2016
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/20/2016 09:31 AM, Dmitry Olshansky wrote:
 On 12/13/16 11:33 PM, Andrei Alexandrescu wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei
Just a thought but with all of proliferation of imports down to each declaration comes the pain that e.g. renaming a module cascades to countless instances of import statements. This is true of local imports as well but the problem gets bigger.
Good point, I'll integrate it in the document. -- Andrei
Dec 20 2016
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/20/2016 09:31 AM, Dmitry Olshansky wrote:
 On 12/13/16 11:33 PM, Andrei Alexandrescu wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files


 Andrei
Just a thought but with all of proliferation of imports down to each declaration comes the pain that e.g. renaming a module cascades to countless instances of import statements. This is true of local imports as well but the problem gets bigger.
https://github.com/dlang/DIPs/pull/51/commits/d4ef6826dacedc38f822e48bec2186d93040fb42 Andrei
Dec 21 2016
prev sibling next sibling parent reply Chris Wright <dhasenan gmail.com> writes:
I'm looking at this part:

 The manual conversion of std.array to the "static import" form is shown
 here. It leads to the expected lengthening of the symbols used in
 declarations, which appears to eliminate one disadvantage by
 introducing another.
You get longer declarations when you're reading and writing code. This shouldn't impact the docs. Not arguing that there is a disadvantage, just pointing out the scope is relatively small. All your references are entirely unambiguous. If I'm reading the source and I'm unfamiliar with phobos, I don't have to wonder if that template constraint comes from std.string or std.traits. It's two benefits for one disadvantage. You can also have renamed imports to reduce the amount of typing: static import trait = std.traits; trait.ForeachType!Range[] array(Range)(Range r) if (trait.isIterable!Range && trait.isNarrowString!Range && !trait.isInfinite!Range)) { }
 The manual conversion of std.array to the "selective import" form is 
 shown here. Conversion was successful but because it collapses all 
 imports at the top, it does not make it much easier to identify e.g. 
 what dependencies would be pulled if a given artifact in std.array were 
 used.
It's still trivial for the compiler to detect which modules a particular declaration depends on. If I'm in vim, I can put my cursor on the declaration, hit '*', navigate to the top of the document, and find the next match. It's like five keypresses. With VS Code or the like, I can similarly go to the top of the file and find the first match without much trouble. And this process tells me which module defines the symbol. Either the first hit is at the top of the file in the import section, and I know which file to look in, or it's not, and I know it's defined locally (and there's a good chance I'm at the declaration). With your proposal, when I'm lucky, I know the declaration must be either in the current file or in the locally imported file. (When I'm unlucky, the file has some top-level imports or multiple local imports.) And since the utility of this is mostly for large modules, I'm still going to pull out grep. It *does* help when I need to import the same thing elsewhere, at least.
 Again the manual process was highly nontrivial.
This is also true for your proposal. CyberShadow also offered to whip up a tool to automate the conversion to selective imports. Presumably they would be able to do something similar with your proposal. So this is kind of irrelevant.
Dec 22 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/22/16 10:31 PM, Chris Wright wrote:
 It's two benefits for one disadvantage.
One won't make decisions by taking the difference of pros and cons, right? -- Andrei
Dec 23 2016
parent Chris Wright <dhasenan gmail.com> writes:
On Fri, 23 Dec 2016 07:48:55 -0500, Andrei Alexandrescu wrote:

 On 12/22/16 10:31 PM, Chris Wright wrote:
 It's two benefits for one disadvantage.
One won't make decisions by taking the difference of pros and cons, right? -- Andrei
You'd use the weighted difference, naturally.
Dec 23 2016
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Major update adding an experiment that shows the cost of top-level imports.

https://github.com/dlang/DIPs/pull/51

https://github.com/dlang/DIPs/blob/a3ef4e25cfb9f884fee29edb5553a3a2b840f679/DIPs/DIP1005.md

Relevant added text:

Another matter we investigated is how reducing top-level imports 
influences build times and the size of the object files produced. We do 
not have an experimental implementation of this DIP, so measuring impact 
directly was not possible. We did the converse experiment---adding 
top-level imports.

In a separate branch of the standard library code, for each module we 
added all nested imports back to the top level. Some hand-editing was 
needed after that because of clashes in symbol names. Also, some imports 
needed to be removed because of circular dependencies and related 
limitations in the language's design and implementation. The resulting 
setup can be seen in 
[PR4992](https://github.com/dlang/phobos/pull/4992). Then for each 
module in the standard library we compiled one file that consists of 
exactly one `import` declaration, monitoring compile time and object 
file size. Appendix B displays build times and size of object files 
produced by this experiment.

|Aggregate|Time (top-level)|Time (nested)|Object size (top-level)|Object 
size (nested)|
|---|---|---|---|---|
|Median|320ms|13788|32ms|4296|
|Average|287.6ms|13437.2|64.6ms|5734.1|

As expected, the experiment shows that both build times and object file 
sizes were improved by moving imports away from the top level. We 
estimate that eliminating the 10.5x slack dependency fan-in will bring 
import costs down to negligible and also bring object file size down.


Andrei
Dec 23 2016
parent reply Chris Wright <dhasenan gmail.com> writes:
On Fri, 23 Dec 2016 11:25:41 -0500, Andrei Alexandrescu wrote:
 Dependency-Carrying Declarations allow scalable template libraries.
 template libraries
*This* is the part I hadn't noticed before, and it explains the confusion I had. If you *only* apply this to templates, it works. The current situation is a result of where template constraints are located. If there were a `static throw` equivalent for indicating that parameters were invalid, or if template constraints were in an `in` block like contracts, then this wouldn't even have come up. If phobos went for object oriented code instead of templates (as an example, not a recommendation), then this wouldn't be an issue. However, at that point, it would be utterly useless to me. I'm looking at my entire dub package cache, plus the ten-ish most recently updated dub packages: * units-d uses allSatisfy. Once. * vibe has two structs that would benefit, except they're inside a unittest. I've never compiled dub's unittests. Template constraints have little adoption outside phobos. When they *are* used, they tend to use language facilities instead of templates to express the condition. And when a template is used, it tends to be defined in the same module where it's used. I grant that everyone uses phobos, and phobos uses template constraints a lot. If it's *just* a problem inside phobos, though, there's another obvious solution: split up modules that tend to house a lot of template constraints. Split up the modules that use a wide variety of template constraints. Then I can decide whether the convenience of not hunting for narrower imports is worth the extra quarter second of compilation. ---- Now, if you want to apply this to things that are *not* templates, then you could get a lot further. However, you would end up with code that compiles when it wouldn't today. Code that compiles because the portions you use would compile if they were on their own, while other bits wouldn't. That's the status quo for templates, even no-arg templates, but a change from what we currently do everywhere else. And *that* is what would make it equivalent to use static or selective imports. It would also increase the utility from my perspective from "why the hell are we even doing this?" to "that's kinda nice".
Dec 23 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/23/2016 05:33 PM, Chris Wright wrote:
 On Fri, 23 Dec 2016 11:25:41 -0500, Andrei Alexandrescu wrote:
 Dependency-Carrying Declarations allow scalable template libraries.
 template libraries
*This* is the part I hadn't noticed before, and it explains the confusion I had. If you *only* apply this to templates, it works. The current situation is a result of where template constraints are located. If there were a `static throw` equivalent for indicating that parameters were invalid, or if template constraints were in an `in` block like contracts, then this wouldn't even have come up. If phobos went for object oriented code instead of templates (as an example, not a recommendation), then this wouldn't be an issue.
I acknowledge that if the language were defined a different way this issue wouldn't have come up. But this is a truism - one can say that about any issue.
 However, at that point, it would be utterly useless to me. I'm looking at
 my entire dub package cache, plus the ten-ish most recently updated dub
 packages:

 * units-d uses allSatisfy. Once.
 * vibe has two structs that would benefit, except they're inside a
 unittest. I've never compiled dub's unittests.
Fair enough. I reckon a number of traditional ways of designing software would not be helped radically by DIP1005.
 Template constraints have little adoption outside phobos.
That will change. We definitely need to do all we can to support and improve language support for template constraints.
 When they *are*
 used, they tend to use language facilities instead of templates to
 express the condition. And when a template is used, it tends to be
 defined in the same module where it's used.
That may be true for some code today but not for future code. std.traits gets larger and better with more interesting introspection capabilities. I envision introspection as a core differentiating feature of D that will put it ahead of all other languages.
 I grant that everyone uses phobos, and phobos uses template constraints a
 lot. If it's *just* a problem inside phobos, though, there's another
 obvious solution: split up modules that tend to house a lot of template
 constraints. Split up the modules that use a wide variety of template
 constraints.
This point is discussed carefully in DIP1005. Please let me know if anything needs to be added.
 Now, if you want to apply this to things that are *not* templates, then
 you could get a lot further. However, you would end up with code that
 compiles when it wouldn't today. Code that compiles because the portions
 you use would compile if they were on their own, while other bits
 wouldn't. That's the status quo for templates, even no-arg templates, but
 a change from what we currently do everywhere else.

 And *that* is what would make it equivalent to use static or selective
 imports. It would also increase the utility from my perspective from "why
 the hell are we even doing this?" to "that's kinda nice".
Lazier compilation is indeed a projected benefit of this DIP. I did not want to dilute the thrust of the proposal with a remote promise. Andrei
Dec 23 2016
parent reply Chris Wright <dhasenan gmail.com> writes:
tldr: the performance impact of this DIP would be too small to easily 
measure and only impacts 26 declarations referencing 14 templates in the 
standard library anyway.

On Fri, 23 Dec 2016 18:55:25 -0500, Andrei Alexandrescu wrote:
 I grant that everyone uses phobos, and phobos uses template constraints
 a lot. If it's *just* a problem inside phobos, though, there's another
 obvious solution: split up modules that tend to house a lot of template
 constraints. Split up the modules that use a wide variety of template
 constraints.
This point is discussed carefully in DIP1005. Please let me know if anything needs to be added.
An estimate for the actual impact on phobos, since that's your primary driver for the change -- both under the status quo and if we try to split modules. The comparison to mach.d is a strawman. When I thought this might be a problem within phobos, I thought we'd probably split std.traits and maybe std.meta up, probably into 2-5 modules each. Not 150 lines per module; more like 1500 to 4000 lines per module. Then I looked at the code. Phobos has 26 templates that would use this special syntax, referencing 14 distinct templates and CTFE functions in three modules. If you kept the same ratios as are found in mach.d, you'd have one file for every template used as a constraint outside its own module, one for everything else, and as many files again with nothing in them. The templates that have to be parsed are std.traits, std.meta, and std.range.primitives. I wrote a test program that imported them without using them. It compiled in too little time to accurately measure -- `time` reports 0.01s wall time. You want to add new language syntax to save me ten milliseconds during compilation. You hope that template constraints become more common. If they do, this change relies on: * projects defining their own constraints (not just using std.traits) * those constraints being defined in modules other than where they're used * those constraints being defined in very large modules * my code not depending on anything in those constraint-defining modules * my code depending on something in a file with a template with one of those constraints * this whole situation being common enough to give me a death by a thousand papercuts If any project aside from the standard library has a 7k line module defining things mainly used in template constraints, something is seriously weird. But on the plus side, it would only cost me 10 milliseconds. Now, if *dozens* of projects did that, well, I'd be running to the relative simplicity of Java APIs long before I worried about compilation speed.
 Lazier compilation is indeed a projected benefit of this DIP. I did not
 want to dilute the thrust of the proposal with a remote promise.
Lazier compilation would *obviate* this DIP. Lazy compilation of selective and static imports would not require any parser changes and would make a lot of code faster (at the cost of allowing some things that don't compile to start compiling, as does your proposal). You can't get any performance advantages outside templates without implementing lazy imports. Appendix A: templates that would receive 'with(import foo)' in Phobos. I used a relatively simple regex to look for this. If someone put more than one line between a template and its constraints, or comments with certain formatting, I may have missed it. However, that would violate the phobos style guide. This doesn't include templates that use constraints defined in their own modules, because that module must already be parsed and no efficiency gains could be realized. It also omits a few cases where the module has a strong reason to import the dependency aside from template constraints. I believe this was only two constraints defined in std.digest, used in one or two other modules. algorithm/comparison.d:98:template among(values...) experimental/typecons.d:82:private template implementsInterface(Source, Targets...) experimental/typecons.d:94: template implementsInterface() experimental/typecons.d:126:private template implementsInterface(Source, Targets...) experimental/typecons.d:184:template wrap(Targets...) experimental/typecons.d:237: template wrap(Source) range/package.d:2069:template Take(R) range/package.d:3501:template Cycle(R) range/interfaces.d:277:template MostDerivedInputRange(R) range/interfaces.d:336:template InputRangeObject(R) numeric.d:678:template FPTemporary(F) conv.d:3894:template octal(alias decimalInteger) utf.d:1136:package template codeUnitLimit(S) typecons.d:1779:private mixin template RebindableCommon(T, U, alias This) typecons.d:1838:template Rebindable(T) typecons.d:4239:template wrap(Targets...) typecons.d:4252: template wrap(Source) typecons.d:4412:template wrap(Targets...) typecons.d:4429:template unwrap(Target) typecons.d:4461:template unwrap(Target) format.d:657:template FormatSpec(Char) algorithm/iteration.d:1055:template filter(alias predicate) if (is(typeof (unaryFun!predicate))) internal/math/biguintcore.d:81:template maxBigDigits(T) if (isIntegral!T) meta.d:248:package template OldAlias(T) if (!isAggregateType!T || is (Unqual!T == T)) meta.d:254:package template OldAlias(T) if (isAggregateType!T && !is (Unqual!T == T)) utf.d:3542:template byUTF(C) if (isSomeChar!C) Appendix B: templates that would need to be extracted out in phobos, if parsing their modules cost a non-negligible amount of time. std.meta: allSatisfy anySatisfy ApplyLeft std.range.primitives: hasSlicing isInputRange isInfinite std.traits: isAggregateType isAssociativeArray isDynamicArray isFloatingPoint isImplicitlyConvertible isIntegral isMutable isSomeChar
Dec 23 2016
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/23/16 9:23 PM, Chris Wright wrote:
 The comparison to mach.d is a strawman.
The mach.d is given as an example of the approach of breaking code into fine-grained module. No comparison is made or implied.
 Then I looked at the code.

 Phobos has 26 templates that would use this special syntax, referencing
 14 distinct templates and CTFE functions in three modules.
Could you please give more detail about the method you used? What special syntax are you referring to - would that be the "with (import ...)" syntax? If that's the case there's a bunch of stuff missing. Consider e.g.: int cmp(alias pred = "a < b", R1, R2)(R1 r1, R2 r2) if (isInputRange!R1 && isInputRange!R2 && !(isSomeString!R1 && isSomeString!R2)); int cmp(alias pred = "a < b", R1, R2)(R1 r1, R2 r2) if (isSomeString!R1 && isSomeString!R2); These and many like them would use the "with import" construct, wouldn't they? They are missing from your list.
 The templates that have to be parsed are std.traits, std.meta, and
 std.range.primitives.
That's where most introspection primitives for the standard library are situated indeed.
 I wrote a test program that imported them without
 using them. It compiled in too little time to accurately measure --
 `time` reports 0.01s wall time.

 You want to add new language syntax to save me ten milliseconds during
 compilation.
DIP1005 enumerates several benefits of the proposed feature. Speed is not the most important and is not presented as such, but it takes most space because the others are more difficult to experiment with without an experimental implementation.
 You hope that template constraints become more common.
Well, to the extent that is the way correct code is written, that's more than just hope.
 If they do, this
 change relies on:

 * projects defining their own constraints (not just using std.traits)
 * those constraints being defined in modules other than where they're used
 * those constraints being defined in very large modules
 * my code not depending on anything in those constraint-defining modules
 * my code depending on something in a file with a template with one of
 those constraints
 * this whole situation being common enough to give me a death by a
 thousand papercuts
It's not only constraints, it's everything in declarations that relies on entities (e.g. types) defined in other modules. And again the feature's benefits go well beyond making general projects faster to build. A simple way to look at things is - local imports are The Right Thing, for many reasons beyond compilation speed. There needs to be a way to do the right thing for the declaration part as well. (I'm glad we're getting to the point of diminished returns - I recall an argument made a while ago was that, on the contrary, our compilation model cannot possibly scale for large projects.)
 Appendix A: templates that would receive 'with(import foo)' in Phobos.
Could you please provide more detail on how you measured this and what assumptions you made? Most templates in Phobos would receive a with clause, so there's a disconnect somewhere. Andrei
Dec 23 2016
parent Chris Wright <dhasenan gmail.com> writes:
On Fri, 23 Dec 2016 23:41:46 -0500, Andrei Alexandrescu wrote:

 On 12/23/16 9:23 PM, Chris Wright wrote:
 The comparison to mach.d is a strawman.
The mach.d is given as an example of the approach of breaking code into fine-grained module. No comparison is made or implied.
"Assuming module size is a project invariant, the number of files scales roughly with project size. This means mach.d would need 2000 files to scale up to the same size as the D standard library" That's a direct comparison that you made. You might want to remove that from the proposal if you don't want to compare them.
 Could you please give more detail about the method you used?
An insufficient one. (I would think this sort of analysis would generally be the burden of the proposer, generally.) https://gist.github.com/dhasenan/681b5178672556aa0e5ec8fb4c9eae7e uses dmd's json output instead of regular expressions and should be rather better. This shows a total of 35 templates used in template constraints, 124 usages in total. The quick usage breakdown: 1 std.experimental.ndslice.internal 1 std.experimental.ndslice.slice 24 std.meta 31 std.range.primitives 66 std.traits 1 std.typecons Notable mentions: the ndslice templates are used only in other ndslice modules, and the std.typecons reference is from std.experimental.typecons. So it's still ~10ms of overhead, by both our measurements.
 And again the
 feature's benefits go well beyond making general projects faster to
 build.
You may as well remove the parts about compilation efficiency from the proposal entirely, then. The DIP's points here aren't invalid. However, static and selective imports work nearly as well. A doc generator that turns referenced identifiers into links (like dpldocs.info) works nearly as well for reading. For moving files between modules, that's relatively rare, and fixing imports is a small part of it. (In the worst case, I can copy all imports over and use dustmite to reduce them. Or just leave them.) It provides small occasional benefits for a nontrivial maintenance cost. At least it doesn't mandate that I use it.
 A simple way to look at things is - local imports are The Right Thing,
 for many reasons beyond compilation speed. There needs to be a way to do
 the right thing for the declaration part as well.
Function-scoped imports are a good way to hide implementation details from people who don't need them.
Dec 24 2016
prev sibling next sibling parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Saturday, 24 December 2016 at 02:23:53 UTC, Chris Wright wrote:
 Appendix A: templates that would receive 'with(import foo)' in 
 Phobos.

 I used a relatively simple regex to look for this. If someone 
 put more than one line between a template and its constraints, 
 or comments with certain formatting, I may have missed it. 
 However, that would violate the phobos style guide.

 This doesn't include templates that use constraints defined in 
 their own modules, because that module must already be parsed 
 and no efficiency gains could be realized.

 It also omits a few cases where the module has a strong reason 
 to import the dependency aside from template constraints. I 
 believe this was only two constraints defined in std.digest, 
 used in one or two other modules.

 algorithm/comparison.d:98:template among(values...)
 experimental/typecons.d:82:private template 
 implementsInterface(Source,
 Targets...)
 experimental/typecons.d:94:    template implementsInterface()
 experimental/typecons.d:126:private template 
 implementsInterface(Source,
 Targets...)
 experimental/typecons.d:184:template wrap(Targets...)
 experimental/typecons.d:237:        template wrap(Source)
 range/package.d:2069:template Take(R)
 range/package.d:3501:template Cycle(R)
 range/interfaces.d:277:template MostDerivedInputRange(R)
 range/interfaces.d:336:template InputRangeObject(R)
 numeric.d:678:template FPTemporary(F)
 conv.d:3894:template octal(alias decimalInteger)
 utf.d:1136:package template codeUnitLimit(S)
 typecons.d:1779:private mixin template RebindableCommon(T, U, 
 alias This)
 typecons.d:1838:template Rebindable(T)
 typecons.d:4239:template wrap(Targets...)
 typecons.d:4252:    template wrap(Source)
 typecons.d:4412:template wrap(Targets...)
 typecons.d:4429:template unwrap(Target)
 typecons.d:4461:template unwrap(Target)
 format.d:657:template FormatSpec(Char)
 algorithm/iteration.d:1055:template filter(alias predicate) if 
 (is(typeof
 (unaryFun!predicate)))
 internal/math/biguintcore.d:81:template maxBigDigits(T) if 
 (isIntegral!T)
 meta.d:248:package template OldAlias(T) if (!isAggregateType!T 
 || is
 (Unqual!T == T))
 meta.d:254:package template OldAlias(T) if (isAggregateType!T 
 && !is
 (Unqual!T == T))
 utf.d:3542:template byUTF(C) if (isSomeChar!C)


 Appendix B: templates that would need to be extracted out in 
 phobos, if parsing their modules cost a non-negligible amount 
 of time.

 std.meta:
     allSatisfy
     anySatisfy
     ApplyLeft

 std.range.primitives:
     hasSlicing
     isInputRange
     isInfinite

 std.traits:
     isAggregateType
     isAssociativeArray
     isDynamicArray
     isFloatingPoint
     isImplicitlyConvertible
     isIntegral
     isMutable
     isSomeChar
There are a lot of templates in Phobos that never use the template keyword. The proposal doesn't only apply to constraints, it applies to the whole declaration.
Dec 24 2016
parent reply Chris Wright <dhasenan gmail.com> writes:
On Sat, 24 Dec 2016 09:20:19 +0000, John Colvin wrote:
 There are a lot of templates in Phobos that never use the template
 keyword. The proposal doesn't only apply to constraints, it applies to
 the whole declaration.
If you have a better way of estimating the impact, I'd love to see it.
Dec 24 2016
parent reply Joakim <dlang joakim.fea.st> writes:
On Saturday, 24 December 2016 at 17:53:04 UTC, Chris Wright wrote:
 On Sat, 24 Dec 2016 09:20:19 +0000, John Colvin wrote:
 There are a lot of templates in Phobos that never use the 
 template keyword. The proposal doesn't only apply to 
 constraints, it applies to the whole declaration.
If you have a better way of estimating the impact, I'd love to see it.
Can we hold off on the performance discussion? Walter says this DIP isn't hard to implement (https://github.com/dlang/DIPs/pull/51#issuecomment-269077790), so we will run some numbers and see what we get. As you know, I too am skeptical of the benefit but Andrei has shown that import fanout has a non-negligible cost with his latest benchmark, and actual measurement will be the best way to decide.
Dec 24 2016
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/24/2016 02:16 PM, Joakim wrote:
 On Saturday, 24 December 2016 at 17:53:04 UTC, Chris Wright wrote:
 On Sat, 24 Dec 2016 09:20:19 +0000, John Colvin wrote:
 There are a lot of templates in Phobos that never use the template
 keyword. The proposal doesn't only apply to constraints, it applies
 to the whole declaration.
If you have a better way of estimating the impact, I'd love to see it.
Can we hold off on the performance discussion? Walter says this DIP isn't hard to implement (https://github.com/dlang/DIPs/pull/51#issuecomment-269077790), so we will run some numbers and see what we get. As you know, I too am skeptical of the benefit but Andrei has shown that import fanout has a non-negligible cost with his latest benchmark, and actual measurement will be the best way to decide.
Also (from the cycle "sounding like a broken record") the impact is in other dimension than import speed. From DIP1005: ========== [...] Dependency-Carrying Declarations have multiple benefits: * Specifies dependencies at declaration level, not at module level. This allows reasoning about the dependency cost of declarations in separation instead of aggregated at module level. * If all declarations use Dependency-Carrying style and there is no top-level import, human reviewers and maintainers can immediately tell where each symbol in a given declaration comes from. This is a highly nontrivial exercise without specialized editor support in projects that pull several other modules and packages wholesale. Even a project newcomer could gather an understanding of a declaration without needing to absorb an arbitrary amount of implied context from the declaration at the top of the module. * Dependency-Carrying Declarations are easier to move around, making for simpler and faster refactorings. * Dependency-Carrying Declarations allow scalable template libraries. [...] Dependency-Carrying Declarations also have drawbacks: * If most declarations in a module need the same imports, then factoring them outside the declarations at top level is simpler and better than repeating them. * Related, renaming one module is likely to require more edits in a Dependency-Carrying Declarations setup. * Traditional dependency-tracking tools such as make and other build systems assume file-level dependencies and need special tooling (such as rdmd) in order to work efficiently. * Dependencies at the top of a module are easier to inspect quickly than dependencies spread through the module. ========== Andrei
Dec 24 2016
prev sibling parent deadalnix <deadalnix gmail.com> writes:
On Saturday, 24 December 2016 at 19:16:40 UTC, Joakim wrote:
 Can we hold off on the performance discussion?  Walter says 
 this DIP isn't hard to implement 
 (https://github.com/dlang/DIPs/pull/51#issuecomment-269077790), 
 so we will run some numbers and see what we get.  As you know, 
 I too am skeptical of the benefit but Andrei has shown that 
 import fanout has a non-negligible cost with his latest 
 benchmark, and actual measurement will be the best way to 
 decide.
The "problem" is that you'd be essentially measuring how inefficient the current DMD import management is rather than anything else. When you push something on the user, you want to make sure it bring an actual benefit, not just allow to work around lousy existing implementation.
Dec 28 2016
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/23/2016 09:23 PM, Chris Wright wrote:
[abbreviated below]

Upon more investigation, I see a large discrepancy between the findings 
of DIP1005 and yours. So there are a number of claims here, which I'll 
summarize:

 tldr: the performance impact of this DIP would be too small to easily
 measure and only impacts 26 declarations referencing 14 templates in the
 standard library anyway.
 Phobos has 26 templates that would use this special syntax, referencing
 14 distinct templates and CTFE functions in three modules.
 The templates that have to be parsed are std.traits, std.meta, and
 std.range.primitives. I wrote a test program that imported them without
 using them. It compiled in too little time to accurately measure --
 `time` reports 0.01s wall time.

 You want to add new language syntax to save me ten milliseconds during
 compilation.
The findings of DIP1005 are the following: * Importing a single std module also imports on average 10.5 other modules. * Importing a single std module costs on average 64.6 ms. * (Not stated in the DIP) A majority of std templates would acquire inline imports. According to the DIP, one may estimate that the proposed feature would reduce additional imports to 0 and the average time to import a single module by a factor of 10 to under 10 ms. By your estimates: * 26 templates in std need inline imports. * Importing a single std module today would only imports 1-3 other modules most of the time (one or more of std.traits, std.meta, and std.range.primitives). * These additional imports cost in aggregate under 10ms, bringing the average cost of importing a module itself to 54.6 ms. * It follows that the average module takes 5.46 more times to import alone than the sum of std.traits, std.meta, and std.range.primitives (which have a total of 11263 lines, 5x more than the average Phobos module). I don't see how your claims can be simultaneously true with the findings of DIP1005. The scripts that compute those numbers are available with the DIP. Were you able to reproduce them? (I confirm that importing std.traits, std.meta, and std.range.primitives together takes 10ms.) Thanks, Andrei
Dec 24 2016
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 24 December 2016 at 09:34:03 UTC, Andrei 
Alexandrescu wrote:
 (I confirm that importing std.traits, std.meta, and 
 std.range.primitives together takes 10ms.)
while compiling std.traits 6 files are opened and read into memory, taking roughly 300 microseconds, which is not even 0.3% of the time spent. Lexing them requires additionally also about 300 microseconds. So Together that makes up 0.6% of the time spent. The real problem here is _NOT_ opening and lexing files. It is rather eager parsing and sema. If that were made more lazy, we could import half of the world with noticing impact. (Which espcially in std.traits, would not make that much of a difference since every template in there depends on nearly every other template in there)
Dec 24 2016
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 24 December 2016 at 10:54:08 UTC, Stefan Koch wrote:
 300 microseconds, which is not even 0.3% of the time spent.
 Lexing them requires additionally also about 300 microseconds.
 So Together that makes up 0.6% of the time spent.
Of course in the above 3% and 6% are the right numbers. (And still conservative.) Since the are obtained by using a -profile build of dmd.
Dec 24 2016
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/24/2016 05:54 AM, Stefan Koch wrote:
 If that were made more lazy, we could import half of the world with
 noticing impact.
That is what 1005 will bring. -- Andrei
Dec 24 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 24 December 2016 at 14:08:48 UTC, Andrei 
Alexandrescu wrote:
 On 12/24/2016 05:54 AM, Stefan Koch wrote:
 If that were made more lazy, we could import half of the world 
 with
 noticing impact.
That is what 1005 will bring. -- Andrei
A compiler enhancement can do this _without_ a language change.
Dec 24 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/24/16 9:20 AM, Stefan Koch wrote:
 On Saturday, 24 December 2016 at 14:08:48 UTC, Andrei Alexandrescu wrote:
 On 12/24/2016 05:54 AM, Stefan Koch wrote:
 If that were made more lazy, we could import half of the world with
 noticing impact.
That is what 1005 will bring. -- Andrei
A compiler enhancement can do this _without_ a language change.
The language addition has additional benefits as described by DIP1005. -- Andrei
Dec 24 2016
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 24 December 2016 at 15:44:18 UTC, Andrei 
Alexandrescu wrote:
 On 12/24/16 9:20 AM, Stefan Koch wrote:
 On Saturday, 24 December 2016 at 14:08:48 UTC, Andrei 
 Alexandrescu wrote:
 On 12/24/2016 05:54 AM, Stefan Koch wrote:
 If that were made more lazy, we could import half of the 
 world with
 noticing impact.
That is what 1005 will bring. -- Andrei
A compiler enhancement can do this _without_ a language change.
The language addition has additional benefits as described by DIP1005. -- Andrei
I just read over the dip, and it is a giant wall of text. I cannot really make heads or tails of it. Maybe you could write the advantages you hit at in short bullet-point form ?
Dec 24 2016
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/24/2016 12:45 PM, Stefan Koch wrote:
 I just read over the dip, and it is a giant wall of text.
 I cannot really make heads or tails of it.
When it was shorter, people complained it had too little information. If you're using the text form, the nicey formatted version may be helpful: https://github.com/dlang/DIPs/blob/0e15d550d6774dfd40ec78dc201dfad33f47a740/DIPs/DIP1005.md
 Maybe you could write the advantages you hit at in short bullet-point
 form ?
There is. Search for "The analysis above reveals that Dependency-Carrying Declarations have multiple benefits:". Thanks, Andrei
Dec 24 2016
parent Chris Wright <dhasenan gmail.com> writes:
On Sat, 24 Dec 2016 14:10:30 -0500, Andrei Alexandrescu wrote:

 On 12/24/2016 12:45 PM, Stefan Koch wrote:
 I just read over the dip, and it is a giant wall of text.
 I cannot really make heads or tails of it.
When it was shorter, people complained it had too little information.
It's not the length that's giving trouble; it's the organization.
Dec 24 2016
prev sibling parent Chris Wright <dhasenan gmail.com> writes:
On Sat, 24 Dec 2016 17:45:35 +0000, Stefan Koch wrote:
 I just read over the dip, and it is a giant wall of text.
 I cannot really make heads or tails of it.
 Maybe you could write the advantages you hit at in short bullet-point
 form ?
We add the syntax: with (import foo, bar, baz) SingleDeclaration * If you're reading it as source code, you have, usually, 1-3 modules to look in for related definitions, instead of lots (plain imports) or one (static / selective imports). * These modules are listed one line up from the declaration (as with static imports, unlike selective imports). * It doesn't look quite as cluttered as static imports. * It saves a little typing compared to static imports when you use the same import twice in one declaration. * If you need to move this declaration to another module, you don't have to worry about imports (provided you assiduously avoid global imports). You still have to worry about everything the declaration references from the module it was previously defined in. * The compiler can delay importing the module until you use the declaration it's attached to. * It might be slightly easier to implement in DMD than lazy static / selective imports. We extend that syntax: with (import foo) with (import bar, baz) { // multiple declarations // possibly other with-import blocks } * You have even less typing if you reuse the same import in a lot of declarations. * The compiler can still delay importing the module until you use the declaration it's attached to. * You get to have a fun search through the entire module to try to find which modules are imported when reading a declaration, instead of looking up one line (first pass on the syntax), looking at the symbol being used (static imports), or looking at the top of the file (status quo or selective imports). * That also applies when you try to move a declaration to another module.
Dec 24 2016
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Saturday, 24 December 2016 at 15:44:18 UTC, Andrei 
Alexandrescu wrote:
 A compiler enhancement can do this _without_ a language change.
The language addition has additional benefits as described by DIP1005. -- Andrei
Yes, question is, are these specific benefits worth adding a language change ? Right now you got : A/ No language change, get X. B/ Language change, get X and Y. It stand to reason that B should be evaluated on the benefit provided by Y, and Y only, rather than X and Y, as X can be provided without the language change.
Dec 28 2016
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/28/16 10:48 AM, deadalnix wrote:
 On Saturday, 24 December 2016 at 15:44:18 UTC, Andrei Alexandrescu wrote:
 A compiler enhancement can do this _without_ a language change.
The language addition has additional benefits as described by DIP1005. -- Andrei
Yes, question is, are these specific benefits worth adding a language change ? Right now you got : A/ No language change, get X. B/ Language change, get X and Y. It stand to reason that B should be evaluated on the benefit provided by Y, and Y only, rather than X and Y, as X can be provided without the language change.
This is exactly what the DIP describes in detail. It consecrates sections to alternatives. Is anything missing? -- Andrei
Dec 28 2016
parent reply deadalnix <deadalnix gmail.com> writes:
On Wednesday, 28 December 2016 at 23:14:48 UTC, Andrei 
Alexandrescu wrote:
 On 12/28/16 10:48 AM, deadalnix wrote:
 On Saturday, 24 December 2016 at 15:44:18 UTC, Andrei 
 Alexandrescu wrote:
 A compiler enhancement can do this _without_ a language 
 change.
The language addition has additional benefits as described by DIP1005. -- Andrei
Yes, question is, are these specific benefits worth adding a language change ? Right now you got : A/ No language change, get X. B/ Language change, get X and Y. It stand to reason that B should be evaluated on the benefit provided by Y, and Y only, rather than X and Y, as X can be provided without the language change.
This is exactly what the DIP describes in detail. It consecrates sections to alternatives. Is anything missing? -- Andrei
This was more a comment about the current discussion than the DIP. A lot of discussion effort was focused on performance, but most of it can be achieved without language change - as the DIP states. I think the performance gain we are looking at here is marginal, and I don't expect people to change their code to get a marginal benefit, so I suggest the performance aspect of the change to be simply left aside. So the question now is would the added expressivity of per declaration import be worth the language change. I have to admit I'm not convinced either way. Finally, while I proposed a variation of the "with import" combo in the past, I'm now much more convinced that using the plain import syntax, without the ';' is better. I was afraid there was a syntax conflict, but it doesn't looks like there is one. This will not introduce a new syntax, but allow an existing one to be used in a new location. This is, IMO, much more valuable.
Dec 30 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 30 December 2016 at 18:11:54 UTC, deadalnix wrote:

 I think the performance gain we are looking at here is 
 marginal, and I don't expect people to change their code to get 
 a marginal benefit, so I suggest the performance aspect of the 
 change to be simply left aside.
Yes, imports are (in most cases) not performance relevant!
 So the question now is would the added expressivity of per 
 declaration import be worth the language change. I have to 
 admit I'm not convinced either way.
Neither am I.
 Finally, while I proposed a variation of the "with import" 
 combo in the past, I'm now much more convinced that using the 
 plain import syntax, without the ';' is better. I was afraid 
 there was a syntax conflict, but it doesn't looks like there is 
 one. This will not introduce a new syntax, but allow an 
 existing one to be used in a new location. This is, IMO, much 
 more valuable.
I am still not sure what problem it is trying to solve :)
Dec 30 2016
parent reply Chris Wright <dhasenan gmail.com> writes:
On Fri, 30 Dec 2016 18:18:12 +0000, Stefan Koch wrote:
 I am still not sure what problem it is trying to solve :)
* Performance improvements, primarily when a module imports another, bulky module for optional functionality. * Making it easier to locate where things are defined when reading code. * Making it easier to move declarations between files.
Dec 30 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 30 December 2016 at 22:29:18 UTC, Chris Wright wrote:
 * Performance improvements, primarily when a module imports 
 another,
 bulky module for optional functionality.
That is solved by selective imports.
 * Making it easier to locate where things are defined when 
 reading code.
That is solved by selective imports.
 * Making it easier to move declarations between files.
That is solved by selective imports. Inline imports are really just a addition retrive the the deprecated behavior of fullyQualifedNames which would implicitly import.
Dec 30 2016
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Stefan Koch <uplink.coder googlemail.com> wrote:
 On Friday, 30 December 2016 at 22:29:18 UTC, Chris Wright wrote:
 * Performance improvements, primarily when a module imports 
 another,
 bulky module for optional functionality.
That is solved by selective imports.
 * Making it easier to locate where things are defined when 
 reading code.
That is solved by selective imports.
 * Making it easier to move declarations between files.
That is solved by selective imports. Inline imports are really just a addition retrive the the deprecated behavior of fullyQualifedNames which would implicitly import.
The main win, which indeed is not emphasized enough, is better encapsulation. Walter pointed that out, and I will redo the DIP to put that front and center.
Dec 30 2016
next sibling parent ArturG <var.spool.mail700 gmail.com> writes:
On Friday, 30 December 2016 at 23:49:23 UTC, Andrei Alexandrescu 
wrote:

 The main win, which indeed is not emphasized enough, is better 
 encapsulation. Walter pointed that out, and I will redo the DIP 
 to put that front and center.
Maybe i can provide an example where i think DCD's would be usefull. Im experimenting with a more Qt like signals and slots implementation, it introspects some type for function declarations with the ("Signal") uda and mixes them into the current type. It currently looks like this: // moda.d interface TestSignals { import modb: TestType; ("Signal"): void someTestSig(TestType); } // modb.d struct TestType{} // test.d class A : SignalObject { import moda: TestSignals; import modb: TestType; // has to be manually imported for the generated signal mixin signalsOf!(SignalList, TestSignals); interface SignalList { ("Signal"): void foo(int); void bar(string, int = 100); int bar(int a, int b); } void someFun(string s) { s.writeln; } } class B : SignalObject { struct SignalList { ("Signal"): void someSig(string); } mixin signalsOf!SignalList; void fooHandler(int i){ i.writeln; } int onBar(int a, int b){ writeln(a + b); return a + b; } void onBar(string s, int i){ writeln(s, i); } import modb: TestType; void someTestSigHandler(TestType t){ t.writeln; } } class C : SignalObject { import modb: TestType; // has to be manually imported for the generated signal mixin signalsOf!(A, B); // i wanted to support string based signal declaration // but that would require double manual import declarations // import modb: TestType; // enum signalList = // q{ // import modb: TestType; // ("Signal"): // void someSig(TestType); // int someOtherSig(); // } // mixin signalsOf!signalList; } void main(string[] args) { auto a = new A; auto b = new B; auto c = new C; a.connect!"foo"(&b.fooHandler); a.connect(b); // connect all bar's with all onBar's a.foo(66); a.bar("asd"); a.bar(4, 6); c.connect!"someSig"(&b.someSig); b.connect!"someSig"(&a.someFun); c.someSig("emitted by c forwarded by b handled by a"); a.connect!"someTestSig"(&b.someTestSigHandler); c.connect!"someTestSig"(&b.someTestSigHandler); import modb: TestType; TestType t; a.someTestSig(t); c.someTestSig(t); } i assume with DCD's i could remove those manual imports if they get some traits.
Dec 30 2016
prev sibling parent reply Chris Wright <dhasenan gmail.com> writes:
On Fri, 30 Dec 2016 23:49:23 +0000, Andrei Alexandrescu wrote:
 The main win, which indeed is not emphasized enough, is better
 encapsulation. Walter pointed that out, and I will redo the DIP to put
 that front and center.
Encapsulation is an abstraction over several concrete benefits. We're not horribly constrained in time or cognitive effort, so we can talk about the concrete benefits instead of the abstraction.
Dec 30 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/30/16 7:34 PM, Chris Wright wrote:
 On Fri, 30 Dec 2016 23:49:23 +0000, Andrei Alexandrescu wrote:
 The main win, which indeed is not emphasized enough, is better
 encapsulation. Walter pointed that out, and I will redo the DIP to put
 that front and center.
Encapsulation is an abstraction over several concrete benefits. We're not horribly constrained in time or cognitive effort, so we can talk about the concrete benefits instead of the abstraction.
Framing things this way would be really ignorant. DIP1005 can't be in the business of arguing that encapsulation is good by means of examples. First off, there is no undeniable proof that encapsulation is advantageous and not even simple obvious examples; encapsulation has had many opponents, most notably Fred Brooks (the author of "The Mythical Man-Month"), who used a process specifically antithetic to encapsulation in the development of IBM System/360 and OS/360. Far as I recall Fred ultimately ceded the point that encapsulation is superior, but IIRC that was after 2000. I have no doubt there are competent folks out there who think encapsulation is a crock. So it would be goofy if DIP1005 took the onus to show the advantages of improved encapsulation by means of concrete examples (such don't exist outside of large-scale projects for any kind of encapsulation). What DIP1005 can do is show to someone who already believes that encapsulation is good, that the proposed feature improves encapsulation. Andrei
Dec 30 2016
parent reply Chris Wright <dhasenan gmail.com> writes:
On Fri, 30 Dec 2016 21:13:19 -0500, Andrei Alexandrescu wrote:
 DIP1005 can't be in
 the business of arguing that encapsulation is good by means of examples.
Right. I said we should talk about the concrete benefits of the proposal instead of talking about encapsulation.
Dec 30 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/30/16 11:10 PM, Chris Wright wrote:
 On Fri, 30 Dec 2016 21:13:19 -0500, Andrei Alexandrescu wrote:
 DIP1005 can't be in
 the business of arguing that encapsulation is good by means of examples.
Right. I said we should talk about the concrete benefits of the proposal instead of talking about encapsulation.
Why would the DIP hamstring itself by not discussing its most important advantage? -- Andrei
Dec 31 2016
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 31.12.2016 13:23, Andrei Alexandrescu wrote:
 On 12/30/16 11:10 PM, Chris Wright wrote:
 On Fri, 30 Dec 2016 21:13:19 -0500, Andrei Alexandrescu wrote:
 DIP1005 can't be in
 the business of arguing that encapsulation is good by means of examples.
Right. I said we should talk about the concrete benefits of the proposal instead of talking about encapsulation.
Why would the DIP hamstring itself by not discussing its most important advantage? -- Andrei
Because it is a slippery slope if you subscribe to the notion that "D's unit of encapsulation is the module". Do we need to rethink encapsulation in D? Should there be an additional 'internal' visibility modifier that hides members even to other declarations in the same module? On an unrelated note: I'm still not a fan of the with(import) syntax as it morally promotes a lack of turtles (even if not technically so).
Dec 31 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/31/16 9:43 AM, Timon Gehr wrote:
 On an unrelated note: I'm still not a fan of the with(import) syntax as
 it morally promotes a lack of turtles (even if not technically so).
Could you please provide more detail? Thanks! -- Andrei
Jan 04
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 05.01.2017 01:51, Andrei Alexandrescu wrote:
 On 12/31/16 9:43 AM, Timon Gehr wrote:
 On an unrelated note: I'm still not a fan of the with(import) syntax as
 it morally promotes a lack of turtles (even if not technically so).
Could you please provide more detail? Thanks! -- Andrei
The declaration with(import foo){ ... } looks like an orthogonal combination of some import expression and the usual with(foo){ ... } statement. (This is true for all other statements and expressions with similar syntax.) However, this is not in fact true here, the two constructs have different scoping rules. Hence I think that the 'with(import foo){ ... }'-syntax would be better split into two orthogonal features: 1. allow 'import foo' as an expression that evaluates to the corresponding module symbol. 2. add 'static with' that is basically like 'with' but is a declaration and has different scoping rules. I.e., my objection is that 'with' should not become 'static' just because it is applied to an import expression.
Jan 04
prev sibling parent Chris Wright <dhasenan gmail.com> writes:
On Sat, 31 Dec 2016 07:23:14 -0500, Andrei Alexandrescu wrote:

 On 12/30/16 11:10 PM, Chris Wright wrote:
 On Fri, 30 Dec 2016 21:13:19 -0500, Andrei Alexandrescu wrote:
 DIP1005 can't be in the business of arguing that encapsulation is good
 by means of examples.
Right. I said we should talk about the concrete benefits of the proposal instead of talking about encapsulation.
Why would the DIP hamstring itself by not discussing its most important advantage? -- Andrei
DIP1005 provides concrete advantages. We can productively discuss its concrete advantages. "Encapsulation" is an abstraction over a number of features. Many features can increase potential encapsulation, and this is only one of them. Encapsulation can provide a wide range of benefits, and this feature only provides one or two of them. It saves us no effort to say "encapsulation". We still have to peel back that label to see what's underneath and evaluate that. But "encapsulation" is an applause light, so we automatically think better of the proposal than if we were merely looking at its concrete benefits.
Dec 31 2016
prev sibling parent reply Chris Wright <dhasenan gmail.com> writes:
On Fri, 30 Dec 2016 22:42:39 +0000, Stefan Koch wrote:

 On Friday, 30 December 2016 at 22:29:18 UTC, Chris Wright wrote:
 * Performance improvements, primarily when a module imports another,
 bulky module for optional functionality.
That is solved by selective imports.
Well, not today, but it could be. Same with static imports. Both accomplish this goal better than DIP1005.
 * Making it easier to locate where things are defined when reading
 code.
That is solved by selective imports.
Static imports do that better than selective imports, though it's more typing. With selective imports, you can either find the declaration in the current module, or find the symbol in an import list. If it's imported, and the module author is using standard code formatting, you will find the import at the top of the module, which will be fast. This is less true with arbitrarily scoped imports. With DIP1005, you rely on there also being selective or static imports. If the module author didn't use them, then you have to pull out grep.
 * Making it easier to move declarations between files.
That is solved by selective imports.
That helps a bit. You still need to exercise discipline in only using selective imports for what you need for visible declarations. Static imports work about as well, except you can probably write a complicated regex replacement to extract out the imports you need. DIP1005 is a solid improvement here. Unfortunately, this is a rather marginal usecase. -- Another thing that just occurred to me: if you're modifying a function signature and that brings in another imported symbol, you don't have to move to the top of the file or struct to add the necessary import. It's a small thing for vim users, who are used to using marks and fast movement commands, but if you're using, say, VSCode, that might be painful.
Dec 30 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/30/16 7:32 PM, Chris Wright wrote:
 On Fri, 30 Dec 2016 22:42:39 +0000, Stefan Koch wrote:

 On Friday, 30 December 2016 at 22:29:18 UTC, Chris Wright wrote:
 * Performance improvements, primarily when a module imports another,
 bulky module for optional functionality.
That is solved by selective imports.
Well, not today, but it could be. Same with static imports. Both accomplish this goal better than DIP1005.
Is this fact or opinion? If the former, could you please point where DIP1005 is getting it wrong. Thanks.
 * Making it easier to locate where things are defined when reading
 code.
That is solved by selective imports.
Static imports do that better than selective imports, though it's more typing.
So whether that's overall better is not settled, is it? We can't really define "better" as whatever each participant believes.
 With selective imports, you can either find the declaration in the
 current module, or find the symbol in an import list. If it's imported,
 and the module author is using standard code formatting, you will find
 the import at the top of the module, which will be fast. This is less
 true with arbitrarily scoped imports.

 With DIP1005, you rely on there also being selective or static imports.
 If the module author didn't use them, then you have to pull out grep.
DIP1005 allows the declaration to encapsulate its own dependencies. Of course if top-level imports are also present, the benefit erodes.
 * Making it easier to move declarations between files.
That is solved by selective imports.
That helps a bit. You still need to exercise discipline in only using selective imports for what you need for visible declarations. Static imports work about as well, except you can probably write a complicated regex replacement to extract out the imports you need. DIP1005 is a solid improvement here. Unfortunately, this is a rather marginal usecase. -- Another thing that just occurred to me: if you're modifying a function signature and that brings in another imported symbol, you don't have to move to the top of the file or struct to add the necessary import. It's a small thing for vim users, who are used to using marks and fast movement commands, but if you're using, say, VSCode, that might be painful.
That's one of the many benefits of encapsulation. Andrei
Dec 30 2016
parent reply Chris Wright <dhasenan gmail.com> writes:
On Fri, 30 Dec 2016 20:56:54 -0500, Andrei Alexandrescu wrote:

 On 12/30/16 7:32 PM, Chris Wright wrote:
 On Fri, 30 Dec 2016 22:42:39 +0000, Stefan Koch wrote:

 On Friday, 30 December 2016 at 22:29:18 UTC, Chris Wright wrote:
 * Performance improvements, primarily when a module imports another,
 bulky module for optional functionality.
That is solved by selective imports.
Well, not today, but it could be. Same with static imports. Both accomplish this goal better than DIP1005.
Is this fact or opinion? If the former, could you please point where DIP1005 is getting it wrong. Thanks.
Consider: with(import std.range.primitives, std.traits) { template Foo(Range) if (isForwardRange!Range && !isArray!Range) {} template Bar(Range) if (isArray!Range) {} } If I invoke Bar, the compiler must read and parse both std.range.primitives and std.traits. With static imports: static import std.range.primitives, std.traits; template Foo(Range) if (std.range.primitives.isForwardRange!Range && !std.traits.isArray!Range) {} template Bar(Range) if (std.traits.isArray!Range) {} If I invoke Bar, the compiler must read and parse std.traits but not std.range.primitives because it is unambiguous that isArray is in std.traits, not std.range.primivites. With selective imports likewise. You're going to respond that I'm only complaining that people can abuse the syntax. Consider that it also happens if I change the code and forget to update the imports. Also consider that people are likely to incorporate several related definitions that tend to use the same imports in order to save typing. "Be more virtuous" isn't a mantra that actually leads to better code.
 * Making it easier to locate where things are defined when reading
 code.
That is solved by selective imports.
Static imports do that better than selective imports, though it's more typing.
So whether that's overall better is not settled, is it? We can't really define "better" as whatever each participant believes.
Whether the feature is overall better is not settled. Whether locating where symbols are defined is easier with DIP1005 than with static imports *is*. Static imports tell you exactly where the symbol is defined at a glance, whereas with DIP1005, you have to look through at least two modules.
 With selective imports, you can either find the declaration in the
 current module, or find the symbol in an import list. If it's imported,
 and the module author is using standard code formatting, you will find
 the import at the top of the module, which will be fast. This is less
 true with arbitrarily scoped imports.

 With DIP1005, you rely on there also being selective or static imports.
 If the module author didn't use them, then you have to pull out grep.
DIP1005 allows the declaration to encapsulate its own dependencies. Of course if top-level imports are also present, the benefit erodes.
Consider: // tons of code with (import std.traits) template Bar(Range) if (isArray!Range && !isAbstractClass!Range) {} // tons more code I can reasonably guess that the import is required for at least one of isArray and isAbstractClass, though for all I know, if I'm unfamiliar with std.traits, the import is unnecessary and both isArray and isAbstractClass are defined in the current module. You can fix that with a static or selective import: with (import std.traits : isArray, isAbstractClass) template Bar(Range) if (isArray!Range && !isAbstractClass!Range) {} Far less ambiguous, but your examples all have unmodified with-imports, and you seem less than keen about selective imports.
 -- Another thing that just occurred to me: if you're modifying a
 function signature and that brings in another imported symbol, you
 don't have to move to the top of the file or struct to add the
 necessary import. It's a small thing for vim users, who are used to
 using marks and fast movement commands, but if you're using, say,
 VSCode, that might be painful.
That's one of the many benefits of encapsulation.
It is a benefit of this DIP. There are many ways to achieve encapsulation, and there are many effects of encapsulation, and most of that is irrelevant to this discussion. FWIW, you could also use current imports immediately before the relevant declaration. That would be a bad idea in general because it makes it much harder to locate imports when trying to locate a definition.
Dec 31 2016
parent Chris Wright <dhasenan gmail.com> writes:
On Sat, 31 Dec 2016 16:41:16 +0000, Chris Wright wrote:
 Far less ambiguous, but your examples all have unmodified with-imports,
 and you seem less than keen about selective imports.
Correction: *tend to* have unmodified with-imports.
Dec 31 2016
prev sibling parent reply Chris Wright <dhasenan gmail.com> writes:
On Wed, 28 Dec 2016 15:48:46 +0000, deadalnix wrote:

 On Saturday, 24 December 2016 at 15:44:18 UTC, Andrei Alexandrescu
 wrote:
 A compiler enhancement can do this _without_ a language change.
The language addition has additional benefits as described by DIP1005. -- Andrei
Yes, question is, are these specific benefits worth adding a language change ?
And the associated detriments. Just imagine opening up phobos and seeing: // 1200 lines of code with (import foo) { // 300 lines of declarations... with (import bar) { // 250 lines of additional declarations... } } All mixed in with structs and functions and conditional compilation, so you have a sea of curly braces to wade through and indentation is only a mild help.
Dec 28 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/28/16 9:47 PM, Chris Wright wrote:
 On Wed, 28 Dec 2016 15:48:46 +0000, deadalnix wrote:

 On Saturday, 24 December 2016 at 15:44:18 UTC, Andrei Alexandrescu
 wrote:
 A compiler enhancement can do this _without_ a language change.
The language addition has additional benefits as described by DIP1005. -- Andrei
Yes, question is, are these specific benefits worth adding a language change ?
And the associated detriments. Just imagine opening up phobos and seeing: // 1200 lines of code with (import foo) { // 300 lines of declarations... with (import bar) { // 250 lines of additional declarations... } } All mixed in with structs and functions and conditional compilation, so you have a sea of curly braces to wade through and indentation is only a mild help.
The D Standard Library will generally specify dependencies with declarations. If the point was that the feature can be misused or abused if one really tries, I agree. -- Andrei
Dec 29 2016
prev sibling parent reply Martin Nowak <code dawg.eu> writes:
On Saturday, 24 December 2016 at 10:54:08 UTC, Stefan Koch wrote:
 If that were made more lazy, we could import half of the world 
 with noticing impact.

 (Which espcially in std.traits, would not make that much of a 
 difference since every template in there depends on nearly 
 every other template in there)
Also the established technique of serializing precompiled AST (after semantic3) of modules to a cache should be applicable as well. Cross-posting from https://github.com/dlang/DIPs/pull/51#issuecomment-269107966, b/c it wasn't answered yet. Were any other means considered? This is proposing to add plenty of additional annotations only to speed up compilation, but none of the classical tools for pre-compilation were assessed. Since D's modules don't have the header problem, even pre-compilation and reuse of semantic3 should be possible, or not?
Dec 31 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/31/2016 12:20 PM, Martin Nowak wrote:
 On Saturday, 24 December 2016 at 10:54:08 UTC, Stefan Koch wrote:
 If that were made more lazy, we could import half of the world with
 noticing impact.

 (Which espcially in std.traits, would not make that much of a
 difference since every template in there depends on nearly every other
 template in there)
Also the established technique of serializing precompiled AST (after semantic3) of modules to a cache should be applicable as well. Cross-posting from https://github.com/dlang/DIPs/pull/51#issuecomment-269107966, b/c it wasn't answered yet.
No worries, I'll make a pass (a rewrite, really) taking all feedback into account.
 Were any other means considered? This is proposing to add plenty of
 additional annotations only to speed up compilation, but none of the
 classical tools for pre-compilation were assessed.
 Since D's modules don't have the header problem, even pre-compilation
 and reuse of semantic3 should be possible, or not?
DIP1005 gives consideration to the speed of compilation aspect in larger proportion than speed's importance; the first and foremost benefit of DIP1005 is it closes the gap on dependency encapsulation, which had been very successfully narrowed by local imports. However, the DIP will keep the experiments and results on speed measurements because they are relevant to it and any related methods of lazily loading of modules. Regarding the ongoing doubts about the advantages of inline imports: they are first and foremost a completion of the nested import feature. As such, most, if not all, arguments against inline imports apply equally to nested imports. Come to think of it, lazy imports vs nested imports: * same improvement in compilation speed? check * no language changes? check * no nasty bugs in the aftermath (such as the infamous https://issues.dlang.org/show_bug.cgi?id=10378)? check * scalable builds? check Yet local imports are overwhelmingly superior to lazy imports because of one thing: they localize dependencies. They introduce modularity and its ancillary perks (fast and scalable builds, easier review and refactoring) not by engineering, but by organically placing dependencies spatially with their dependents. (The scope statement does the same thing with temporal dependencies.) That the DIP does not make it clear that it is a necessary and sufficient extension of local imports is a problem with it. I now am really glad we slipped local imports before the formalization of DIPs. The feature could have been easily demeaned out of existence. Allow me to make an appeal regarding the review of any DIP. There seems to be a tendency of some reviewers to get attached and emotionally invested to their opinion, to the extent they'd be hurt by being "wrong" and would go to great lengths to argue they're "right". This has obvious negative effects on the entire process. Please don't. There's no loss of face to worry about. The only commitment we all should have is to the good of the D language. If DIP1005 reaches the conclusion of its own uselessness, I'd be the first one to write it up and close the PR. Thanks, Andrei
Jan 02
next sibling parent reply Dominikus Dittes Scherkl <Dominikus.Scherkl continental-corporation.com> writes:
On Monday, 2 January 2017 at 21:23:19 UTC, Andrei Alexandrescu 
wrote:
 DIP1005 gives consideration to the speed of compilation aspect 
 in larger proportion than speed's importance; the first and 
 foremost benefit of DIP1005 is it closes the gap on dependency 
 encapsulation, which had been very successfully narrowed by 
 local imports.
I love that idea. But I still can't see why this requires any new syntax. Simply extending the scope of local inports to include the function header is enough. Only for .di-generation it may be useful to move all local imports to the declaration (maybe with this new syntax "with" before it) - but that should be done with ALL local imports, because today the .di-files are incomplete and will stay so if the new syntax is introduced but "old-style" local imports still valid and not exported to the .di. Or the old local imports become deprecated together with the introduction of the new "with" syntax and vanish soon after that.
Jan 02
next sibling parent Chris Wright <dhasenan gmail.com> writes:
On Tue, 03 Jan 2017 07:19:41 +0000, Dominikus Dittes Scherkl wrote:

 On Monday, 2 January 2017 at 21:23:19 UTC, Andrei Alexandrescu wrote:
 DIP1005 gives consideration to the speed of compilation aspect in
 larger proportion than speed's importance; the first and foremost
 benefit of DIP1005 is it closes the gap on dependency encapsulation,
 which had been very successfully narrowed by local imports.
I love that idea. But I still can't see why this requires any new syntax. Simply extending the scope of local inports to include the function header is enough.
You mean: Socket connect() { import std.socket; return new Socket(); } Which should already work if you used auto for the return type.
Jan 03
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 01/03/2017 02:19 AM, Dominikus Dittes Scherkl wrote:
 On Monday, 2 January 2017 at 21:23:19 UTC, Andrei Alexandrescu wrote:
 DIP1005 gives consideration to the speed of compilation aspect in
 larger proportion than speed's importance; the first and foremost
 benefit of DIP1005 is it closes the gap on dependency encapsulation,
 which had been very successfully narrowed by local imports.
I love that idea. But I still can't see why this requires any new syntax. Simply extending the scope of local inports to include the function header is enough.
I'll mention this possibility in the DIP.
 Only for .di-generation it may be useful to move all local imports to
 the declaration (maybe with this new syntax "with" before it) - but that
 should be done with ALL local imports, because today the .di-files are
 incomplete and will stay so if the new syntax is introduced but
 "old-style" local imports still valid and not exported to the .di.
 Or the old local imports become deprecated together with the
 introduction of the new "with" syntax and vanish soon after that.
Local imports don't pertain to the interface, they are an implementation detail. Why would those be made a part of the .di? Andrei
Jan 03
parent Dominikus Dittes Scherkl <Dominikus.Scherkl continental-corporation.com> writes:
On Tuesday, 3 January 2017 at 12:57:22 UTC, Andrei Alexandrescu 
wrote:
 On 01/03/2017 02:19 AM, Dominikus Dittes Scherkl wrote:
 On Monday, 2 January 2017 at 21:23:19 UTC, Andrei Alexandrescu 
 wrote:
 DIP1005 gives consideration to the speed of compilation 
 aspect in
 larger proportion than speed's importance; the first and 
 foremost
 benefit of DIP1005 is it closes the gap on dependency 
 encapsulation,
 which had been very successfully narrowed by local imports.
I love that idea. But I still can't see why this requires any new syntax. Simply extending the scope of local inports to include the function header is enough.
I'll mention this possibility in the DIP.
Good.
 Only for .di-generation it may be useful to move all local 
 imports to
 the declaration (maybe with this new syntax "with" before it) 
 - but that
 should be done with ALL local imports, because today the 
 .di-files are
 incomplete and will stay so if the new syntax is introduced but
 "old-style" local imports still valid and not exported to the 
 .di.
 Or the old local imports become deprecated together with the
 introduction of the new "with" syntax and vanish soon after 
 that.
Local imports don't pertain to the interface, they are an implementation detail. Why would those be made a part of the .di?
.di are needed to ship with a library. If a function locally imports some type, the library is dependant on that import, so if I want to use the library, I have to install all stuff it depends on too. And to find out, what exactly the library depends on the .di is the place I look, so there need to be mentioned each and every import - no matter how deeply local that may be. If that is not the case it renders .di-files completely useless to me (as they are at the moment), because I need to find out the dependencies somewhere else (e.g. in some documentation of the library). While it is likely that the dependencies of a library may be documented somewhere, this is not guaranteed. But .di-files are guaranteed to ship with a lib because else it cannot link - at least if it contains any templates, which is about 100% sure for a language like D. So I would like to see local imports in the .di-file, even if they are not strictly needed to compile because the imported types are not exposed in the function signature.
Jan 04
prev sibling next sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Monday, 2 January 2017 at 21:23:19 UTC, Andrei Alexandrescu 
wrote:
 Regarding the ongoing doubts about the advantages of inline 
 imports: they are first and foremost a completion of the nested 
 import feature. As such, most, if not all, arguments against 
 inline imports apply equally to nested imports. Come to think 
 of it, lazy imports vs nested imports:

 * same improvement in compilation speed? check
 * no language changes? check
 * no nasty bugs in the aftermath (such as the infamous 
 https://issues.dlang.org/show_bug.cgi?id=10378)? check
 * scalable builds? check

 Yet local imports are overwhelmingly superior to lazy imports 
 because of one thing: they localize dependencies. They 
 introduce modularity and its ancillary perks (fast and scalable 
 builds, easier review and refactoring) not by engineering, but 
 by organically placing dependencies spatially with their 
 dependents. (The scope statement does the same thing with 
 temporal dependencies.) That the DIP does not make it clear 
 that it is a necessary and sufficient extension of local 
 imports is a problem with it.

 I now am really glad we slipped local imports before the 
 formalization of DIPs. The feature could have been easily 
 demeaned out of existence.
Except that almost nobody has argued against local imports. Rather, the argument is that local imports mostly solved this problem, so why bother adding new syntax for the dozen remaining symbols from 2-3 modules that are commonly used in template constraints? Arguing that local imports have been successful so we should simply do more of it is not a good argument, as there comes a point of diminishing returns. You need to show that there are still worthile gains to be made from changing the language again, which is why I want to benchmark this feature with Walter before deciding.
 Allow me to make an appeal regarding the review of any DIP. 
 There seems to be a tendency of some reviewers to get attached 
 and emotionally invested to their opinion, to the extent they'd 
 be hurt by being "wrong" and would go to great lengths to argue 
 they're "right". This has obvious negative effects on the 
 entire process. Please don't. There's no loss of face to worry 
 about. The only commitment we all should have is to the good of 
 the D language. If DIP1005 reaches the conclusion of its own 
 uselessness, I'd be the first one to write it up and close the 
 PR.
We could level this analysis back at you: you consider this DIP so "obvious" that you are not engaging with our concerns, making flip, incorrect remarks about how we would have bikeshedded local imports also. In the end, this is a minor DIP that is easily bikeshedded, as everybody can grasp it and have an opinion on it. I have refrained from commenting recently because I will let benchmarking settle it for me. Obviously, that won't suffice for others.
Jan 03
next sibling parent Chris Wright <dhasenan gmail.com> writes:
On Tue, 03 Jan 2017 08:10:04 +0000, Joakim wrote:
 Except that almost nobody has argued against local imports. Rather, the
 argument is that local imports mostly solved this problem, so why bother
 adding new syntax for the dozen remaining symbols from 2-3 modules that
 are commonly used in template constraints?
That was my misunderstanding, and it only applied to the speed argument anyway. This is usable with top-level functions, top-level templates, and classes with inheritance.
Jan 03
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 01/03/2017 03:10 AM, Joakim wrote:
 On Monday, 2 January 2017 at 21:23:19 UTC, Andrei Alexandrescu wrote:
 Regarding the ongoing doubts about the advantages of inline imports:
 they are first and foremost a completion of the nested import feature.
 As such, most, if not all, arguments against inline imports apply
 equally to nested imports. Come to think of it, lazy imports vs nested
 imports:

 * same improvement in compilation speed? check
 * no language changes? check
 * no nasty bugs in the aftermath (such as the infamous
 https://issues.dlang.org/show_bug.cgi?id=10378)? check
 * scalable builds? check

 Yet local imports are overwhelmingly superior to lazy imports because
 of one thing: they localize dependencies. They introduce modularity
 and its ancillary perks (fast and scalable builds, easier review and
 refactoring) not by engineering, but by organically placing
 dependencies spatially with their dependents. (The scope statement
 does the same thing with temporal dependencies.) That the DIP does not
 make it clear that it is a necessary and sufficient extension of local
 imports is a problem with it.

 I now am really glad we slipped local imports before the formalization
 of DIPs. The feature could have been easily demeaned out of existence.
Except that almost nobody has argued against local imports.
I don't have time to research this, but my recollection is that at least some framed the bugs regarding lookups as a fundamental problem of local imports, not a simple matter of getting it right. Overall, yes, local imports have been a success (really saving scalability of large project builds which looked pretty bleak at a time), which should increase trust in the authors of the feature... hmmm... :o)
 Rather, the
 argument is that local imports mostly solved this problem, so why bother
 adding new syntax for the dozen remaining symbols from 2-3 modules that
 are commonly used in template constraints?
I understand. It is my humble opinion that we are "mostly pregnant" for as long as we require top-level imports. The real change of phase occurs when there are ZERO imports at top level. That's the prize DIP1005 is after.
 Arguing that local imports have been successful so we should simply do
 more of it is not a good argument, as there comes a point of diminishing
 returns.  You need to show that there are still worthile gains to be
 made from changing the language again, which is why I want to benchmark
 this feature with Walter before deciding.
You can be reasonably certain the benchmarks will improve as projected - starting in proportion to the the transitive fanout of top-level imports (10.5x for the standard library) and going down thereafter with the size, complexity, and actual dependencies of the module being compiled. All in all measurable but not dramatic. The same improvement will be brought about by lazy imports, and it won't be the deal maker/breaker. If you're waiting for the numbers to get convinced of anything, you already consider DIP1005 useless.
 Allow me to make an appeal regarding the review of any DIP. There
 seems to be a tendency of some reviewers to get attached and
 emotionally invested to their opinion, to the extent they'd be hurt by
 being "wrong" and would go to great lengths to argue they're "right".
 This has obvious negative effects on the entire process. Please don't.
 There's no loss of face to worry about. The only commitment we all
 should have is to the good of the D language. If DIP1005 reaches the
 conclusion of its own uselessness, I'd be the first one to write it up
 and close the PR.
We could level this analysis back at you: you consider this DIP so "obvious" that you are not engaging with our concerns, making flip, incorrect remarks about how we would have bikeshedded local imports also.
I apologize it my remarks seem flippant, though I honestly have difficulty finding evidence of that. All I did was note that many of the arguments pitching lazy imports as a better solution than DIP1005 (except of course for the Amdahl one) apply directly to local imports.
 In the end, this is a minor DIP that is easily bikeshedded, as everybody
 can grasp it and have an opinion on it.  I have refrained from
 commenting recently because I will let benchmarking settle it for me.
 Obviously, that won't suffice for others.
I do agree that if framed as a modest improvement in build economics, it is quite unimportant. But that's a problem with the DIP; its main strength is better encapsulation, which is no small thing. Andrei
Jan 03
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 01/03/2017 03:07 PM, Andrei Alexandrescu wrote:
 If you're waiting for the numbers to get convinced of anything, you
 already consider DIP1005 useless.
Eh, sorry, this came out wrong - as if I'm trying to put words in someone's mouth. I meant to say: "If you're waiting for the numbers to get convinced of anything, they won't be overwhelming so you may already write off DIP1005 as useless." -- Andrei
Jan 03
prev sibling parent Joakim <dlang joakim.fea.st> writes:
On Tuesday, 3 January 2017 at 20:07:59 UTC, Andrei Alexandrescu 
wrote:
 Arguing that local imports have been successful so we should 
 simply do
 more of it is not a good argument, as there comes a point of 
 diminishing
 returns.  You need to show that there are still worthile gains 
 to be
 made from changing the language again, which is why I want to 
 benchmark
 this feature with Walter before deciding.
You can be reasonably certain the benchmarks will improve as projected - starting in proportion to the the transitive fanout of top-level imports (10.5x for the standard library) and going down thereafter with the size, complexity, and actual dependencies of the module being compiled. All in all measurable but not dramatic. The same improvement will be brought about by lazy imports, and it won't be the deal maker/breaker. If you're waiting for the numbers to get convinced of anything, you already consider DIP1005 useless.
I don't need dramatic, a solid win is fine. A small 10% or less improvement in the speed of the import phase of compilation probably isn't worth it.
 Rather, the
 argument is that local imports mostly solved this problem, so 
 why bother
 adding new syntax for the dozen remaining symbols from 2-3 
 modules that
 are commonly used in template constraints?
I understand. It is my humble opinion that we are "mostly pregnant" for as long as we require top-level imports. The real change of phase occurs when there are ZERO imports at top level. That's the prize DIP1005 is after.
If you believe the benchmarks won't be dramatically different, I wonder why you think there will be a phase change.
 In the end, this is a minor DIP that is easily bikeshedded, as 
 everybody
 can grasp it and have an opinion on it.  I have refrained from
 commenting recently because I will let benchmarking settle it 
 for me.
 Obviously, that won't suffice for others.
I do agree that if framed as a modest improvement in build economics, it is quite unimportant. But that's a problem with the DIP; its main strength is better encapsulation, which is no small thing.
Encapsulation is not an end in itself. You have listed four main advantages of this change in the DIP. I have noted that I don't find the first three relating to reasoning and refactoring worth adding syntax for, while admitting that may be subjective. The fourth talks about scalability, which I interpreted as either compilation speed or some other measurable gain, hence my asking for benchmarks. If scalability refers to something else- it can't be reasoning or refactoring as those are listed as separate advantages- I'd like to hear what it is. If it's measurable, I'd like to measure it first.
Jan 05
prev sibling next sibling parent reply Paulo Pinto <pjmlp progrools.org> writes:
On Monday, 2 January 2017 at 21:23:19 UTC, Andrei Alexandrescu 
wrote:
 On 12/31/2016 12:20 PM, Martin Nowak wrote:
 On Saturday, 24 December 2016 at 10:54:08 UTC, Stefan Koch 
 wrote:
 If that were made more lazy, we could import half of the 
 world with
 noticing impact.

 (Which espcially in std.traits, would not make that much of a
 difference since every template in there depends on nearly 
 every other
 template in there)
Also the established technique of serializing precompiled AST (after semantic3) of modules to a cache should be applicable as well. Cross-posting from https://github.com/dlang/DIPs/pull/51#issuecomment-269107966, b/c it wasn't answered yet.
No worries, I'll make a pass (a rewrite, really) taking all feedback into account.
 Were any other means considered? This is proposing to add 
 plenty of
 additional annotations only to speed up compilation, but none 
 of the
 classical tools for pre-compilation were assessed.
 Since D's modules don't have the header problem, even 
 pre-compilation
 and reuse of semantic3 should be possible, or not?
DIP1005 gives consideration to the speed of compilation aspect in larger proportion than speed's importance; the first and foremost benefit of DIP1005 is it closes the gap on dependency encapsulation, which had been very successfully narrowed by local imports. However, the DIP will keep the experiments and results on speed measurements because they are relevant to it and any related methods of lazily loading of modules. Regarding the ongoing doubts about the advantages of inline imports: they are first and foremost a completion of the nested import feature. As such, most, if not all, arguments against inline imports apply equally to nested imports. Come to think of it, lazy imports vs nested imports: * same improvement in compilation speed? check * no language changes? check * no nasty bugs in the aftermath (such as the infamous https://issues.dlang.org/show_bug.cgi?id=10378)? check * scalable builds? check Yet local imports are overwhelmingly superior to lazy imports because of one thing: they localize dependencies. They introduce modularity and its ancillary perks (fast and scalable builds, easier review and refactoring) not by engineering, but by organically placing dependencies spatially with their dependents. (The scope statement does the same thing with temporal dependencies.) That the DIP does not make it clear that it is a necessary and sufficient extension of local imports is a problem with it. I now am really glad we slipped local imports before the formalization of DIPs. The feature could have been easily demeaned out of existence. Allow me to make an appeal regarding the review of any DIP. There seems to be a tendency of some reviewers to get attached and emotionally invested to their opinion, to the extent they'd be hurt by being "wrong" and would go to great lengths to argue they're "right". This has obvious negative effects on the entire process. Please don't. There's no loss of face to worry about. The only commitment we all should have is to the good of the D language. If DIP1005 reaches the conclusion of its own uselessness, I'd be the first one to write it up and close the PR. Thanks, Andrei
Allow me just to share a worthless outsider opinion. I never contributed anything worthwhile and decided it was better to just focus on JVM, .NET languages., alongside C++, as those are the skills I get paid for, thus stop polluting D forums. Looking from the outside, and watching what was reached from 2016 roadmap, it is clear the DIPs evaluated thus dar aren't about fixing the library or runtime issues that prevent D's adoption at large as a systems programming language. Meanwhile Swift, Go and Rust have a clear roadmap how their future is supposed to look like, and drive just in that direction, with C++ taking all remaining good D ideas. This DIP discussion and the latest ones about splitting the runtime again, don't do anything to earn D any credibility it might still have left.
Jan 03
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 01/03/2017 04:28 AM, Paulo Pinto wrote:
 Looking from the outside, and watching what was reached from 2016
 roadmap, it is clear the DIPs evaluated thus dar aren't about fixing the
 library or runtime issues that prevent D's adoption at large as a
 systems programming language.

 Meanwhile Swift, Go and Rust have a clear roadmap how their future is
 supposed to look like, and drive just in that direction, with C++ taking
 all remaining good D ideas.

 This DIP discussion and the latest ones about splitting the runtime
 again, don't do anything to earn D any credibility it might still have
 left.
Thanks for taking the time to write this. All languages have an improvement process, which runs as a background task that is not always correlated with the current principal objectives of the language's leadership. We have recently improved our DIP process, but positive examples of compelling DIPs are missing. We were facing the problem that upcoming DIPs, if unguided, could create busywork for the leadership on trivial matters. Whether approved, rejected, or put on hold, DIP1005 will serve as a yardstick for future DIPs regarding the necessary work involved in defining the problem, investigating it as rigorously as possible, comparing with alternatives, and defining the feature. Walter and I do agree that the DIP is ahead of its time because (a) there are more pressing and more important matters to keep me busy, and (b) encapsulation in D is a larger topic that we haven't yet put front and center because of (a). For DIP1005 I invite interested collaborators to contact me about future iterations; it is important for the matter of encapsulation and also for the meta-challenge of setting an example for other DIPs to follow. Thanks. Andrei
Jan 03
prev sibling parent Joakim <dlang joakim.fea.st> writes:
On Tuesday, 3 January 2017 at 09:28:06 UTC, Paulo Pinto wrote:
 Allow me just to share a worthless outsider opinion.

 I never contributed anything worthwhile and decided it was 
 better to just focus on JVM, .NET languages., alongside C++, as 
 those are the skills I get paid for, thus stop polluting D 
 forums.

 Looking from the outside, and watching what was reached from 
 2016 roadmap, it is clear the DIPs evaluated thus dar aren't 
 about fixing the library or runtime issues that prevent D's 
 adoption at large as a systems programming language.
Without listing what you think those real issues are, such a pronouncement is useless.
 Meanwhile Swift, Go and Rust have a clear roadmap how their 
 future is supposed to look like, and drive just in that 
 direction, with C++ taking all remaining good D ideas.
Heading in the wrong direction at full speed ahead is worse than meandering along in no discernible direction, because worst case, you will likely be closer to the right direction after simply meandering. In any case, you clearly think D headed in some good directions, or C++ would have nothing to copy. ;) As for the other roadmaps, I see it for Rust and Swift https://github.com/aturon/rfcs/blob/roadmap-2017/text/0000-roadmap-2017.md https://github.com/apple/swift-evolution but nothing for Go. Do you mean these milestone lists with hundreds of PRs attached? https://github.com/golang/go/milestones Or is it somewhere else that google can't easily find it? You could read the changelog for the dmd/phobos master branches and get the same result as that Go list. Those languages are all backed by large tech companies and are driven by their company agendas. As I've pointed out in this forum before, with some quotes from Linus (http://forum.dlang.org/post/dluoruxmwxnfjtyvmgbh forum.dlang.org), directed development is good for niches, which is where all those languages are now (iOS, network services, Servo), but only general-purpose tech lasts in the medium- to long-term. D may never gain the resources it needs to make it that long, but a community employing D in _no one direction_, as Linus says happened with linux, is more likely to make it general-purpose enough to survive.
 This DIP discussion and the latest ones about splitting the 
 runtime again, don't do anything to earn D any credibility it 
 might still have left.
I kind of agree that both are not that worthwhile, but without stating your reasons for thinking the discussion _alone_ hurts D's credibility, which I mostly disagree with as such discussion is a core part of the distributed OSS process, that opinion is again useless.
Jan 04
prev sibling next sibling parent reply Daniel N <no public.email> writes:
On Monday, 2 January 2017 at 21:23:19 UTC, Andrei Alexandrescu 
wrote:
 No worries, I'll make a pass (a rewrite, really) taking all 
 feedback into account.
Are all the proposals mutually exclusive? When it comes to declarations, the DIP actually won me over! However I was hoping for it to solve a different issue inside the body with local imports. I tend to write short helper functions, with the result that in the common case, an imported symbol is often used only once per function. Currently I consider it good style to list all imports in the beginning of a scope, but based on past experience with C89, this is not entirely optimal, however Timon Gehr's proposal could solve it. import and directly invoke: import std.traits : isArray(...) ======================================================= [Proposed Style - analogous to C99] body { static if(...) import std.range : zip(...); static if(import std.traits : isArray(...))) import std.range : join(...); } ======================================================= [Old Style - analogous to C89 - Suffers from "unused" imports] body { import std.range : zip, join; import std.traits : isArray; static if(...) zip(...) static if(isArray(...))) join(...) } ======================================================= [Alternate Style - Suffers from DRY and can't limit scope of isArray] body { import std.traits : isArray; static if(...) { import std.range : zip; zip(...); } static if(is(isArray(...))) { import std.range : join; join(...); } } C89 vs C99 style I was referring to if it wasn't clear. ======================================================= [C89] void foo(void) { #ifdef ... /* needed to silence unused variable warnings */ int x; #endif ... lots of code ... #ifdef ... x = bar(); #endif } [C99] void foo(void) { #ifdef ... int x = bar(); #endif } =======================================================
Jan 03
parent Daniel N <no public.email> writes:
On Tuesday, 3 January 2017 at 10:18:57 UTC, Daniel N wrote:
 static if(is(isArray(...)))
 {
   import std.range : join;
   join(...);
 }
PS Actually double {} is needed for the 'alternate' style to be meaningful, preventing the import from bleeding into the outer scope. static if(isArray(...)) {{ import std.range : join; join(...); }}
Jan 03
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
There are quite a few fallacies in there.

On Monday, 2 January 2017 at 21:23:19 UTC, Andrei Alexandrescu 
wrote:
 Regarding the ongoing doubts about the advantages of inline 
 imports: they are first and foremost a completion of the nested 
 import feature. As such, most, if not all, arguments against 
 inline imports apply equally to nested imports. Come to think 
 of it, lazy imports vs nested imports:

 * same improvement in compilation speed? check
 * no language changes? check
 * no nasty bugs in the aftermath (such as the infamous 
 https://issues.dlang.org/show_bug.cgi?id=10378)? check
 * scalable builds? check

 Yet local imports are overwhelmingly superior to lazy imports 
 because of one thing: they localize dependencies. They 
 introduce modularity and its ancillary perks (fast and scalable 
 builds, easier review and refactoring) not by engineering, but 
 by organically placing dependencies spatially with their 
 dependents. (The scope statement does the same thing with 
 temporal dependencies.) That the DIP does not make it clear 
 that it is a necessary and sufficient extension of local 
 imports is a problem with it.
There is a major difference with this DIP. Lazy import is not a language change, but a compiler implementation detail. As such, it doesn't require a DIP or anything specific. Nested import are a language simplification. Declaration can appear anywhere, import is a declaration, the fact that import couldn't appear anywhere was an arbitrary limitation, and removing it makes the language simpler. As such, the burden of proof is on maintaining the limitation rather than removing it. This DIP is a language addition. Therefore, contrary to nested or lazy import, the burden of proof is on it. This DIP should be considered as follow: how much complexity does it add and how much benefit does it bring, compared to alternatives. The obvious benefit is localizing dependencies. I think I'm not too far off track by considering most of the speedup and scalable build can be achieved with lazy import and, while I'm sure there are example where this is superior, we are talking marginal gains as lazy and nested imports squeezed most of the juice already. The cost is the language addition. The first obvious improvement that can be made to this DIP to reduce its cost is to not introduce a new syntax. As such, the addition is limited to allowing the existing syntax in a new place rather than adding a whole new syntax for imports. I like the extra expressivity. I'm not 100% convinced it is worth the extra cost, but the more the cost is reduced, the more rational it seems to me that this option should be pursued.
 I now am really glad we slipped local imports before the 
 formalization of DIPs. The feature could have been easily 
 demeaned out of existence.
Good you also notice how broken the DIP process is. One suggestion: let's keep the DIP describing the change to be made. Some examples are fine to illustrate, but it is not the DIp's purpose to be easy to understand or expand too much in argumentation, or it'll be useless as a spec document, and trying to have the DIP be a spec, a tutorial, a essay on why the feature, and so on just lead to endless rewriting lead to perpetual motion but no progress.
Jan 04
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 04.01.2017 16:00, deadalnix wrote:
 Nested import are a language simplification. Declaration can appear
 anywhere, import is a declaration, the fact that import couldn't appear
 anywhere was an arbitrary limitation, and removing it makes the language
 simpler. As such, the burden of proof is on maintaining the limitation
 rather than removing it.
 ...
I don't fully agree. Nested imports, the way they have been implemented, pose a new symbol hijacking hazard. (A symbol hijacking anecdote (not directly related): https://github.com/tgehr/d-compiler/pull/1#discussion-diff-89697186L85 )
 This DIP is a language addition. Therefore, contrary to nested or lazy
 import, the burden of proof is on it. This DIP should be considered as
 follow: how much complexity does it add and how much benefit does it
 bring, compared to alternatives.
It adds basically no implementation complexity [1]. I consider the benefit real, but minor enough to oppose the DIP based on its wacky syntax. [1] Both static if and static foreach (once it lands) need the same kind of scoping rules.
Jan 04
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 4 January 2017 at 15:56:13 UTC, Timon Gehr wrote:

 [1] Both static if and static foreach (once it lands) need the 
 same kind of scoping rules.
Please do contact me if you are working on static foreach, there are dmd and implementation specific issues to be taken into account.
Jan 04
next sibling parent deadalnix <deadalnix gmail.com> writes:
On Wednesday, 4 January 2017 at 16:03:29 UTC, Stefan Koch wrote:
 On Wednesday, 4 January 2017 at 15:56:13 UTC, Timon Gehr wrote:

 [1] Both static if and static foreach (once it lands) need the 
 same kind of scoping rules.
Please do contact me if you are working on static foreach, there are dmd and implementation specific issues to be taken into account.
I think the best path forward is to define them properly.
Jan 04
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 04.01.2017 17:03, Stefan Koch wrote:
 On Wednesday, 4 January 2017 at 15:56:13 UTC, Timon Gehr wrote:

 [1] Both static if and static foreach (once it lands) need the same
 kind of scoping rules.
Please do contact me if you are working on static foreach,
Not currently. I might implement it in https://github.com/tgehr/d-compiler soon though. I'll let you know.
 there are dmd and implementation specific issues to be taken into account.
What kind of issues? The compiler implementation shouldn't matter for the definition of the static foreach behaviour. Generation of the sequence of values assigned to the loop variables should be exactly like runtime foreach, scoping of the loop variables should be local. For scoping of the declarations within the static foreach body, there are multiple possibilities. (The most obvious thing is to just insert them into the outer scope, but then we need some other improvements in order to avoid having to string mixin each and every declaration in the body.) There are some questions about which dependency structures should be allowed for the loop aggregate and the declarations in the loop body, but I think this is mostly orthogonal to static foreach itself.
Jan 04
prev sibling parent deadalnix <deadalnix gmail.com> writes:
On Wednesday, 4 January 2017 at 15:56:13 UTC, Timon Gehr wrote:
 I don't fully agree. Nested imports, the way they have been 
 implemented, pose a new symbol hijacking hazard.
I'd argue this was an existing bug in import handling. This is why I like to have very orthogonal definitions.
 It adds basically no implementation complexity [1]. I consider 
 the benefit real, but minor enough to oppose the DIP based on 
 its wacky syntax.

 [1] Both static if and static foreach (once it lands) need the 
 same kind of scoping rules.
I know about [1], this is why I did not mentioned it. I don't really mind about implementation complexity, I care about complexity of the definition. For the following reasons: - If the implementation may be complex, it can be isolated and/or abstracted away. - Interaction with other parts of the language are more predictable, including future parts that do not exists yet. - It obviate the explosion of trivia experienced devs needs to know to use the language.
Jan 04
prev sibling parent reply Chris Wright <dhasenan gmail.com> writes:
On Sat, 24 Dec 2016 04:34:03 -0500, Andrei Alexandrescu wrote:
 Upon more investigation, I see a large discrepancy between the findings
 of DIP1005 and yours.
There's no discrepancy. In part, you are misinterpreting most of what I said. In part, you are assuming that imports on non-template declarations will be handled lazily, even though that is not part of this DIP, even though that is likewise possible with static and selective imports. In part, you are using lines of code as a proxy for compile time. In part, you dispute that this only affects template constraints, but: * An import used only in the body of a template can be made a local import today. * An import used in the declaration of a templated type or function can be addressed by using explicit template syntax, offering a place to insert your imports. * An import used anywhere else must still be processed, even assuming this DIP is implemented. * If, in a future DIP, we make it so that `with(import)` is handled lazily, we can also make it so that static and selective imports are handled lazily.
 The findings of DIP1005 are the following:
 
 * Importing a single std module also imports on average 10.5 other
 modules.
Seems reasonable. Between 2 and 3.5 direct dependencies, by my count, and you're counting transitive dependencies. We're concerned with the effects of DIP1005, though, which only affects template constraints.
 * Importing a single std module costs on average 64.6 ms.
55-ish for your hardware, you reported elsewhere. 47-ish for mine.
 * (Not stated in the DIP) A majority of std templates would acquire
 inline imports.
Again, that wouldn't impact compile times because these aren't template constraints. You can make a separate DIP to make imports lazy. That can impact static, selective, and `with` imports equally well. But it's not part of what we're discussing today.
 According to the DIP, one may estimate that the proposed feature would
 reduce additional imports to 0 and the average time to import a single
 module by a factor of 10 to under 10 ms.
"The proposed feature" must be lazy semantic analysis, especially of imports. That isn't part of DIP1005. You won't get to zero additional imports. You might get to zero *extraneous* imports -- that is, only the set of imports required to create a custom *.di file containing only the parts of the module that your application uses.
 By your estimates:
 
 * 26 templates in std need inline imports.
I said that 26 templates *could possibly benefit from* your new style of imports. There's a difference between possibly benefitting from a change and needing that change.
 * Importing a single std module today would only imports 1-3 other
 modules most of the time (one or more of std.traits, std.meta, and
 std.range.primitives).
No, that's not what I said at all. I said that the only modules you would sometimes *stop* processing because of DIP1005 are std.traits, std.meta, and std.range.primitives. That's because those modules contain templates used in other modules as template constraints. In order to get any additional improvements, you need lazy imports, which can also apply to static or selective imports without any syntax changes.
 * These additional imports cost in aggregate under 10ms, bringing the
 average cost of importing a module itself to 54.6 ms.
~10ms is the upper bound of the added cost if you import just one module in std that has a template constraint you don't use. The way you state it implies that every module brings in std.traits, std.meta, and std.range.primitives unnecessarily, instead of 26 templates across at most 26 modules importing them for a reason.
 * It follows that the average module takes 5.46 more times to import
 alone than the sum of std.traits, std.meta, and std.range.primitives
 (which have a total of 11263 lines, 5x more than the average Phobos
 module).
More like 4.7 on my hardware, but yeah. 11k lines that have to be parsed and 0 lines that require semantic analysis. Not terribly surprising.
 I don't see how your claims can be simultaneously true with the findings
 of DIP1005.
You found that the average cost of importing a std module is 54ms or thereabouts. std.traits, std.meta, and std.range.primitives are well below average. No conflict there. They aren't even the cheapest modules in the standard library. The modules in question are mostly unittests. The compiler doesn't run semantic on unittests in a module that wasn't included in the command line. (Even if you pass -unittest. Try it out -- you can even have a unittest that says `static assert(false);` and it does nothing.) The parts of the modules that are not unittests are templates. The compiler doesn't run semantic analysis on templates until you use them. So it should be pretty obvious why these modules are so cheap to import and not use.
 The scripts that compute those numbers are available with
 the DIP. Were you able to reproduce them?
The times it reported on my hardware: Min: 5ms Max: 300ms Median: 21ms Average: 47ms The minimum isn't terribly useful because it gets to the point of testing the process scheduler and IO more than the compiler. If we want numbers that we can trust on the low end, we'll need to put timing information into the compiler, maybe control for IO by using a ramfs, that sort of thing. You also reproduced my test, so this isn't a quirk of my installation.
Dec 24 2016
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 24 December 2016 at 17:52:04 UTC, Chris Wright wrote:
 The minimum isn't terribly useful because it gets to the point 
 of testing the process scheduler and IO more than the compiler. 
 If we want numbers that we can trust on the low end, we'll need 
 to put timing information into the compiler, maybe control for 
 IO by using a ramfs, that sort of thing.
As a good approximation you can use a profile build of dmd. It will give you a breakdown of how long the individual functions took. Due to the way the profiling works, it does homogenize the values a bit. Files are not streamed into dmd, it reads them as one Block. Therefore there is no need for a ramfs.
Dec 24 2016
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/24/2016 12:52 PM, Chris Wright wrote:
 On Sat, 24 Dec 2016 04:34:03 -0500, Andrei Alexandrescu wrote:
 Upon more investigation, I see a large discrepancy between the findings
 of DIP1005 and yours.
There's no discrepancy. In part, you are misinterpreting most of what I said.
Yah, I suspected so. * Your post provided no description of the methodology used beyond "relatively simple regex" to come up with the magic number 26, so I tried to use interpretation. * There is no answer to "where is cmp on that list for example?" to shed light on what 26 is about. * I have no access to your method of measurement, the scripts you used, or really what the claims are. So I really don't have a good understanding of what hypothesis you have and how you arrived to it. DIP1005 carefully describes methodology and publishes measurements and results (which I'd be in your debt if you used - not those "published elsewhere"). Then it interprets the results.
 In part, you are assuming that imports on non-template declarations will
 be handled lazily, even though that is not part of this DIP, even though
 that is likewise possible with static and selective imports.
Imports in non-template declarations ARE handled lazily when those non-templates are imported. Consider: module test; void fun() { import std.stdio; writeln("hello"); } module test2; import test; void main() {} // let's not call fun dmd -v test2.d ==== binary dmd version v2.073.0-devel-bd0dec2 config /home/andrei/bin/dmd.conf parse test2 importall test2 import object (/home/andrei/d/druntime/import/object.d) import test (test.d) semantic test2 entry main test2.d semantic2 test2 semantic3 test2 code test2 function D main cc test2.o -o test2 -m64 -L/home/andrei/d/phobos/generated/linux/release/64 -Xlinker -Bstatic -lphobos2 -Xlinker -Bdynamic -lpthread -lm -lrt -ldl ==== Destroyed? This might be another misinterpretation of what you said though.
 In part, you are using lines of code as a proxy for compile time.
What do you suggest to use?
 In part, you dispute that this only affects template constraints, but:
 * An import used only in the body of a template can be made a local
 import today.
Correct. To the best of my knowledge, that fruit has been picked in Phobos.
 * An import used in the declaration of a templated type or function can
 be addressed by using explicit template syntax, offering a place to
 insert your imports.
Good thought. Would be a bit overkill though.
 * An import used anywhere else must still be processed, even assuming
 this DIP is implemented.
How do you mean that? On the face of it the sentence is true. Are you saying that you need to remove top-level imports to benefit from the DIP? That would indeed be the case.
 * If, in a future DIP, we make it so that `with(import)` is handled
 lazily, we can also make it so that static and selective imports are
 handled lazily.
That would be nice, but DIP1005 provides advantages beyond latency. BTW it would be great if the discussion focused on (a) things that are true but the DIP doesn't mention (or misrepresents), or (b) things that are false that the DIP claims. I clearly concede that the DIP cannot convince someone with enough cognitive bias without an implementation and a few years of experience (and even then - aren't there folks out there who believe things like constraints, static if, or ranges are failures?).
 The findings of DIP1005 are the following:

 * Importing a single std module also imports on average 10.5 other
 modules.
Seems reasonable. Between 2 and 3.5 direct dependencies, by my count, and you're counting transitive dependencies.
Shouldn't I? Shouldn't you?
 We're concerned with the effects of DIP1005, though, which only affects
 template constraints.

 * Importing a single std module costs on average 64.6 ms.
55-ish for your hardware, you reported elsewhere. 47-ish for mine.
Let's stick with the numbers published in the DIP.
 * (Not stated in the DIP) A majority of std templates would acquire
 inline imports.
Again, that wouldn't impact compile times because these aren't template constraints.
I don't understand this. YES it would impact compile times!
 You can make a separate DIP to make imports lazy. That can impact static,
 selective, and `with` imports equally well. But it's not part of what
 we're discussing today.

 According to the DIP, one may estimate that the proposed feature would
 reduce additional imports to 0 and the average time to import a single
 module by a factor of 10 to under 10 ms.
"The proposed feature" must be lazy semantic analysis, especially of imports. That isn't part of DIP1005.
The proposed feature is DIP1005. I really don't understand how your discourse goes.
 You won't get to zero additional imports. You might get to zero
 *extraneous* imports -- that is, only the set of imports required to
 create a custom *.di file containing only the parts of the module that
 your application uses.
You are wrong. With no top-level imports and all imports hoisted into inline imports, the fixed cost of importing one module will be that module alone. To focus the discussion, could we please focus on this one point? Andrei
Dec 24 2016
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/24/2016 02:03 PM, Andrei Alexandrescu wrote:
 With no top-level imports and all imports hoisted into inline imports,
 the fixed cost of importing one module will be that module alone.
I'd really like to clarify this because that's where scalability comes from. The moment you need to open an unbounded amount of files for one import, it's gone. If I missed something and more than one module needs to be looked at I need to rethink the whole DIP. -- Andrei
Dec 24 2016
prev sibling parent Chris Wright <dhasenan gmail.com> writes:
On Sat, 24 Dec 2016 14:03:26 -0500, Andrei Alexandrescu wrote:
 * Your post provided no description of the methodology used beyond
 "relatively simple regex" to come up with the magic number 26, so I
 tried to use interpretation.
 
 * There is no answer to "where is cmp on that list for example?" to shed
 light on what 26 is about.
You mentioned that in another branch of the conversation. It took some time for me to get to it. I'm a human.
 In part, you are assuming that imports on non-template declarations
 will be handled lazily, even though that is not part of this DIP, even
 though that is likewise possible with static and selective imports.
Imports in non-template declarations ARE handled lazily when those non-templates are imported. Consider:
... Y'know, I make assumptions, but I tend to test them. But that kind of depends on me writing valid tests, and in this case, I didn't. My face is egged, and I concede. Thank you for providing example code. That cleared everything up very fast. With this new understanding, most modules become nearly free to import under your proposal. It does require a lot of work to get there, but on the plus side, it should be possible to automate that with a tool. I'd be willing to add that to my build process for anything I publish if it'll help others.
 In part, you are using lines of code as a proxy for compile time.
What do you suggest to use?
You did write a script to check compile times for importing individual phobos modules.
 The findings of DIP1005 are the following:

 * Importing a single std module also imports on average 10.5 other
 modules.
Seems reasonable. Between 2 and 3.5 direct dependencies, by my count, and you're counting transitive dependencies.
Shouldn't I? Shouldn't you?
I was agreeing with you. At the time, I didn't have a convenient way to check the number of transitive dependencies of a module.
 We're concerned with the effects of DIP1005, though, which only affects
 template constraints.

 * Importing a single std module costs on average 64.6 ms.
55-ish for your hardware, you reported elsewhere. 47-ish for mine.
Let's stick with the numbers published in the DIP.
Sure. You insinuated that I should run your script to verify your numbers, so I did.
Dec 24 2016
prev sibling next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Tuesday, 13 December 2016 at 22:33:24 UTC, Andrei Alexandrescu 
wrote:
 Destroy.

 https://github.com/dlang/DIPs/pull/51/files
FYI, from the DIP, this is false: "The static import setup does not share the issue above, at the cost of being cumbersome to use---all imported symbols must use full lookup everywhere. A reasonable engineering approach would be to define shorter names:" It then proceeds to list a bunch of local aliases. That's not how I'd do it because you can also simply use a renamed import: static import r = std.range.primitives; void foo(R)(R range) if(r.isInputRange!(range)) // works today If the import were lazy - which there's no reason not to be with a static import, renamed or not, since its use is always explicit - then this gives most the same advantages of the DIP. Granted, you'd still have to look up what `r` means, so it isn't 100% dependency carrying. Another current language option not discussed in the document is an eponymous template. But there is another language option today: an eponymous template. Consider the following: with (import std.stdio) void process(File input) ; Let's rewrite that to be: template process() { import std.stdio; void process(File input) { } } The whole symbol is now lazy and encapsulated... and thanks to the existing eponymous rules, it'd be semi-transparent to the user (just error messages would be the main source of leakage, oh god the error messages could be so ugly). But it works in today's D. BTW it is my opinion that the "one file, one module" rule is a mistake. Even with all these things, the declarations are still not *guaranteed* to carry their dependencies, since top-level imports still leak in. Separate modules don't have that problem, and if we could just define several modules in one file, we'd basically destroy this DIP in one swift stroke. I doubt y'all would change that, but still, I'd list that as an alternative in the section that lists many small files as being a major disadvantage, since while we are discussing changing the language, that's a possible change too unless explicitly off the table.
Dec 31 2016
next sibling parent Chris Wright <dhasenan gmail.com> writes:
On Sat, 31 Dec 2016 15:05:00 +0000, Adam D. Ruppe wrote:
 static import r = std.range.primitives;
For that to pass code review, you'd need a readable name -- perhaps 'ranges' instead of 'r' -- which makes it more obvious, more searchable, and more typing.
 with (import std.stdio) void process(File input) ;
 
 Let's rewrite that to be:
 
 template process() {
     import std.stdio;
     void process(File input) {
 
     }
 }
But that's a lot of typing, so nobody's going to do it. Just like how we use normal imports rather than static or selective imports everywhere. It's even more verbose than DIP1005.
 BTW it is my opinion that the "one file, one module" rule is a mistake.
 Even with all these things, the declarations are still not *guaranteed*
 to carry their dependencies, since top-level imports still leak in.
 Separate modules don't have that problem, and if we could just define
 several modules in one file, we'd basically destroy this DIP in one
 swift stroke.
That's rather elegant, though I don't see how you'd import a module in a file that defines several modules. If they were anonymous and importing the file gave you access to all of them, that makes sense. Otherwise it's tricky to figure out which sub-file modules exist.
Dec 31 2016
prev sibling parent Meta <jared771 gmail.com> writes:
On Saturday, 31 December 2016 at 15:05:00 UTC, Adam D. Ruppe 
wrote:
 Consider the following:

 with (import std.stdio) void process(File input) ;

 Let's rewrite that to be:

 template process() {
    import std.stdio;
    void process(File input) {

    }
 }
I absolutely hate making functions templated if they don't have to be. It breaks a whole bunch of things and is not at all transparent to the user. You can't take its address, pass it as a delegate/function pointer, use certain traits to introspect it (e.g. isSomeFunction, ReturnType, etc.), not to mention how templates really don't work well with inheritance. This all goes out the window when you turn a regular function into a template.
Dec 31 2016
prev sibling parent reply Chris Wright <dhasenan gmail.com> writes:
On Tue, 13 Dec 2016 17:33:24 -0500, Andrei Alexandrescu wrote:

 Destroy.
 
 https://github.com/dlang/DIPs/pull/51/files
 
 
 Andrei
 Inside any function, with (Import ImportList) is a statement that
 introduces a scope.
So I can't write: with (import std.stdio, std.conv) int count = readln().to!int; assert(count > 0); It is in fact entirely equivalent to write: { import std.stdio, std.conv; int count = readln().to!int; } I think I'd just put a line in the DIP: for consistency, this syntax works anywhere a with statement is currently allowed, but it's not recommended to use it inside function bodies in general.
 This extension removes an unforced limitation of the current with
 syntax (allows it to occur at top level)
In other words, another aspect of this DIP is that I can write: module foo; static import std.traits; static import bar; with (std.traits) { template Foo(T) if (isAbstractClass!T) {} } with (bar.SomeEnum) { enum something = SomeEnumValue; } Which *almost*, but not quite, obviates the "you can put the import list in the with clause" part of the DIP. It's got all the same benefits for reading, but you might occasionally have to jump to the top of the file to add a new static import.
Dec 31 2016
parent reply pineapple <meapineapple gmail.com> writes:
On Saturday, 31 December 2016 at 17:02:55 UTC, Chris Wright wrote:
 This extension removes an unforced limitation of the current 
 with syntax (allows it to occur at top level)
In other words, another aspect of this DIP is that I can write: module foo; static import std.traits; static import bar; with (std.traits) { template Foo(T) if (isAbstractClass!T) {} } with (bar.SomeEnum) { enum something = SomeEnumValue; }
This is the only expression of the feature I've seen so far that makes intuitive sense to me. I'm still not sold on it being a worthy addition but, if it were, then this is the most promising syntax I've seen so far.
Jan 01
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 02.01.2017 01:47, pineapple wrote:
 On Saturday, 31 December 2016 at 17:02:55 UTC, Chris Wright wrote:
 This extension removes an unforced limitation of the current with
 syntax (allows it to occur at top level)
In other words, another aspect of this DIP is that I can write: module foo; static import std.traits; static import bar; with (std.traits) { template Foo(T) if (isAbstractClass!T) {} } with (bar.SomeEnum) { enum something = SomeEnumValue; }
This is the only expression of the feature I've seen so far that makes intuitive sense to me. I'm still not sold on it being a worthy addition but, if it were, then this is the most promising syntax I've seen so far.
Should be 'static with'. ('with' already has a meaning.)
Jan 01
prev sibling parent Chris Wright <dhasenan gmail.com> writes:
On Mon, 02 Jan 2017 00:47:41 +0000, pineapple wrote:

 On Saturday, 31 December 2016 at 17:02:55 UTC, Chris Wright wrote:
 This extension removes an unforced limitation of the current with
 syntax (allows it to occur at top level)
In other words, another aspect of this DIP is that I can write: module foo; static import std.traits; static import bar; with (std.traits) { template Foo(T) if (isAbstractClass!T) {} } with (bar.SomeEnum) { enum something = SomeEnumValue; }
This is the only expression of the feature I've seen so far that makes intuitive sense to me. I'm still not sold on it being a worthy addition but, if it were, then this is the most promising syntax I've seen so far.
This is a minor part of the proposal that's mentioned as an aside, though, and I'm not even sure it was even intentional. This alone doesn't get the two benefits Andrei is most after -- specifically, that you can add the import in place without going to the top of the file and that you can move the declaration to another module without worrying about imports.
Jan 01