www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Spurious imports in Phobos ?

reply Somedude <lovelydear mailmetrash.com> writes:
Hello,

When I display the dependencies with "dmd -deps=depends", I see that
simply importing std.stdio imports dozens of modules, among which
std.ranges, std.datetime, std.c.windows.winsock, std.regex, etc
In fact, the depends file is 433 lines long.

I noticed that std.string imports quite a lot of stuff.
Here is a copypasta of the code:

import core.exception : onRangeError;
import core.vararg, core.stdc.stdlib, core.stdc.string,
    std.algorithm, std.ascii, std.conv, std.exception, std.format,
std.functional,
    std.metastrings, std.range, std.regex, std.traits,
    std.typetuple, std.uni, std.utf;

//Remove when repeat is finally removed. They're only here as part of the
//deprecation of these functions in std.string.
public import std.algorithm : startsWith, endsWith, cmp, count;
public import std.array : join, split;

So as an experiment, I tried to remove a few imports:

std.range, std.regex, std.traits and std.algorithm
as well as the two lines of public import.

To my surprise, it still compiles !
The depends file goes down to 402 lines, but the executable size is
understandably not reduced because all the removed dependencies are
still used elsewhere.

Then I thought this may be due to public imports from other modules,
which makes me think that public imports are a bad idea, as we can
compile a module only because we imported unknowingly a namespace from
imported modules.

My question is: how do we know if std.range, std.regex, std.traits and
std.algorithm are spurious imports or if we can (and threfore should)
remove them safely from std.string ?

Dude
Nov 09 2011
next sibling parent Somedude <lovelydear mailmetrash.com> writes:
Le 09/11/2011 10:14, Somedude a écrit :
 
 My question is: how do we know if std.range, std.regex, std.traits and
 std.algorithm are spurious imports or if we can (and threfore should)
 remove them safely from std.string ?
 
 Dude

I meant: how do we know if std.range, std.regex, std.traits and
 std.algorithm are necessary imports or if we can (and threfore should)
 remove them safely from std.string ?

Nov 09 2011
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2011-11-09 10:14, Somedude wrote:
 Hello,

 When I display the dependencies with "dmd -deps=depends", I see that
 simply importing std.stdio imports dozens of modules, among which
 std.ranges, std.datetime, std.c.windows.winsock, std.regex, etc
 In fact, the depends file is 433 lines long.

 I noticed that std.string imports quite a lot of stuff.
 Here is a copypasta of the code:

 import core.exception : onRangeError;
 import core.vararg, core.stdc.stdlib, core.stdc.string,
      std.algorithm, std.ascii, std.conv, std.exception, std.format,
 std.functional,
      std.metastrings, std.range, std.regex, std.traits,
      std.typetuple, std.uni, std.utf;

 //Remove when repeat is finally removed. They're only here as part of the
 //deprecation of these functions in std.string.
 public import std.algorithm : startsWith, endsWith, cmp, count;
 public import std.array : join, split;

 So as an experiment, I tried to remove a few imports:

 std.range, std.regex, std.traits and std.algorithm
 as well as the two lines of public import.

 To my surprise, it still compiles !
 The depends file goes down to 402 lines, but the executable size is
 understandably not reduced because all the removed dependencies are
 still used elsewhere.

 Then I thought this may be due to public imports from other modules,
 which makes me think that public imports are a bad idea, as we can
 compile a module only because we imported unknowingly a namespace from
 imported modules.

 My question is: how do we know if std.range, std.regex, std.traits and
 std.algorithm are spurious imports or if we can (and threfore should)
 remove them safely from std.string ?

 Dude

Phobos contains a lot of templates and if a template isn't instantiated it won't be compiled. Meaning there can be hidden compile errors if you start to remove imports and they will not show until a template that uses something from the import is instantiate. -- /Jacob Carlborg
Nov 09 2011
next sibling parent reply Somedude <lovelydear mailmetrash.com> writes:
Le 09/11/2011 13:15, Jacob Carlborg a écrit :
 Phobos contains a lot of templates and if a template isn't instantiated
 it won't be compiled. Meaning there can be hidden compile errors if you
 start to remove imports and they will not show until a template that
 uses something from the import is instantiate.

I see, but then 1. is there a systematic procedure to know if a an import is really needed ? 2. what is your opinion about public import ? In C++, "hidden" or "implicit" #includes is a common source of compilation problems (order of #includes), I tend to think it's a bad thing.
Nov 09 2011
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2011-11-09 13:45, Somedude wrote:
 Le 09/11/2011 13:15, Jacob Carlborg a écrit :
 Phobos contains a lot of templates and if a template isn't instantiated
 it won't be compiled. Meaning there can be hidden compile errors if you
 start to remove imports and they will not show until a template that
 uses something from the import is instantiate.

I see, but then 1. is there a systematic procedure to know if a an import is really needed ?

Not that I know of. You can make sure that every template is instantiated at least once.
 2. what is your opinion about public import ? In C++, "hidden" or
 "implicit" #includes is a common source of compilation problems (order
 of #includes), I tend to think it's a bad thing.

Sometimes public imports are useful. It's possible to emulate Java's import foo.* using public imports: // a._.d public import a.foo; public import a.bar; // foobar.d import a._; It can also be useful to have public imports if you have a module with array functions and a module with string functions. Then the string module can publicly import the array module since all(most) array functions will work with strings as well. -- /Jacob Carlborg
Nov 09 2011
parent Somedude <lovelydear mailmetrash.com> writes:
Le 09/11/2011 14:50, Jacob Carlborg a écrit :

 2. what is your opinion about public import ? In C++, "hidden" or
 "implicit" #includes is a common source of compilation problems (order
 of #includes), I tend to think it's a bad thing.

Sometimes public imports are useful. It's possible to emulate Java's import foo.* using public imports: // a._.d public import a.foo; public import a.bar; // foobar.d import a._; It can also be useful to have public imports if you have a module with array functions and a module with string functions. Then the string module can publicly import the array module since all(most) array functions will work with strings as well.

As I said, this is considered sloppy programming, and both in Java and in Python ("from xxx import *"), this practice is highly discouraged. This is because you bind modules together more than necessary. If you need in module A name B.b and deprecate B.b later, then there is no reason to have imported everything from B. In Java, the IDE does the work of importing the exact packages/classes needed for you, but in Python, you have to do it by hand. It seems that it would be just as bad in D as in Python since compilation errors don't appear until templates are instantiated. Dude
Nov 10 2011
prev sibling parent Somedude <lovelydear mailmetrash.com> writes:
Le 09/11/2011 14:15, Trass3r a écrit :
 2. what is your opinion about public import ? In C++, "hidden" or
 "implicit" #includes is a common source of compilation problems (order
 of #includes), I tend to think it's a bad thing.

It can be quite useful. I use it often for C library wrappers. As soon as you import the wrapper code you automatically import the bindings to be able to use constants etc.

I agree it's useful in this special case, but in the general case, I think it encourages sloppy programming: it binds modules together more than necessary, and it seems that once public import is used, it can be very hard to remove it afterward.
Nov 09 2011
prev sibling next sibling parent Trass3r <un known.com> writes:
 2. what is your opinion about public import ? In C++, "hidden" or  
 "implicit" #includes is a common source of compilation problems (order  
 of #includes), I tend to think it's a bad thing.

It can be quite useful. I use it often for C library wrappers. As soon as you import the wrapper code you automatically import the bindings to be able to use constants etc.
Nov 09 2011
prev sibling parent reply Trass3r <un known.com> writes:
 Phobos contains a lot of templates and if a template isn't instantiated  
 it won't be compiled. Meaning there can be hidden compile errors if you  
 start to remove imports and they will not show until a template that  
 uses something from the import is instantiate.

Wouldn't it be possible/better then to move the imports into those template functions?
Nov 09 2011
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2011-11-09 14:16, Trass3r wrote:
 Phobos contains a lot of templates and if a template isn't
 instantiated it won't be compiled. Meaning there can be hidden compile
 errors if you start to remove imports and they will not show until a
 template that uses something from the import is instantiate.

Wouldn't it be possible/better then to move the imports into those template functions?

I probably would. Having imports in non-global scope is quite a new feature and I guess nobody has either thought of the idea or just haven't had the time yet. -- /Jacob Carlborg
Nov 09 2011
parent Jacob Carlborg <doob me.com> writes:
On 2011-11-09 19:14, Jonathan M Davis wrote:
 On Wednesday, November 09, 2011 05:42 Jacob Carlborg wrote:
 On 2011-11-09 14:16, Trass3r wrote:
 Phobos contains a lot of templates and if a template isn't
 instantiated it won't be compiled. Meaning there can be hidden compile
 errors if you start to remove imports and they will not show until a
 template that uses something from the import is instantiate.

Wouldn't it be possible/better then to move the imports into those template functions?

I probably would. Having imports in non-global scope is quite a new feature and I guess nobody has either thought of the idea or just haven't had the time yet.

You also get the question of what to do if a lot of templated functions use the same import. Sure, you could put the import in each individual function so that it doesn't get imported if you don't use any of those functions, but then you've duplicated the same import in a bunch of places. It's not an entirely clear issue.

Didn't think of that.
 And I don't think that it ever affects the size of the executable. If you're
 dealing with static libraries, then all of the functions that aren't used
 shouldn't be compiled in. And if you're using a shared library, everything in
 the library is going to need to be there anyway (which would just affect the
 size of the library, not the executably anyway). So, I don't see how
 unnecessary imports would be a problem with regards to executable size.

 The bigger problem is static constructors and circular dependencies. If the
 wrong modules import one another when they don't need to, it can result in
 circular dependencies which will then cause the runtime to exit with an error
 when you try and run your program. So that - and the fact that it's just
 cleaner - is a good reason to not have unnecessary imports in a module.
 However, the circular dependency situation can be made _worse_ by putting
 imports within a template, since not only is there no way to detect the
 circular dependency at compile time, but whether you _have_ a circular
 dependency can depend on whether a particular template was instantiated or
 not. So, this is definitely not a clear-cut issue.

That would be bad.
 I'd definitely say that if there are only one or two functions in a module
 which need a particular import, then it's probably better to restrict that
 import to those functions, but beyond that, it's better to just stick them at
 the module level. But since (as you point out) local imports are quite new, I
 really don't think that much of Phobos is taking advantage of them even if it
 should be. That'll probably change over time, but it's not exactly a high
 priority.

 - Jonathan M Davis

Yes, exactly. -- /Jacob Carlborg
Nov 09 2011
prev sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, November 09, 2011 05:42 Jacob Carlborg wrote:
 On 2011-11-09 14:16, Trass3r wrote:
 Phobos contains a lot of templates and if a template isn't
 instantiated it won't be compiled. Meaning there can be hidden compile
 errors if you start to remove imports and they will not show until a
 template that uses something from the import is instantiate.

Wouldn't it be possible/better then to move the imports into those template functions?

I probably would. Having imports in non-global scope is quite a new feature and I guess nobody has either thought of the idea or just haven't had the time yet.

You also get the question of what to do if a lot of templated functions use the same import. Sure, you could put the import in each individual function so that it doesn't get imported if you don't use any of those functions, but then you've duplicated the same import in a bunch of places. It's not an entirely clear issue. And I don't think that it ever affects the size of the executable. If you're dealing with static libraries, then all of the functions that aren't used shouldn't be compiled in. And if you're using a shared library, everything in the library is going to need to be there anyway (which would just affect the size of the library, not the executably anyway). So, I don't see how unnecessary imports would be a problem with regards to executable size. The bigger problem is static constructors and circular dependencies. If the wrong modules import one another when they don't need to, it can result in circular dependencies which will then cause the runtime to exit with an error when you try and run your program. So that - and the fact that it's just cleaner - is a good reason to not have unnecessary imports in a module. However, the circular dependency situation can be made _worse_ by putting imports within a template, since not only is there no way to detect the circular dependency at compile time, but whether you _have_ a circular dependency can depend on whether a particular template was instantiated or not. So, this is definitely not a clear-cut issue. I'd definitely say that if there are only one or two functions in a module which need a particular import, then it's probably better to restrict that import to those functions, but beyond that, it's better to just stick them at the module level. But since (as you point out) local imports are quite new, I really don't think that much of Phobos is taking advantage of them even if it should be. That'll probably change over time, but it's not exactly a high priority. - Jonathan M Davis
Nov 09 2011