www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Module/Package "Best Practices"

reply "news.digitalmars.com" <dbarrett quinthar.com> writes:
What's the "best practice" surrounding modules and packages?

I want to have a library of reusable code that I can import into an
app-specific .d file with a single statement.  This library is composed of
dozens of classes, each in its own file/module.

In C++ I'd create a single global.h file that includes the .h files for all
library classes.  In this way, the app-specific code can include the entire
library with a single #include statement.

In D I *think* I can do the same thing: just create a global.d file that
imports all the .d files of the library.  Thus the app-specific code just
imports the library.global module, and it gets the whole thing.

Is this the way it's supposed to be done?

I'd prefer some kind of wildcard import, like:

#
# import library.*;
#

But that doesn't appear to work.

What have people been doing to organize and separate their reusable and
app-specific code?

-david
Jul 02 2004
next sibling parent Charlie <Charlie_member pathlink.com> writes:
For libraries thats what ive done, create module project.api; which contains all
nessecary imports.  

For most other imports its usually necessary to import them at the class level
to avoid forward reference problems, which for large projects can be a show
stopper ( DWT I know has struggled with it ).

Chuck

In article <cc5ipd$22jv$1 digitaldaemon.com>, news.digitalmars.com says...
What's the "best practice" surrounding modules and packages?

I want to have a library of reusable code that I can import into an
app-specific .d file with a single statement.  This library is composed of
dozens of classes, each in its own file/module.

In C++ I'd create a single global.h file that includes the .h files for all
library classes.  In this way, the app-specific code can include the entire
library with a single #include statement.

In D I *think* I can do the same thing: just create a global.d file that
imports all the .d files of the library.  Thus the app-specific code just
imports the library.global module, and it gets the whole thing.

Is this the way it's supposed to be done?

I'd prefer some kind of wildcard import, like:

#
# import library.*;
#

But that doesn't appear to work.

What have people been doing to organize and separate their reusable and
app-specific code?

-david

Jul 03 2004
prev sibling parent reply Andy Friesen <andy ikagames.com> writes:
news.digitalmars.com wrote:

 What's the "best practice" surrounding modules and packages?
 
 I want to have a library of reusable code that I can import into an
 app-specific .d file with a single statement.  This library is composed of
 dozens of classes, each in its own file/module.
 
 In D I *think* I can ... just create a global.d file that
 imports all the .d files of the library.  Thus the app-specific code just
 imports the library.global module, and it gets the whole thing.
 
 Is this the way it's supposed to be done?
 
 I'd prefer some kind of wildcard import, like:
 
 #
 # import library.*;
 #
 
 But that doesn't appear to work.
 
 What have people been doing to organize and separate their reusable and
 app-specific code?

I just make a module of the same name as the package. It's a tad awkward to say import libfork.libfork; But I think the meaning is clear enough. I've seen 'all.d' used as well. It'd be nice to have an explicit convention for this sort of thing, but I'm happy as long as it says what it is. :) -- andy
Jul 03 2004
parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <cc6qas$112n$1 digitaldaemon.com>, Andy Friesen says...

I just make a module of the same name as the package.  It's a tad 
awkward to say

     import libfork.libfork;

But I think the meaning is clear enough.  I've seen 'all.d' used as 
well.

Yeah, that's what I do too. Hence etc.bigint.bigint, etc.crypto.crypto and etc.unicode.unicode. (The latter two aren't actually there yet).
It'd be nice to have an explicit convention for this sort of 
thing, but I'm happy as long as it says what it is. :)

Well, I borrowed the idea from std.windows.windows and std.linux.linux. I guess if Phobos can do it, that gotta at least count towards a convention. Arcane Jill
Jul 03 2004
parent reply Ant <Ant_member pathlink.com> writes:
In article <cc76jk$1i2p$1 digitaldaemon.com>, Arcane Jill says...
In article <cc6qas$112n$1 digitaldaemon.com>, Andy Friesen says...

I just make a module of the same name as the package.  It's a tad 
awkward to say

     import libfork.libfork;

But I think the meaning is clear enough.  I've seen 'all.d' used as 
well.

etc.bigint.bigint etc.crypto.crypto etc.unicode.unicode std.windows.windows std.linux.linux

I don't like it, it's not really descriptive. both all.d and api.d look better to me. DUI has "All.d" but it will change to "all.d" but this is a minor issue. Ant
Jul 03 2004
parent reply Sam McCall <tunah.d tunah.net> writes:
Ant wrote:
 In article <cc76jk$1i2p$1 digitaldaemon.com>, Arcane Jill says...
 
In article <cc6qas$112n$1 digitaldaemon.com>, Andy Friesen says...


I just make a module of the same name as the package.  It's a tad 
awkward to say

    import libfork.libfork;

But I think the meaning is clear enough.  I've seen 'all.d' used as 
well.

etc.bigint.bigint etc.crypto.crypto etc.unicode.unicode std.windows.windows std.linux.linux

I don't like it, it's not really descriptive. both all.d and api.d look better to me. DUI has "All.d" but it will change to "all.d" but this is a minor issue. Ant

Walter, can we have * added to the valid identifier characters? ;-) </Java> Sam
Jul 04 2004
parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <ccaoqp$fli$1 digitaldaemon.com>, Sam McCall says...

<Java>
Walter, can we have * added to the valid identifier characters? ;-)
</Java>

Is there any reason why the import statement can't be made to allow: # import abc.def.*; meaning, import all .d files in directory .../abc/def? (just like Java does) Arcane Jill
Jul 05 2004
next sibling parent reply Gold Dragon <dragonwing dragonu.net> writes:
I was thinking the same thing, but then again maybe you wouldn't want to 
include all of the modules. What Java does mostly with the classes that 
Sun created are built in to the VM so that when you compile the code, 
you aren't actually including any of those classes. Just opcodes of the 
classes so that the VM can tell what class you were using. This allows 
you to use the '*' without having bloat. D would only include a module 
once in the program but would include/import ALL of the modules if you 
use them or not.

If it can tell if you used a module or not then I would agree that it 
would be useful.

 Is there any reason why the import statement can't be made to allow:
 
 #    import abc.def.*;
 
 meaning, import all .d files in directory .../abc/def?
 (just like Java does)
 
 Arcane Jill

Jul 05 2004
parent Sam McCall <tunah.d tunah.net> writes:
Gold Dragon wrote:

 I was thinking the same thing, but then again maybe you wouldn't want to 
 include all of the modules. What Java does mostly with the classes that 
 Sun created are built in to the VM so that when you compile the code, 
 you aren't actually including any of those classes. Just opcodes of the 
 classes so that the VM can tell what class you were using.  This allows
 you to use the '*' without having bloat.

This is also how D always works. Sam
Jul 06 2004
prev sibling parent reply Benji Smith <dlanguage xxagg.com> writes:
On Mon, 5 Jul 2004 20:40:37 +0000 (UTC), Arcane Jill
<Arcane_member pathlink.com> wrote:

Is there any reason why the import statement can't be made to allow:

#    import abc.def.*;

meaning, import all .d files in directory .../abc/def?
(just like Java does)

Actually, I really dislike Java's wildcard imports. It makes it non-obvious which sub-packages are being imported. This can be a pain when using a tool like grep to search for source files which import a particular package or module. As a matter of style, when I'm writing Java code, I always include a separate import statement for each distinct package/module that I'll be using. Not only does this provide for clearer, more precise code, it also prevents enormous amounts of runtime bloat that would otherwise result from loading classes that won't be used during program execution. (I imagine that a similar amount of bloat is avoided in D by getting rid of the wildcard imports.) That's my 2c, anyhow. --Benji Smith
Jul 05 2004
next sibling parent Arcane Jill <Arcane_member pathlink.com> writes:
In article <g6sje0po6iulbho0t8j5fab8gga4nr1kkb 4ax.com>, Benji Smith says...
On Mon, 5 Jul 2004 20:40:37 +0000 (UTC), Arcane Jill
<Arcane_member pathlink.com> wrote:

Is there any reason why the import statement can't be made to allow:

#    import abc.def.*;

meaning, import all .d files in directory .../abc/def?
(just like Java does)

As a matter of style, when I'm writing Java code, I always include a separate import statement for each distinct package/module that I'll be using.

Understood, and actually I agree with you. However, that's not really what we're talking about. Suppose I write a "distinct package" comprising two hundred individual files. It would makes sense for me to provide a single importable module which imports all the others. The original question on this thread was "What should that module be called"? Phobos (and now Deimos) uses: # import abc.def.def; Others prefer: # import abc.def.all; Of course, what we'd ALL prefer is: # import abc.def; But D doesn't allow that (module and package with same name). And so, as a kind of "second choice", I ventured that: # import abc.def.*; might be an acceptable alternative. I guess what we really need here is a proclaimation in the style-guide. I'm not hung up on ANY particular convention, but I do want "what people expect" to "just work", and what people expect is entirely dependent on consistency across multiple projects. Arcane Jill
Jul 06 2004
prev sibling parent reply Sam McCall <tunah.d tunah.net> writes:
Benji Smith wrote:

 On Mon, 5 Jul 2004 20:40:37 +0000 (UTC), Arcane Jill
 <Arcane_member pathlink.com> wrote:
 
 
Is there any reason why the import statement can't be made to allow:

#    import abc.def.*;

meaning, import all .d files in directory .../abc/def?
(just like Java does)

Actually, I really dislike Java's wildcard imports. It makes it non-obvious which sub-packages are being imported.

Technically, java.awt.image (for example), doesn't have any magic relationship with java.awt, they're two unrelated packages.
 This can be a pain
 when using a tool like grep to search for source files which import a
 particular package 

class/interfaces (java) in that package, in which case search for 'import package.[^.]+;' for "any module in a package", or 'import package.*;' for "all modules in a package". If * was available, sure, you couldn't search for "all modules in a package, imported separately", but you can't do that now and it's pretty meaningless.
 or module.

 Not only does this provide for clearer, more precise code.

ambiguities.
 it also prevents enormous amounts of runtime bloat that would
 otherwise result from loading classes that won't be used during
 program execution.

symbollic level, and only needed code is linked in. If that's not the case, then we've got big problems anyway. Arcane Jill wrote:
 Phobos (and now Deimos) uses:
 #    import abc.def.def;
 Others prefer:
 #    import abc.def.all;
 Of course, what we'd ALL prefer is:
 #    import abc.def;
 But D doesn't allow that (module and package with same name).

import abc.def; (where abc.def is a package) imported the module abc.def.package, or if that wasn't found, all modules in abc.def (but not subpackages of abc.def)? (I say not subpackages because subpackages could be used for internal implementation stuff). Seems fairly simple and useful. Arguments for/against? Sam
Jul 06 2004
parent reply Regan Heath <regan netwin.co.nz> writes:
On Tue, 06 Jul 2004 21:12:50 +1200, Sam McCall <tunah.d tunah.net> wrote:
 Benji Smith wrote:

 On Mon, 5 Jul 2004 20:40:37 +0000 (UTC), Arcane Jill
 <Arcane_member pathlink.com> wrote:


 Is there any reason why the import statement can't be made to allow:

 #    import abc.def.*;

 meaning, import all .d files in directory .../abc/def?
 (just like Java does)

Actually, I really dislike Java's wildcard imports. It makes it non-obvious which sub-packages are being imported.

Technically, java.awt.image (for example), doesn't have any magic relationship with java.awt, they're two unrelated packages.
 This can be a pain
 when using a tool like grep to search for source files which import a
 particular package

class/interfaces (java) in that package, in which case search for 'import package.[^.]+;' for "any module in a package", or 'import package.*;' for "all modules in a package". If * was available, sure, you couldn't search for "all modules in a package, imported separately", but you can't do that now and it's pretty meaningless.
 or module.

 Not only does this provide for clearer, more precise code.

ambiguities.
 it also prevents enormous amounts of runtime bloat that would
 otherwise result from loading classes that won't be used during
 program execution.

symbollic level, and only needed code is linked in. If that's not the case, then we've got big problems anyway. Arcane Jill wrote: > Phobos (and now Deimos) uses: > # import abc.def.def; > Others prefer: > # import abc.def.all; > Of course, what we'd ALL prefer is: > # import abc.def; > But D doesn't allow that (module and package with same name).

I think having something like this is a good idea. I have just done some hashing routines, the modules include: etc.crypto.hash.md2 etc.crypto.hash.md4 etc.crypto.hash.md5 etc.crypto.hash.sha0 etc.crypto.hash.sha1 etc.crypto.hash.tiger which are intended for public importation, and etc.crypto.hash.tigerboxes etc.crypto.hash.misc etc.crypto.hash.sha etc.crypto.hash.md which are used internally. I have created a etc.crypto.hash.hash module which imports the ones intended for public import, so you can simply import that and get them all. What would be better is being able to say either: import etc.crypto.hash or import etc.crypto.hash.* to import everything in the etc.crypto.hash module level, but, not the ones below, thus I could move the internal ones to: etc.crypto.hash.sys.tigerboxes etc.crypto.hash.sys.misc etc.crypto.hash.sys.sha etc.crypto.hash.sys.md so as they are not imported by the above import directives. I think I'll do this anyway, what does everyone think of the 'sys' package level name for internal modules? anyone got a better name? 'internal' maybe, I was hoping for a more terse suggestion :)
 What about if
 import abc.def;
 (where abc.def is a package) imported the module abc.def.package

This would not be obvious when you read "import abc.def;" so I think it's a bad idea.
 , or if that wasn't found, all modules in abc.def (but not subpackages 
 of abc.def)?

I like this.
 (I say not subpackages because subpackages could be used for internal 
 implementation stuff).

Yeah :)
 Seems fairly simple and useful.

 Arguments for/against?

I only have the 1 against that .package idea. Regan -- Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/
Jul 06 2004
next sibling parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <opsapm5e0b5a2sq9 digitalmars.com>, Regan Heath says...

so as they are not imported by the above import directives. I think I'll 
do this anyway, what does everyone think of the 'sys' package level name 
for internal modules? anyone got a better name? 'internal' maybe, I was 
hoping for a more terse suggestion :)

So, generalizing this, you mean... If a package exists, (previously) called abc.def, containing modules abc.def.m1, abc.def.m2 and abc.def.m3, then, instead of doing this (which is what we do now): # Create a module abc.def.def (or abc.def.all) which imports m1, m2 and m3 We instead relocate m1, m2 and m3 to: # abc.def.sys.m1 # abc.def.sys.m2 # abc.def.sys.m3 and use # abc.def to import the above? I don't think that will work, because you still have a module and package with the same name. I think I may have misunderstood you though. Of course, you could instead relocate m1, m2 and m3 to: # abc.def_sys.m1 # abc.def_sys.m2 # abc.def_sys.m3 or even: # abc.def_m1 # abc.def_m2 # abc.def_m3 (Simplest convention I can think of - replace the last dot with an underscore). Once upon a time, I did something like that with the Int class. My subdirectory was called bigint_files. It seemed rather ugly, so I changed it. The thing is, though conventions only become conventions if used by a whole bunch of people, and I'd rather "join in" (do what everyone else does) than "cheerlead" (push for a particular convention). With one exception, of course. I'd like to push for being able to import the module: # abc.def which in turn imports # abc.def.m1 # abc.def.m2 # abc.def.m3 This is currently disallowed (module and package with same name), but I /really/ don't understand why this should be disallowed. The only argument against I've heard so far is Walter saying something like "it wouldn't work", which, as reasoned arguments go, I didn't find entirely convincing. I'd be really interested in hearing other people's ideas on this one. Arcane Jill
Jul 06 2004
parent Regan Heath <regan netwin.co.nz> writes:
On Tue, 6 Jul 2004 11:28:17 +0000 (UTC), Arcane Jill 
<Arcane_member pathlink.com> wrote:
 In article <opsapm5e0b5a2sq9 digitalmars.com>, Regan Heath says...

 so as they are not imported by the above import directives. I think I'll
 do this anyway, what does everyone think of the 'sys' package level name
 for internal modules? anyone got a better name? 'internal' maybe, I was
 hoping for a more terse suggestion :)

So, generalizing this, you mean... If a package exists, (previously) called abc.def, containing modules abc.def.m1, abc.def.m2 and abc.def.m3, then, instead of doing this (which is what we do now): # Create a module abc.def.def (or abc.def.all) which imports m1, m2 and m3 We instead relocate m1, m2 and m3 to: # abc.def.sys.m1 # abc.def.sys.m2 # abc.def.sys.m3 and use # abc.def to import the above?

No. That's not what I meant to say :) I was simply saying that import abc.def should import abc.def.* BUT not abc.def.*.* soo.. abc.def.m1; abc.def.m2; abc.def.m3; would all be imported by import abc.def; if you did not want m3 to be imported you'd move it to somewhere i.e. abc.def.sys.m3; so that import abc.def would not get it.
 I don't think that will work, because you still have a
 module and package with the same name. I think I may have misunderstood 
 you
 though.

 Of course, you could instead relocate m1, m2 and m3 to:

 #    abc.def_sys.m1
 #    abc.def_sys.m2
 #    abc.def_sys.m3

 or even:

 #    abc.def_m1
 #    abc.def_m2
 #    abc.def_m3

 (Simplest convention I can think of - replace the last dot with an 
 underscore).

 Once upon a time, I did something like that with the Int class. My 
 subdirectory
 was called bigint_files. It seemed rather ugly, so I changed it.

 The thing is, though conventions only become conventions if used by a 
 whole
 bunch of people, and I'd rather "join in" (do what everyone else does) 
 than
 "cheerlead" (push for a particular convention).

 With one exception, of course. I'd like to push for being able to import 
 the
 module:

 #    abc.def

 which in turn imports

 #    abc.def.m1
 #    abc.def.m2
 #    abc.def.m3

 This is currently disallowed (module and package with same name), but I 
 /really/
 don't understand why this should be disallowed. The only argument 
 against I've
 heard so far is Walter saying something like "it wouldn't work", which, 
 as
 reasoned arguments go, I didn't find entirely convincing.

 I'd be really interested in hearing other people's ideas on this one.

 Arcane Jill

-- Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/
Jul 06 2004
prev sibling parent reply Sam McCall <tunah.d tunah.net> writes:
Regan Heath wrote:

 What about if
 import abc.def;
 (where abc.def is a package) imported the module abc.def.package

This would not be obvious when you read "import abc.def;" so I think it's a bad idea.

relevant stuff, and possibly provide some nice aliases. If this just consisted of importing every module, it would be omitted. Even so, I agree, that's exactly what I thought at first. However my question is: Does it need to be obvious? In the code you write yourself, the distinction between a module and a package is clear: a module is a compilation unit/file/bunch of code, and a package is a namespace to store logically grouped code. However when you're using someone else's library, how they split up their modules is less important. A module is (small) bunch of logically grouped code that you want to import, and a package is a (large) bunch of logically grouped code you want to import (and you usually achieve this by importing a "magic" module). In view of this, importing packages and modules with the same syntax is a feature, not a bug. It hides the organisation details of the library code and allows library authors to switch between the two at will and according to preference. At the moment, monolithic modules IMO are a pain to edit but easy to import and split up packages are nicer to edit but a pain for importing. Removing the syntactic difference should give us the best of both worlds.
 , or if that wasn't found, all modules in abc.def (but not subpackages 
 of abc.def)?


your subpackages pulled in (and obviously, sometimes you don't). Also, people may have private implementation packages directly in the package. IMO, the language shouldn't force people to use subpackages.
 so as they are not imported by the above import directives. I think
 I'll do this anyway, what does everyone think of the 'sys' package
 level name for internal modules? anyone got a better name? 'internal' 
 maybe, I was hoping for a more terse suggestion  :)

common as a classname suffix and I've seen it used for packages. Sam
Jul 06 2004
next sibling parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <cce9r5$2v52$1 digitaldaemon.com>, Sam McCall says...

In view of this, importing packages and modules with the same syntax is 
a feature, not a bug.

Well, it would be, if you could do it. But so far as I am aware, importing packages is currently not possible with /any/ syntax. If you tell me I'm wrong, I'll be really happy.
It hides the organisation details of the library 
code and allows library authors to switch between the two at will and 
according to preference.

Yes of course. But are you describing an existing feature or a wished-for feature? Arcane Jill
Jul 06 2004
parent Sam McCall <tunah.d tunah.net> writes:
Arcane Jill wrote:
 Yes of course. But are you describing an existing feature or a wished-for
 feature?

suggestion I made: I wrote:
 What about if
 import abc.def;
 (where abc.def is a package) imported the module abc.def.package, or
 if that wasn't found, all modules in abc.def (but not subpackages of
 abc.def)?
 (I say not subpackages because subpackages could be used for internal
 implementation stuff).

 Seems fairly simple and useful.

Sam
Jul 06 2004
prev sibling parent Regan Heath <regan netwin.co.nz> writes:
On Wed, 07 Jul 2004 01:38:38 +1200, Sam McCall <tunah.d tunah.net> wrote:
 Regan Heath wrote:

 What about if
 import abc.def;
 (where abc.def is a package) imported the module abc.def.package

This would not be obvious when you read "import abc.def;" so I think it's a bad idea.

relevant stuff, and possibly provide some nice aliases. If this just consisted of importing every module, it would be omitted. Even so, I agree, that's exactly what I thought at first. However my question is: Does it need to be obvious?

It makes it easier to learn. If anyone can look at it and make a guess at what it does and be right most of the time, that's a good thing. I don't like hidden behaviour, to me suggesting that "import abc.def;" looks for abc.def.package is describing hidden behaviour. I'd prefer "import abc.def;" simply imported everything in that package i.e. abc.def.* but not abc.def.*.*
 In the code you write yourself, the distinction between a module and a 
 package is clear: a module is a compilation unit/file/bunch of code, and 
 a package is a namespace to store logically grouped code.
 However when you're using someone else's library, how they split up 
 their modules is less important. A module is (small) bunch of logically 
 grouped code that you want to import, and a package is a (large) bunch 
 of logically grouped code you want to import (and you usually achieve 
 this by importing a "magic" module).
 In view of this, importing packages and modules with the same syntax is 
 a feature, not a bug.

I agree. I say: import x.y; should import module 'y' in package 'x'. OR all modules in package 'y', in package 'x'. depending on whether y is a module or a package. The person doing this import does not and should not care whether y is a module or a package.
 It hides the organisation details of the library code and allows library 
 authors to switch between the two at will and according to preference.

Agreed.
 At the moment, monolithic modules IMO are a pain to edit but easy to 
 import and split up packages are nicer to edit but a pain for importing. 
 Removing the syntactic difference should give us the best of both worlds.

 , or if that wasn't found, all modules in abc.def (but not subpackages 
 of abc.def)?


your subpackages pulled in (and obviously, sometimes you don't). Also, people may have private implementation packages directly in the package. IMO, the language shouldn't force people to use subpackages.

I'm on the fence here, on one hand I think it *should* force them to have sub-packages, as that forces a consistent organisation to the code. On the other, I don't like being told what to do. The other alternative is to allow module declarations like: public module a.foobar; which simply means it *is* imported when the package 'a' is imported, or alternately private module a.foobar; which means it *is not* imported when the package 'a' is imported. I think I prefer forcing people to organise the internals in sub-packages.
  > so as they are not imported by the above import directives. I think
  > I'll do this anyway, what does everyone think of the 'sys' package
  > level name for internal modules? anyone got a better name? 'internal' 
  > maybe, I was hoping for a more terse suggestion  :)
 What about "impl"? (standing for implementation). At least in java, it's 
 common as a classname suffix and I've seen it used for packages.

I like it. Regan -- Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/
Jul 06 2004