digitalmars.D - A suggestion for modules names / sharing code between projects
- Sebastien Alaiwan (44/44) Feb 29 2016 Hi all,
- ag0aep6g (8/27) Feb 29 2016 How is not going to scale? You have to add an import path for every
- Jonathan M Davis (10/13) Feb 29 2016 My solution would be to never have the same file be part of
- Sebastien Alaiwan (9/16) Feb 29 2016 Thanks for your answer ; from what I understand, you're basically
- Jonathan M Davis (26/43) Feb 29 2016 Phobos is a library, and templates work with it just fine. You're
- Jesse Phillips (20/36) Feb 29 2016 I've used this pattern.
- Sebastien Alaiwan (5/23) Feb 29 2016 Yeah, I'm using this pattern too at the moment ;
- Adam D. Ruppe (8/11) Feb 29 2016 Those module declarations aren't redundant - they are virtually
- Sebastien Alaiwan (12/23) Feb 29 2016 $ find
- Adam D. Ruppe (5/9) Feb 29 2016 You have to tell the compiler where it is. Either way, the module
- Sebastien Alaiwan (7/16) Feb 29 2016 Because it reduces turnaround time (this is something I actually
- Mike Parker (41/47) Feb 29 2016 Given two modules, src/lib/foo.d and and src/main.d, you can pass
- Sebastien Alaiwan (18/18) Feb 29 2016 Hi Mike, thanks for taking the time to answer.
- Guillaume Piolat (13/20) Mar 02 2016 Let me hype DUB a bit. :)
- Adam D. Ruppe (5/8) Feb 29 2016 That's easy to automate, just run a find/replace across all the
- Sebastien Alaiwan (4/12) Feb 29 2016 Yes, I know ; but how can you say there's no redundancy when I
- Guillaume Piolat (11/39) Mar 01 2016 Would this work?
- Sebastien Alaiwan (12/21) Mar 01 2016 Yeah, this is what I going to do: flat hierarchy, like one 'src'
- Mike Parker (22/33) Mar 02 2016 Every module you create will have a module name whether you use a
- Sebastien Alaiwan (119/122) Mar 02 2016 First, you must know that I've been a C++ programmer for way too
- ag0aep6g (31/52) Mar 02 2016 dmd gives you COFF with -m64 or -m32mscoff (-m32 is the default is => OM...
- Sebastien Alaiwan (37/52) Mar 02 2016 Hi guys,
- Laeeth Isharc (10/53) Mar 03 2016 See modules doc page off dlang.org.
- Sebastien Alaiwan (11/23) Mar 03 2016 Thansk, but I was talking about module name collisions, e.g if my
- ag0aep6g (13/24) Mar 03 2016 Yeah, that's a thing. For published libraries one should choose a unique...
- Adam D. Ruppe (2/3) Mar 03 2016 LOL at the code section of that page.
- Mike Parker (57/124) Mar 02 2016 The package name is part of the module name. If you use
- Mike Parker (16/22) Mar 02 2016 I should also say that I get the impression that you aren't
- Guillaume Piolat (4/9) Mar 02 2016 TBH I'm not sure what module directives do now that you mention
Hi all, I've came across the following problem number of times. Let's say I have project A and B, sharing code but having a different tree structure. $ find projectA ./projectA/internal/math/algo.d ./projectA/internal/math/lcp.d ./projectA/internal/math/optimize.d ./projectA/gui/main.d $ find projectB ./projectB/app/render/gfx.d ./projectB/app/render/algo.d ./projectB/app/physics/math/algo.d ./projectB/app/physics/math/lcp.d ./projectB/app/physics/math/optimize.d ./projectB/main.d The directory "math" is shared between projects. It actually is an external/submodule. So it has a standalone existence as a library, and might one day be used by projectC. (In the context of this issue, I'm using separate compilation). I'd like to be able to write, in projectA's main: import internal.math.optimize; This requires me to add, at the beginning of "optimize.d" file, this module definition: module internal.math.optimize; However, this "optimize.d" file is shared between projects, now it's becoming specific to projectA. How am I supposed to share code between projects then? Clearly, putting everyone back into the same "file namespace" by adding every subdirectory of my project as import paths through the command line is not going to scale. After many projects spent thinking of this issue, I came to the conclusion that being able to specify, on the command line, the module name of the module currently being compiled, thus allowing to override it to be inferred to the basename of the file, would solve this problem. As a side-problem related to this one, having to repeat the "absolute" module path to all module declarations/importations is tedious. What if I change the name of the package? Now I might have to change hundreds of module declarations from "module oldname.algo;" to "module newname.algo;". Clearly, something must be wrong here (and I hope it's me!) I'd be very happy if someone showed me how solve this problem, this has bothered me for ages.
Feb 29 2016
On 29.02.2016 20:03, Sebastien Alaiwan wrote:$ find projectA ./projectA/internal/math/algo.d ./projectA/internal/math/lcp.d ./projectA/internal/math/optimize.d ./projectA/gui/main.d $ find projectB ./projectB/app/render/gfx.d ./projectB/app/render/algo.d ./projectB/app/physics/math/algo.d ./projectB/app/physics/math/lcp.d ./projectB/app/physics/math/optimize.d ./projectB/main.d The directory "math" is shared between projects. It actually is an external/submodule. So it has a standalone existence as a library, and might one day be used by projectC. (In the context of this issue, I'm using separate compilation).[...]Clearly, putting everyone back into the same "file namespace" by adding every subdirectory of my project as import paths through the command line is not going to scale.How is not going to scale? You have to add an import path for every location where packages are to be found. How many such locations are you going to have that this is unreasonable? With your example, you'd have -Iinternal in project A, and -Iapp/physics in project B. The module declaration would of course be `module math.optimize;`.
Feb 29 2016
On Monday, 29 February 2016 at 19:03:53 UTC, Sebastien Alaiwan wrote:Clearly, something must be wrong here (and I hope it's me!) I'd be very happy if someone showed me how solve this problem, this has bothered me for ages.My solution would be to never have the same file be part of multiple projects. Share code using libraries rather than just sharing the files. Then modules won't be changing where they are in a project or anything like that. The shared code goes into a library, and whichever projects need it link against that library. You get better modularization and encapsulation that way and completely avoid the problem that you're having. - Jonathan M Davis
Feb 29 2016
On Monday, 29 February 2016 at 19:23:08 UTC, Jonathan M Davis wrote:My solution would be to never have the same file be part of multiple projects. Share code using libraries rather than just sharing the files. Then modules won't be changing where they are in a project or anything like that. The shared code goes into a library, and whichever projects need it link against that library. You get better modularization and encapsulation that way and completely avoid the problem that you're having.Thanks for your answer ; from what I understand, you're basically telling me to only import binary/precompiled libraries, right? Although I had to admit I had never considered this solution, never have the same file be part of multiple projects seems extremely restrictive to me. How is it going to work if I want to share heavily templated code, like a container library?
Feb 29 2016
On Monday, 29 February 2016 at 20:00:48 UTC, Sebastien Alaiwan wrote:On Monday, 29 February 2016 at 19:23:08 UTC, Jonathan M Davis wrote:Phobos is a library, and templates work with it just fine. You're still importing the .d files. They're just compiled in as part of the library that you link against rather than compiled as part of your project - e.g. pretty much every D program is linked against libphobos, and none of them compile in modules like std.stdio. They just import them. They were compiled as part of libphobos are aside from templates are not compiled as part of your program, just linked. As for templated code, the templates are generated when your code is built, so they're not really linked in in the normal sense, but those modules are still part of the library and not part of your code, so you'd never do something like add std/algorithm/searching.d to the list of files that you're compiling. You'd just import it. If someone wants to hide code, then they'd use .di files and give you those to link against, in which case non-templated code could be hidden, but templated code would still be in those files, because the compiler has to see their source when they're used. And most folks will likely just use .d files, since it's simpler and allows stuff like CTFE and inlining to work. Most of the projects on code.dlang.org are libraries, and you're using dub, they're easy to pull in and link against. You can do the same with your code. - Jonathan M DavisMy solution would be to never have the same file be part of multiple projects. Share code using libraries rather than just sharing the files. Then modules won't be changing where they are in a project or anything like that. The shared code goes into a library, and whichever projects need it link against that library. You get better modularization and encapsulation that way and completely avoid the problem that you're having.Thanks for your answer ; from what I understand, you're basically telling me to only import binary/precompiled libraries, right? Although I had to admit I had never considered this solution, never have the same file be part of multiple projects seems extremely restrictive to me. How is it going to work if I want to share heavily templated code, like a container library?
Feb 29 2016
On Monday, 29 February 2016 at 19:03:53 UTC, Sebastien Alaiwan wrote:Hi all, I've came across the following problem number of times. Let's say I have project A and B, sharing code but having a different tree structure. $ find projectA ./projectA/internal/math/algo.d ./projectA/internal/math/lcp.d ./projectA/internal/math/optimize.d ./projectA/gui/main.d $ find projectB ./projectB/app/render/gfx.d ./projectB/app/render/algo.d ./projectB/app/physics/math/algo.d ./projectB/app/physics/math/lcp.d ./projectB/app/physics/math/optimize.d ./projectB/main.dI've used this pattern. ./projectA/lib/math/algo.d ./projectA/lib/math/lcp.d ./projectA/lib/math/optimize.d ./projectA/gui/main.d ./projectB/app/render/gfx.d ./projectB/app/render/algo.d ./projectB/lib/math/algo.d ./projectB/lib/math/lcp.d ./projectB/lib/math/optimize.d ./projectB/main.d Dub doesn't like me too much, but in general it works. Note that it doesn't matter where your modules live, if they have the declared module name in the file: module math.optimize; If DMD reads this file all your imports should look like: import math.optimize; Even if the file is in app/physics/math.
Feb 29 2016
On Monday, 29 February 2016 at 19:56:20 UTC, Jesse Phillips wrote:I've used this pattern. ./projectA/lib/math/algo.d ./projectA/lib/math/lcp.d ./projectA/lib/math/optimize.d ./projectA/gui/main.d ./projectB/app/render/gfx.d ./projectB/app/render/algo.d ./projectB/lib/math/algo.d ./projectB/lib/math/lcp.d ./projectB/lib/math/optimize.d ./projectB/main.d Dub doesn't like me too much, but in general it works. Note that it doesn't matter where your modules live, if they have the declared module name in the file: module math.optimize; If DMD reads this file all your imports should look like: import math.optimize; Even if the file is in app/physics/math.Yeah, I'm using this pattern too at the moment ; Although, I'm trying to avoid having these redundant module declaration directives at the beginning of each of my library files.
Feb 29 2016
On Monday, 29 February 2016 at 20:05:11 UTC, Sebastien Alaiwan wrote:Although, I'm trying to avoid having these redundant module declaration directives at the beginning of each of my library files.Those module declarations aren't redundant - they are virtually required (I think it is a mistake that they aren't explicitly required in all cases, actually) The file layout does not matter to the language itself. Only that module declaration does - it is NOT optional if you want a package name.
Feb 29 2016
On Monday, 29 February 2016 at 20:59:45 UTC, Adam D. Ruppe wrote:On Monday, 29 February 2016 at 20:05:11 UTC, Sebastien Alaiwan wrote:$ find main.d foo/hello.d $ cat main.d import foo.hello; $ cat foo/hello.d module foo.hello; Ok so now let's say I rename the directory "lib" to "foo". If I don't change the "import lib.hello" to "import foo.hello", how is the compiler going to find "hello.d"? (As a reminder, as I said, I'm using separate compilation)Although, I'm trying to avoid having these redundant module declaration directives at the beginning of each of my library files.Those module declarations aren't redundant - they are virtually required (I think it is a mistake that they aren't explicitly required in all cases, actually) The file layout does not matter to the language itself. Only that module declaration does - it is NOT optional if you want a package name.
Feb 29 2016
On Monday, 29 February 2016 at 21:04:52 UTC, Sebastien Alaiwan wrote:Ok so now let's say I rename the directory "lib" to "foo". If I don't change the "import lib.hello" to "import foo.hello", how is the compiler going to find "hello.d"?You have to tell the compiler where it is. Either way, the module name is *not* optional.(As a reminder, as I said, I'm using separate compilation)meh that's part of your problem, why are you doing it that way?
Feb 29 2016
On Monday, 29 February 2016 at 21:33:37 UTC, Adam D. Ruppe wrote:On Monday, 29 February 2016 at 21:04:52 UTC, Sebastien Alaiwan wrote:Is it only possible with separate compilation?Ok so now let's say I rename the directory "lib" to "foo". If I don't change the "import lib.hello" to "import foo.hello", how is the compiler going to find "hello.d"?You have to tell the compiler where it is.Either way, the module name is *not* optional.Because it reduces turnaround time (this is something I actually measured on the project I'm currently working on). Having to recompile everything each time I make a modification takes up to 25s on this project. By the way, I'm using gdc. Is it also part of my problem? :-)(As a reminder, as I said, I'm using separate compilation)meh that's part of your problem, why are you doing it that way?
Feb 29 2016
On Monday, 29 February 2016 at 21:43:31 UTC, Sebastien Alaiwan wrote:Given two modules, src/lib/foo.d and and src/main.d, you can pass them both to the command line and not worry about passing -I to the compiler. dmd src/main.d src/lib/foo.d In this case, foo.d can have a module statement that gives it any name at all. It does not need to match its directory structure: module wicked.cool.mod; This works because you've given the module directly to the compiler and it doesn't need to search for it. If you compile them separately: dmd -c src/lib/foo.d dmd src/main.d foo.obj Now you will see a compiler error. You've given the compiler foo's object file, but it doesn't know where the source file is. When main.d imported wicked.foo.mod, the compiler tried to find wicked/foo/mod.d from its current working directory. No joy. You can test this changing the module declaration in foo.d to lib.foo, then importing lib.foo in main.d. The you can do this: cd src dmd -c lib/foo.d dmd main.d foo.obj Now when it encounters import lib.foo, it will look for lib/foo.d and find it. Now do this: cd .. dmd -c src/lib/foo.d dmd src/main.d foo.obj Again, failure, because it's looking for lib/foo.d. So now you have to tell it where to find lib/foo.d: dmd -Isrc src/main.d foo.obj Now it will look for src/lib/foo.d and find it. This is why you shouldn't muck about with package names. If you want to change the name of the src directory, that's fine. You can simply pass -I to the compiler with the correct value. But if you want to change the name of lib, you are now affecting the compiler's ability to find imports. This becomes a problem with separate compilation (as above), which also means it's a problem when using libraries. If you change a package name for something you're distributing as a library, then all the import statements in client code will need to be changed as well.Is it only possible with separate compilation?Ok so now let's say I rename the directory "lib" to "foo". If I don't change the "import lib.hello" to "import foo.hello", how is the compiler going to find "hello.d"?You have to tell the compiler where it is.
Feb 29 2016
Hi Mike, thanks for taking the time to answer. If I understand you correctly, you're advising me to keep my file hierarchy mostly in sync with my module hierarchy (except for one potential leading "src/" or "libs/" that I would add to the import search directories). That's fine with me. However, what's the point of having module declarations directives in the language then? If I understand correctly, the only way to make an interesting use of these directives makes separate compilation impossible. I know dmd is lightning-fast and everything, and not having to manage build dependencies anymore is very attractive. However, we still need separate compilation. Otherwise your turnaround time is going to resemble a tractor pulling competition as your project grows. Recompiling everything everytime you change one module is not an option ; Some of the source files I work on in my company require 30s to compile *alone* ; it's easy to reach, especially when the language has features like templates and CTFE (Pegged anyone?).
Feb 29 2016
On Tuesday, 1 March 2016 at 06:33:06 UTC, Sebastien Alaiwan wrote:However, we still need separate compilation. Otherwise your turnaround time is going to resemble a tractor pulling competition as your project grows. Recompiling everything everytime you change one module is not an option ; Some of the source files I work on in my company require 30s to compile *alone* ; it's easy to reach, especially when the language has features like templates and CTFE (Pegged anyone?).Let me hype DUB a bit. :) DUB provides completely composable dependencies, in a chain (A => B => C) with A not knowing about C. Compile-times are excellent since DUB can do package-wise, file-wise or all-at-once builds from the same description and use caching. Having a declarative build description allows for some goodies like IDE project generation. The package namespace is global, but hierarchical with names like "myorg:mypackage". However, you can't have versionned dependencies for private packages [painlessly] which is quite a big limitation.
Mar 02 2016
On Monday, 29 February 2016 at 19:03:53 UTC, Sebastien Alaiwan wrote:What if I change the name of the package? Now I might have to change hundreds of module declarations from "module oldname.algo;" to "module newname.algo;".That's easy to automate, just run a find/replace across all the files, or if you want to go really fancy, modify hackerpilot's dfix to do it for you.
Feb 29 2016
On Monday, 29 February 2016 at 21:35:48 UTC, Adam D. Ruppe wrote:On Monday, 29 February 2016 at 19:03:53 UTC, Sebastien Alaiwan wrote:Yes, I know ; but how can you say there's no redundancy when I have to rely on find/replace to keep all these declarations in sync?What if I change the name of the package? Now I might have to change hundreds of module declarations from "module oldname.algo;" to "module newname.algo;".That's easy to automate, just run a find/replace across all the files, or if you want to go really fancy, modify hackerpilot's dfix to do it for you.
Feb 29 2016
On Monday, 29 February 2016 at 19:03:53 UTC, Sebastien Alaiwan wrote:Hi all, I've came across the following problem number of times. Let's say I have project A and B, sharing code but having a different tree structure. $ find projectA ./projectA/internal/math/algo.d ./projectA/internal/math/lcp.d ./projectA/internal/math/optimize.d ./projectA/gui/main.d $ find projectB ./projectB/app/render/gfx.d ./projectB/app/render/algo.d ./projectB/app/physics/math/algo.d ./projectB/app/physics/math/lcp.d ./projectB/app/physics/math/optimize.d ./projectB/main.d The directory "math" is shared between projects. It actually is an external/submodule. So it has a standalone existence as a library, and might one day be used by projectC. (In the context of this issue, I'm using separate compilation). I'd like to be able to write, in projectA's main: import internal.math.optimize; This requires me to add, at the beginning of "optimize.d" file, this module definition: module internal.math.optimize; However, this "optimize.d" file is shared between projects, now it's becoming specific to projectA. How am I supposed to share code between projects then?Would this work? 1. pick a single module name like module math.optimize; 2. import that module with: import math.optimize; 3. put this module in a hierarchy like that: math/optimize.d 4. pass -I<directory-containing-math-directory> to the compiler However it may clutter your module namespace a bit more.
Mar 01 2016
On Wednesday, 2 March 2016 at 00:16:57 UTC, Guillaume Piolat wrote:Would this work? 1. pick a single module name like module math.optimize; 2. import that module with: import math.optimize; 3. put this module in a hierarchy like that: math/optimize.d 4. pass -I<directory-containing-math-directory> to the compiler However it may clutter your module namespace a bit more.Yeah, this is what I going to do: flat hierarchy, like one 'src' directory and one 'extra' directory in the project root, globally unique package names, passing -Isrc and -Iextra to the compiler, and ... module declaration directives. I was trying to avoid them, but maybe it's not possible. I still think that software entities shouldn't depend upon their own name, or their absolute location in the top-level namespace. This is true for classes, functions/methods, variables, structures ... maybe this rule becomes invalid when third-party packages enter a project?
Mar 01 2016
On Wednesday, 2 March 2016 at 06:40:53 UTC, Sebastien Alaiwan wrote:Yeah, this is what I going to do: flat hierarchy, like one 'src' directory and one 'extra' directory in the project root, globally unique package names, passing -Isrc and -Iextra to the compiler, and ... module declaration directives. I was trying to avoid them, but maybe it's not possible.Every module you create will have a module name whether you use a module declaration or not. If you don't use one, the compiler uses the file name by default, but *only* the file name. The path on the file system is in no way taken into account (I suppose it could be by adding a flag to the compiler, but it seems to me that would needlessly complicate what is a really simple and easy to use system). A module with no package name is in the default package, i.e. given src/foo.d, using the following declaration: module foo; Is the same as using none at all. If you are content with keeping all of your modules in one or more top-level directory and use no packages, then you can leave out the module declarations and just pass each top-level directory to the compiler with -I.I still think that software entities shouldn't depend upon their own name, or their absolute location in the top-level namespace. This is true for classes, functions/methods, variables, structures ... maybe this rule becomes invalid when third-party packages enter a project?The module names and package names are mangled into the symbol names in the binary. They also serve as namespaces to distinguish conflicting symbols in code. So yes, all of the symbols in a module are dependent on their location in the namespace. It's baked into the language. I'm curious what sort of system you have in mind. How does the current system not work for you and what would you prefer to see?
Mar 02 2016
On Wednesday, 2 March 2016 at 08:50:50 UTC, Mike Parker wrote:I'm curious what sort of system you have in mind. How does the current system not work for you and what would you prefer to see?First, you must know that I've been a C++ programmer for way too much time ; and that my way of thinking about modules has probably been - badly - shaped accordingly :-). However, it feels wrong that modules (foo, bar) inside the same package ("pkg") need to repeat the "absolute" package name when referencing each other, I mean "import pkg.foo" instead of "import foo". Not because of the extra typing, but because of the dependency it introduces. But as I said, maybe my vision is wrong. I'm going to answer your first question, I hope you don't get bored before the end ;-) First some generic stuff: - The libraries I write are source-only, and I never install them system-wide, as many projects might depend on different revisions of one library. - I only work with separate compilation, using gdc (Windows-dmd produces OMF, which is a dealbreaker ; but I use rdmd a lot for small programs). - I often mix C code with D, sometimes, C++ code. I compile them using gcc/g++. - My build system is composed of simple dependency-aware non-recursive makefiles (BTW, this is a great way to keep your code decoupled, otherwise you might not get feedback that you're making a dependency mess soon enough!) This is the best way I found to be able to checkout one project, type 'make', and have everything build properly, wether I'm doing it on GNU/Linux or on Windows (MSYS). Everything is under version control using GNU Bazaar (which basically is to git what D is to C++). I have a small library, which I call "lib_algo", containing semi-related utilities (containers, topological sort, etc.) which I use in many projects. $ cd myLibs $ find lib_algo -name "*.d" -or -name "*.mk" -or name "Makefile" lib_algo/options.d lib_algo/misc.d lib_algo/pointer.d lib_algo/set.d lib_algo/queue.d lib_algo/algebraic.d lib_algo/vector.d lib_algo/bresenham.d lib_algo/topological.d lib_algo/fix_array.d lib_algo/test.d lib_algo/pool.d lib_algo/stack.d lib_algo/list.d lib_algo/array2d.d lib_algo/project.mk lib_algo/Makefile (I know, there's some overlap with Phobos today) The project.mk simply contains the list of source files of the library. The test.d file actually contains a main function, which I use to run manual tests of the library functionalities. It's not included in the project.mk list. And it obviously must import the files needing to be tested. With this flat directory structure, the only way test.d can import files is by saying, "import stack;", not "import lib_algo.stack". (adding ".." to the list of import directories might do the trick, but this would definitely feel wrong). Now, let's see a project using this "lib_algo", I call it "myGame". $ cd myProjects $ find <files relevant to this discussion> myGame/lib_algo/options.d myGame/lib_algo/.... myGame/lib_algo/array2d.d myGame/lib_algo/project.mk myGame/lib_algo/Makefile myGame/src/main.d myGame/Makefile "lib_algo" simply is an external (aka "git submodule"), pointing to a specific revision of the repository "lib_algo". The top-level "Makefile" of the projects includes "lib_algo/project.mk", and all is well, I can compose the list of source files to compile without having to rely on "convenience/intermediate" libraries. Now, if "src/main.d" wants to import "array2d.d", currently, I must write "import array2d.d", and add "-I myGame/lib_algo" to my compiler command line. I said in a previous post that this would not scale. That's because with this scheme, obviously, I can't have two modules having the same name, even if they belong to different libraries. Let's say I have "lib_algo/utils.d" and "lib_geom/utils.d", if "main.d" says "import utils.d", it's ambiguous. Clearly, "main.d" needs to say "import lib_algo.utils". However, if I do this, I must add a module declaration in "lib_algo/utils.d", and I must change "lib_algo/test.d" so it says "import lib_algo.utils". This is where it begins to feel clumsy. Why should lib_algo need to be modified in order to resolve an ambiguity happening in "myGame" ? Moreover, we saw earlier that this modification of the import done by "lib_algo/test.d" had consequences on the directory structure of lib_algo. In order to be able to build the test executable for lib_algo, I now need to have the following structure: $ find lib_algo lib_algo/lib_algo/options.d lib_algo/lib_algo/.... lib_algo/lib_algo/array2d.d lib_algo/test.d (test.d can be put inside the inner "lib_algo") Am I the only one to find this counterintuitive - and probably wrong? What am I missing there? (And please, don't tell me that I need to switch to dub+git+dmd+D_source_files_only ; I like my tools to be composable) Thanks for reading!
Mar 02 2016
On 02.03.2016 21:40, Sebastien Alaiwan wrote:- I only work with separate compilation, using gdc (Windows-dmd produces OMF, which is a dealbreaker ; but I use rdmd a lot for small programs).dmd gives you COFF with -m64 or -m32mscoff (-m32 is the default is => OMF). [...]Clearly, "main.d" needs to say "import lib_algo.utils". However, if I do this, I must add a module declaration in "lib_algo/utils.d", and I must change "lib_algo/test.d" so it says "import lib_algo.utils".That's the way you're supposed to do it, yes. One source file <-> one module name.This is where it begins to feel clumsy. Why should lib_algo need to be modified in order to resolve an ambiguity happening in "myGame" ?I don't know the exact reasons why the module system works like it does. Maybe you're on to something, maybe not. Anyway, here are some problems/surprises I see with your alternative idea (which basically means using relative file paths, if I get you right): * You need to be aware of the directory structure when reading/writing code, as `import foo;` may mean different things depending on where you are. And you don't have a module declaration to tell you where you are. * You're not able to copy imports from one file to another anymore. * When moving a module, you have to update all its imports. You can introduce a subtle bug when you miss one, because the old import may still be valid but have a different meaning. * The name of a module depends on the current working directory of the compiler run. You can't change directories between compiler calls and link the object files together. * You can deliberately or accidentally import one source file under two different module names. Two modules from one source file is weird, especially when it happens accidentally. * Can't just throw source files from different packages at the compiler anymore without thinking about their relative locations or the -I switch. [...]Moreover, we saw earlier that this modification of the import done by "lib_algo/test.d" had consequences on the directory structure of lib_algo. In order to be able to build the test executable for lib_algo, I now need to have the following structure: $ find lib_algo lib_algo/lib_algo/options.d lib_algo/lib_algo/.... lib_algo/lib_algo/array2d.d lib_algo/test.d (test.d can be put inside the inner "lib_algo")Or, you know: `-I..` :P But changing the directory structure is probably nicer.Am I the only one to find this counterintuitive - and probably wrong? What am I missing there?I don't think your alternative is obviously superior. The current way of having exactly one fixed name per module makes it easy to see what's going on at any point. You have to pay for that by typing a bit more, I guess.
Mar 02 2016
Hi guys, thanks a lot for your answers! On Wednesday, 2 March 2016 at 22:42:18 UTC, ag0aep6g wrote:On 02.03.2016 21:40, Sebastien Alaiwan wrote:Thanks for the tip, I didn't know that. I have actually deeper reasons for not using dmd (seemless cross-compilation, e.g "make CROSS_COMPILE=i686-w64-mingw32" or "make CROSS_COMPILE=arm-linux-gnueabi" without any special case in the Makefile ; and the way its unusual command line (-oftarget) conflicts with MSYS's path conversion, etc. ).- I only work with separate compilation, using gdc (Windows-dmd produces OMF, which is a dealbreaker ; but I use rdmd a lot for small programs).dmd gives you COFF with -m64 or -m32mscoff (-m32 is the default is => OMF).I don't know the exact reasons why the module system works like it does. Maybe you're on to something, maybe not. Anyway, here are some problems/surprises I see with your alternative idea (which basically means using relative file paths, if I get you right):Yes, I think we can say that. I 100% agree with all the limitations you listed ; which makes a forced absolute-import system appear to be a lot more comfortable! There's also one big limitation I see that you didn't list: * If you're importing lib_a and lib_b both depending on lib_c, you need them to expect lib_c to be at the same path/namespace location. The current module system guarantees this, because all package names are global. (And linking with precompiled binaries might become a nightmare) However, using global package names means you're relying on the library providers to avoid conflicts, in their package names, although they might not even know each other. Which basically implies to always put your name / company name in your package name, like "import alaiwan.lib_algo.array2d;", or "import apple.lib_algo.array2d". Or rename my "lib_algo" to something meaningless / less common, like "import diamond.array2d" (e.g Derelict, Pegged, imaged, etc.). If, unfortunately, I happen to run into a conflict, i.e my project uses two unrelated libraries sharing a name for their top-level namespace, there's no way out for me, right? (Except relying on shared objects to isolate symbols)I don't think your alternative is obviously superior.Me neither :-) I'm just sharing my doubts about the current system!The current way of having exactly one fixed name per module makes it easy to see what's going on at any point. You have to pay for that by typing a bit more, I guess.Yes, I guess you're right. I'm going to follow your advice and Mike's for my projects, this will allow me to see things more clearly. Anyway, thanks again to all of you for your answers!
Mar 02 2016
On Thursday, 3 March 2016 at 06:53:33 UTC, Sebastien Alaiwan wrote:Hi guys, thanks a lot for your answers! On Wednesday, 2 March 2016 at 22:42:18 UTC, ag0aep6g wrote:See modules doc page off dlang.org. import io = std.stdio; void main() { io.writeln("hello!"); // ok, calls std.stdio.writeln std.stdio.writeln("hello!"); // error, std is undefined writeln("hello!"); // error, writeln is undefined } Also static import and scoped imports.On 02.03.2016 21:40, Sebastien Alaiwan wrote:Thanks for the tip, I didn't know that. I have actually deeper reasons for not using dmd (seemless cross-compilation, e.g "make CROSS_COMPILE=i686-w64-mingw32" or "make CROSS_COMPILE=arm-linux-gnueabi" without any special case in the Makefile ; and the way its unusual command line (-oftarget) conflicts with MSYS's path conversion, etc. ).- I only work with separate compilation, using gdc (Windows-dmd produces OMF, which is a dealbreaker ; but I use rdmd a lot for small programs).dmd gives you COFF with -m64 or -m32mscoff (-m32 is the default is => OMF).I don't know the exact reasons why the module system works like it does. Maybe you're on to something, maybe not. Anyway, here are some problems/surprises I see with your alternative idea (which basically means using relative file paths, if I get you right):Yes, I think we can say that. I 100% agree with all the limitations you listed ; which makes a forced absolute-import system appear to be a lot more comfortable! There's also one big limitation I see that you didn't list: * If you're importing lib_a and lib_b both depending on lib_c, you need them to expect lib_c to be at the same path/namespace location. The current module system guarantees this, because all package names are global. (And linking with precompiled binaries might become a nightmare) However, using global package names means you're relying on the library providers to avoid conflicts, in their package names, although they might not even know each other. Which basically implies to always put your name / company name in your package name, like "import alaiwan.lib_algo.array2d;", or "import apple.lib_algo.array2d". Or rename my "lib_algo" to something meaningless / less common, like "import diamond.array2d" (e.g Derelict, Pegged, imaged, etc.). If, unfortunately, I happen to run into a conflict, i.e my project uses two unrelated libraries sharing a name for their top-level namespace, there's no way out for me, right? (Except relying on shared objects to isolate symbols)
Mar 03 2016
On Thursday, 3 March 2016 at 08:39:51 UTC, Laeeth Isharc wrote:On Thursday, 3 March 2016 at 06:53:33 UTC, Sebastien AlaiwanThansk, but I was talking about module name collisions, e.g if my project uses two unrelated libraries, both - sloppily - defining a module called "geom.utils". Can this be worked-around using renamed imports? How is "import myUtils = geom.utils" going to know which of both "geom.utils" I'm talking about? As far as I understand, _module_ renaming at import is rather a tool for defining shorthands, than for disambiguation. I agree that it might not be a problem in practice - just wanting to make sure I'm not missing something.If, unfortunately, I happen to run into a conflict, i.e my project uses two unrelated libraries sharing a name for their top-level namespace, there's no way out for me, right? (Except relying on shared objects to isolate symbols)See modules doc page off dlang.org. import io = std.stdio; void main() { io.writeln("hello!"); // ok, calls std.stdio.writeln std.stdio.writeln("hello!"); // error, std is undefined writeln("hello!"); // error, writeln is undefined }
Mar 03 2016
On 03.03.2016 07:53, Sebastien Alaiwan wrote:However, using global package names means you're relying on the library providers to avoid conflicts, in their package names, although they might not even know each other. Which basically implies to always put your name / company name in your package name, like "import alaiwan.lib_algo.array2d;", or "import apple.lib_algo.array2d". Or rename my "lib_algo" to something meaningless / less common, like "import diamond.array2d" (e.g Derelict, Pegged, imaged, etc.).Yeah, that's a thing. For published libraries one should choose a unique name. I think reverse domain name notation [1] is a convention for exactly that. As far as I know, not many D projects do that, though, if any at all. But it would be a way out if collisions become a problem. Also, dub/code.dlang.org plays a role here, as people may assume that a name is free as long as it's not on the dub list. This works well when dub is indeed the one place to go for libraries. If it isn't, then things may get messy.If, unfortunately, I happen to run into a conflict, i.e my project uses two unrelated libraries sharing a name for their top-level namespace, there's no way out for me, right? (Except relying on shared objects to isolate symbols)Yeah, as far as I can tell, you would have to change the name of one of the libraries, and update all related module declarations and imports. [1] https://en.wikipedia.org/wiki/Reverse_domain_name_notation
Mar 03 2016
On Thursday, 3 March 2016 at 20:05:03 UTC, ag0aep6g wrote:[1] https://en.wikipedia.org/wiki/Reverse_domain_name_notationLOL at the code section of that page.
Mar 03 2016
On Wednesday, 2 March 2016 at 20:40:39 UTC, Sebastien Alaiwan wrote:However, it feels wrong that modules (foo, bar) inside the same package ("pkg") need to repeat the "absolute" package name when referencing each other, I mean "import pkg.foo" instead of "import foo". Not because of the extra typing, but because of the dependency it introduces. But as I said, maybe my vision is wrong.The package name is part of the module name. If you use thislib.somemod and thatlib.somemod, how would you disambiguate them without the package name? Even in C and C++, libraries are often distributed with header files in a sub directory, where the top-level directory is intended to be added to the include path and not the subdirecotry, otherwise two includes with the same name conflict. Some libraries go further and prefix their header files with a namespace (like sdl_foo.h, sdl_bar.h), because they get no help from the compiler with this. Making the package name part of the fully qualified module name helps minimize the likelihood of such conflicts. It also gives a natural namespace for conflicting symbols in two modules named foo. You can use thispack.somemod.foo and thatpack.somemod.foo to disambiguate.- I only work with separate compilation, using gdc (Windows-dmd produces OMF, which is a dealbreaker ; but I use rdmd a lot for small programs).-m64 or -m32mscoff- I often mix C code with D, sometimes, C++ code. I compile them using gcc/g++.Assuming MinGW, the GDC is definitely your best bet here. The flags above will get DMD to produce COFF for the MS linker, but MinGW COFF and MS COFF are not always compatible.With this flat directory structure, the only way test.d can import files is by saying, "import stack;", not "import lib_algo.stack". (adding ".." to the list of import directories might do the trick, but this would definitely feel wrong).It is wrong. You should add ../mylibs :)Now, let's see a project using this "lib_algo", I call it "myGame". $ cd myProjects $ find <files relevant to this discussion> myGame/lib_algo/options.d myGame/lib_algo/.... myGame/lib_algo/array2d.d myGame/lib_algo/project.mk myGame/lib_algo/Makefile myGame/src/main.d myGame/Makefile "lib_algo" simply is an external (aka "git submodule"), pointing to a specific revision of the repository "lib_algo". The top-level "Makefile" of the projects includes "lib_algo/project.mk", and all is well, I can compose the list of source files to compile without having to rely on "convenience/intermediate" libraries. Now, if "src/main.d" wants to import "array2d.d", currently, I must write "import array2d.d", and add "-I myGame/lib_algo" to my compiler command line. I said in a previous post that this would not scale. That's because with this scheme, obviously, I can't have two modules having the same name, even if they belong to different libraries. Let's say I have "lib_algo/utils.d" and "lib_geom/utils.d", if "main.d" says "import utils.d", it's ambiguous.Assuming that all of the libraries you are going to use are set up the same way, then you can simply pass -ImyGame. Then you can use lib_algo.utils, lib_geom.utils, and so on. What's the problem with that?Clearly, "main.d" needs to say "import lib_algo.utils". However, if I do this, I must add a module declaration in "lib_algo/utils.d", and I must change "lib_algo/test.d" so it says "import lib_algo.utils". This is where it begins to feel clumsy. Why should lib_algo need to be modified in order to resolve an ambiguity happening in "myGame" ?You shouldn't have to "change" it. This is how it should be set up from the beginning. The idea is that you decide on a module structure as soon as you start your project and then you stick to it. The module declaration is necessary when using packages because otherwise DMD has no idea how far up the path to go to create a default package name. That's why the it uses only the file name when no module declaration is present. The package name is the solution to *prevent* ambiguities, not to resolve them after the fact. If you don't want lib_algo/util.d and lib_geom/util.d to conflict, then the use both lib_algo and lib_geom as package names in module declarations. How else would you propose resolving the ambiguity?Moreover, we saw earlier that this modification of the import done by "lib_algo/test.d" had consequences on the directory structure of lib_algo. In order to be able to build the test executable for lib_algo, I now need to have the following structure: $ find lib_algo lib_algo/lib_algo/options.d lib_algo/lib_algo/.... lib_algo/lib_algo/array2d.d lib_algo/test.d (test.d can be put inside the inner "lib_algo") Am I the only one to find this counterintuitive - and probably wrong? What am I missing there?No, you don't have to set it up that way. If your top-level directory is always named the same as your top-level package name, then you can just use its parent directory. mylibs/lib_algo/*.d mylibs/lib_geom/*.d dmd -Imylibs main.d Done. Does not the same issue exist in C++? You either have to keep all of your headers in a single directory or pass multiple directories on the command line with -I. What I used to do before I started using DUB exclusively was model the system I used for C and C++ projects. For every new project I created, I had created an imports subdirectory in the project, then ran a script (written in D) to copy over the source tree of the specific version of each library I wanted to use. Then I added -Iimport on the command line when compiling. Done. The package and module system is very intuitive to me and has been from the beginning, so I'm still not quite sure what it is about it that bothers you. The two major issues you bring up here, the need to import package.module and how to structure your projects, are easily resolved. Am I misunderstanding something?
Mar 02 2016
On Thursday, 3 March 2016 at 01:47:08 UTC, Mike Parker wrote:The package and module system is very intuitive to me and has been from the beginning, so I'm still not quite sure what it is about it that bothers you. The two major issues you bring up here, the need to import package.module and how to structure your projects, are easily resolved. Am I misunderstanding something?I should also say that I get the impression that you aren't considering the package name as part of the module name. Maybe that's the root of your problem. As I said in a previous post, your desire to use the default package for everything so that you can simply import foo is perfectly fine. The price of doing that, as you point out here, is that you can easily run into conflicts. Packages are the solution to avoiding them and package names are part of the module name, such that module foo is no longer just foo, but mypack.foo. Without a module declaration, the compiler would need some other way to know that C:\\src\libs\mylib\mypack\foo.d is actually mypack.foo and not c.src.lib.mylib.mypack.foo (which again, is why it only uses the file name when no module declaration is given). I can't think of a better way to do it than putting it right there in the source file. Using a command line switch would not be scalable.
Mar 02 2016
On Wednesday, 2 March 2016 at 06:40:53 UTC, Sebastien Alaiwan wrote:Yeah, this is what I going to do: flat hierarchy, like one 'src' directory and one 'extra' directory in the project root, globally unique package names, passing -Isrc and -Iextra to the compiler, and ... module declaration directives. I was trying to avoid them, but maybe it's not possible.TBH I'm not sure what module directives do now that you mention them.
Mar 02 2016