digitalmars.D - Proposal for a standard for D library naming
- Gregor Richards (130/130) Sep 18 2006 (I'm reposting this, since my last post had some issues)
- Derek Parnell (11/11) Sep 18 2006 On Mon, 18 Sep 2006 21:10:46 -0700, Gregor Richards wrote:
- Gregor Richards (4/4) Sep 18 2006 Per Derek's suggestion, I've put this on wiki4d:
- =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= (20/38) Sep 19 2006 Not to rain on your parade, but Darwin libraries use:
- Gregor Richards (24/79) Sep 19 2006 I didn't mean to imply that these extensions were the only ones
- Gregor Richards (8/15) Sep 19 2006 Another reason for the D prefix is for bindings. Many bindings will
- =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= (3/9) Sep 19 2006 Yes, this is a very valid reason. I used a d suffix, for same reason.
- =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= (10/25) Sep 19 2006 No problem, just thought I'd add to the little list since
- Gregor Richards (9/36) Sep 19 2006 Yes, but if you've ever seen the naming convention, it would be easily
- =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= (6/12) Sep 19 2006 I think there is a Windows standard that libraries with d suffix
- Sean Kelly (8/22) Sep 19 2006 I didn't use Build until recently, and now I wouldn't do without it.
- =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= (5/10) Sep 19 2006 Build is like that cool power tool, that refuses to start when I try it.
- =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= (8/14) Sep 19 2006 Why does the fact that it uses D have to be reflected in the name ?
- =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= (11/16) Sep 19 2006 Just to check I understood this, would these be standard names:
- Gregor Richards (16/39) Sep 19 2006 That would be one means of dividing it.
- Sean Kelly (5/40) Sep 19 2006 For D import files I settled on using /usr/import instead of
- =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= (5/8) Sep 19 2006 Sure, but /usr/import is not in the Filesystem Hierarchy Standard (FHS).
- =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= (8/13) Sep 19 2006 I was just trying to make it more concrete, since those are
- Lars Ivar Igesund (6/7) Sep 19 2006 Sounds good.
- Gregor Richards (22/22) Sep 20 2006 Since there seems to be a lot of confusion as per the purpose of this
- Bruno Medeiros (6/14) Sep 23 2006 Why so? Why should a given library be splited across multiple so/dll
- Lars Ivar Igesund (7/21) Sep 23 2006 You totally missed the point, if the library _is_ split, then there shou...
- Bruno Medeiros (16/35) Sep 26 2006 I see.
- Gregor Richards (9/48) Sep 26 2006 No it doesn't - the rule is that a module is in the /most specific/
- Bruno Medeiros (28/79) Sep 28 2006 It doesn't have to in *your* rule. But I was not describing your rule, I...
- Gregor Richards (12/44) Sep 28 2006 That was my original thought.
- Bruno Medeiros (12/50) Oct 02 2006 I'm uncertain of that. Ultimately it depends on how one organizes his
- Frank Benoit (20/20) Sep 23 2006 I like the idea of having this naming schema.
- Derek Parnell (17/19) Sep 23 2006 In fact, are there any? I vaguely remember reading that D does not yet
- Gregor Richards (30/53) Sep 24 2006 This last paragraph makes me think that I've miscommunicated my concept
- Gregor Richards (19/82) Sep 24 2006 According to DMD's docs, it has an -fPIC flag. I see no compelling
- Derek Parnell (35/64) Sep 24 2006 Oh, I wouldn't blame my failure to understand on your ability to
- Gregor Richards (12/88) Sep 24 2006 Aha! This is a core misunderstanding. All of this library stuff is
- Derek Parnell (21/33) Sep 24 2006 Ok! So the example should be more like ...
- Gregor Richards (9/43) Sep 24 2006 Exactly! Using libraries isn't supposed to /replace/ using source or
- Miles (1/6) Sep 24 2006 I think that DMD looks for a/b/c.o (a\b\c.obj on windows).
- Derek Parnell (8/15) Sep 24 2006 Oops. Of course your right.
- Sean Kelly (4/8) Sep 24 2006 Shouldn't a module be recompiled if its object file is older than the
(I'm reposting this, since my last post had some issues) Since there aren't many libraries for D (that is, builds that produce .so files or the ilk), there's no standard on file naming for libraries. I'd like to interject a standard before it starts getting chaotic (as with every other language there is). Please respond with any opinions. A Modest Proposal for Standardization in Naming of D Libraries -------------------------------------------------------------- (by Gregor Richards) Outline ------- At present, there is to my knowledge no standard for naming D libraries (that is, .a, .lib, .so and .dll files). There's no immediate problem posed, as there is very little D code that forms libraries - but with any luck, the amount of code doing so will increase. For background on me and why I'm interested in this, both my job and several of my personal projects involve compilation and linking of software in UNIX environments, so I'm constantly exposed to inconsistent libraries. My experience with the D community has shown that few people at the moment seem to care about the creation of libraries, but I believe that desire will increase as popularity does. I consider the fact that there is no standard for naming to be a significant problem, as a lack of a clear naming convention will likely make the entire compilation process difficult (think of autoconf). I intend to outline in this document a naming convention for library files which will be easily parseable by both humans and software (such as [d]build). Note that this document does not currently target Windows .dll and .lib files, as I know nothing whatsoever about them. If anyone would like to step in and make modifications, feel free. Don't Break from Convention --------------------------- (This section applies only to UNIX) The UNIX convention for naming of libraries is: lib<name>.so.<major>.<minor>.<revision> with symlinks for each of: lib<name>.so.<major>.<minor> lib<name>.so.<major> and also a symlink of: lib<name>.so if the library has its headers installed. There is no compelling reason to break from this standard, so it is left as such in this document. For .a files, the file name is more simple: lib<name>.a Packages and Libraries ---------------------- Package names should be directly reflected in the file name of the library. Furthermore, the fact that this library is written in D should also be in the name of the library. To this end, the basic form is: libD.<package name>.<extension> Of course, packages are more complex than that, with subpackages, etc. The division becomes more complex for packages such as: a.* a.b.* a.c.* a.d.* Each library must only be inclusive of /modules/ in the package, not subpackages (though it may be either at the creator's whimsy). One way to comply would be to simply have a single file: libD.a.<extension> However, if one wanted to subdivide, it could also have several: libD.a.b.<extension> libD.a.c.<extension> libD.a.d.<extension> Now let's imagine the situation of the package 'a' having a module named 'foo'. a.foo doesn't belong in its own library (we've already determined that libraries correspond to packages), so it will go into its own file: libD.a.<extension> With this subdivision, we now have four files: libD.a.<extension> libD.a.b.<extension> libD.a.c.<extension> libD.a.d.<extension> Now, in all likelihood, one or all of the subpackages depend on modules in the superpackage. Luckily, this is trivial to detect in a build system. Since all of the imports will be iterated over, a list of packages to test for can be built that is guaranteed to capture everything (so long as the installation is complete). Version Hell ------------ (Or DLL Hell as it's affectionately known on Windows) Version Hell isn't too bad with UNIX-style .so files, so long as people comply to this simple standard: 1) Only increase the major version if you've made non-backwards- compatible changes. 2) Increase the minor version when you've added but not removed symbols. 3) Increase the revision for all other changes. 4) Don't worry about D ABI changes - they're well beyond the scope of library naming convention. I would recommend that a .DLL version of this spec incorporate some sort of versioning similar to UNIX. Build Tools ----------- With this setup, a build tool (such as build) has a very trivial task to generate a library name from an import. It simply needs to, starting with the most specific and expanding to the most general, check whether files corresponding to the name exist. So, if an import for a.b.c.d.e.f.g had been specified and the imported file was not in the included source, it would search (in this order): libD.a.b.c.d.e.f.g.so libD.a.b.c.d.e.f.g.a libD.a.b.c.d.e.f.so libD.a.b.c.d.e.f.a libD.a.b.c.d.e.so libD.a.b.c.d.e.a libD.a.b.c.d.so libD.a.b.c.d.a libD.a.b.c.so libD.a.b.c.a libD.a.b.so libD.a.b.a libD.a.so libD.a.a If none of them were found, it fails. Because each file carries its own dependencies, the build tool only needs to link against the one it found (and know how to read .add files) to get all of the symbols resolved. The generation of these files is a bit more complex, but any build tool would only need a tiny bit of input from the user (exactly where to subdivide) to be able to produce them. Questions? ---------- If anything is confusing here, please respond, and I will attempt to clarify. It's all quite clear in my head :)
Sep 18 2006
On Mon, 18 Sep 2006 21:10:46 -0700, Gregor Richards wrote: I was wondering if this sort of thing might be better on the D Wiki http://www.prowiki.org/wiki4d/wiki.cgi?FrontPage instead of this newsgroup. You could always put a link to the wiki entry on the newsgroup, but the wiki is a good place to host discussions. -- Derek (skype: derek.j.parnell) Melbourne, Australia "Down with mediocrity!" 19/09/2006 3:00:48 PM
Sep 18 2006
Per Derek's suggestion, I've put this on wiki4d: http://www.prowiki.org/wiki4d/wiki.cgi?LibraryNamingConvention Feel free to respond via either medium. - Gregor Richards
Sep 18 2006
Gregor Richards wrote:(This section applies only to UNIX) The UNIX convention for naming of libraries is: lib<name>.so.<major>.<minor>.<revision> with symlinks for each of: lib<name>.so.<major>.<minor> lib<name>.so.<major> and also a symlink of: lib<name>.so if the library has its headers installed. There is no compelling reason to break from this standard, so it is left as such in this document.Not to rain on your parade, but Darwin libraries use: lib<name>.<major>.<minor>.<revision>.dylib lib<name>.<major>.<minor>.dylib -> lib<name>.<major>.dylib -> lib<name>.dylib -> So you might want to make the "so" name flexible... ? Using the -L and -l flags works as expected, though. (With GDC that is, DMD uses those flags differently) Everything else applies. --anders SIDE NOTE: However, most Mac OS X libraries are instead packaged as "frameworks" which include *both* libs and includes. (Similar to how the Mac OS X applications are bundled up as "apps" which include both executables and resources) Frameworks are linked with the -framework flag, and the headers are automatically searched for in the system dirs. Don't think that the D libraries will ever be packaged as frameworks, so it only applies to libs like OpenGL and SDL.
Sep 19 2006
Anders F Björklund wrote:Gregor Richards wrote:I didn't mean to imply that these extensions were the only ones possible, I used them because they were the ones I knew about.(This section applies only to UNIX) The UNIX convention for naming of libraries is: lib<name>.so.<major>.<minor>.<revision> with symlinks for each of: lib<name>.so.<major>.<minor> lib<name>.so.<major> and also a symlink of: lib<name>.so if the library has its headers installed. There is no compelling reason to break from this standard, so it is left as such in this document.Not to rain on your parade, but Darwin libraries use: lib<name>.<major>.<minor>.<revision>.dylib lib<name>.<major>.<minor>.dylib -> lib<name>.<major>.dylib -> lib<name>.dylib -> So you might want to make the "so" name flexible... ? Using the -L and -l flags works as expected, though. (With GDC that is, DMD uses those flags differently) Everything else applies. --anders SIDE NOTE: However, most Mac OS X libraries are instead packaged as "frameworks" which include *both* libs and includes. (Similar to how the Mac OS X applications are bundled up as "apps" which include both executables and resources)Frameworks are linked with the -framework flag, and the headers are automatically searched for in the system dirs. Don't think that the D libraries will ever be packaged as frameworks, so it only applies to libs like OpenGL and SDL.But it /doesn't/ apply to libraries like OpenGL and SDL, they're not written in D, and would still need to be manually included.Why does the fact that it uses D have to be reflected in the name ?So that a unique namespace can be guaranteed - remember, these library names are to be not only human-understandable but machine-generatable.I don't see any C++ libraries advertising that they are using that, so I still need to remember to link them to libstdc++ or get errors.This is a good thing?Don't think it would be that bad to avoid prefixing them with "D.", and instead get similar link errors from Phobos (and the D runtime).But what's the advantage? The D prefix is more of a note that a D package name is coming than something to say "this is written in D" eg if you wrote a library in D that exports a C interface, you may be inclined to name it in a more traditional way, as that library is for use by C (or D), so the D build tool does not necessarily need to generate the paths in the same way.So I would rather continue to name them as lib<name>.a/<name>.lib...The entire idea behind my naming proposal is that it makes everything 100% computer and human generatable while adding very little confusion. Having things named lib<name>.<extension> makes associating packages and libraries difficult. Because C/++ don't have such a nice, structured package style, they don't have a naming convention for library files (there really couldn't be one), but I think that's a terrible reason to not use a naming convention for D. It would building, even against libraries, ridiculously easy.--anders- Gregor Richards
Sep 19 2006
Gregor Richards wrote:Anders F Björklund wrote: > I don't see any C++ libraries advertising that they are using that, > so I still need to remember to link them to libstdc++ or get errors. This is a good thing?Another reason for the D prefix is for bindings. Many bindings will have package names similar to their C library names - so similar in fact that the names could overlap if there was nothing to uniquely identify the D one. And if the names overlapped, it would need to break the naming convention, which would defeat the whole purpose of having a naming convention in the first place :) - Gregor Richards
Sep 19 2006
Gregor Richards wrote:Another reason for the D prefix is for bindings. Many bindings will have package names similar to their C library names - so similar in fact that the names could overlap if there was nothing to uniquely identify the D one. And if the names overlapped, it would need to break the naming convention, which would defeat the whole purpose of having a naming convention in the first place :)Yes, this is a very valid reason. I used a d suffix, for same reason. --anders
Sep 19 2006
Gregor Richards wrote:No problem, just thought I'd add to the little list since Mac OS X is identified by GDC as being a "Unix" platform...So you might want to make the "so" name flexible... ?I didn't mean to imply that these extensions were the only ones possible, I used them because they were the ones I knew about.Think I missed the "auto-generated" part of the proposal... :-) Currently I have the "wx" modules defined in a libwxd.a (or wxd.lib), partly since wx is already the C++ library and wxc are the C wrappers. Guess I could change that to libD.wx.a, but it looks somewhat strange.So I would rather continue to name them as lib<name>.a/<name>.lib...The entire idea behind my naming proposal is that it makes everything 100% computer and human generatable while adding very little confusion. Having things named lib<name>.<extension> makes associating packages and libraries difficult.Because C/++ don't have such a nice, structured package style, they don't have a naming convention for library files (there really couldn't be one), but I think that's a terrible reason to not use a naming convention for D. It would building, even against libraries, ridiculously easy.Actually I don't find it that hard, and e.g. the difference in which flags to use between DMD and GDC to be more "trouble"... --anders
Sep 19 2006
Anders F Björklund wrote:Gregor Richards wrote:Yes, but if you've ever seen the naming convention, it would be easily recognizeable. Also, I wouldn't be particularly opposed to moving the 'D' to the end, I just think that makes the suffixing a but confusing *shrugs*The entire idea behind my naming proposal is that it makes everything 100% computer and human generatable while adding very little confusion. Having things named lib<name>.<extension> makes associating packages and libraries difficult.Think I missed the "auto-generated" part of the proposal... :-) Currently I have the "wx" modules defined in a libwxd.a (or wxd.lib), partly since wx is already the C++ library and wxc are the C wrappers. Guess I could change that to libD.wx.a, but it looks somewhat strange.Remember, the entire purpose of this is for a build tool (say, build) to conform to a standard. Only the build tool needs to figure out the flags for DMD/GDC, the person compiling should get the libraries "for free."Because C/++ don't have such a nice, structured package style, they don't have a naming convention for library files (there really couldn't be one), but I think that's a terrible reason to not use a naming convention for D. It would building, even against libraries, ridiculously easy.Actually I don't find it that hard, and e.g. the difference in which flags to use between DMD and GDC to be more "trouble"...--anders- Gregor Richards
Sep 19 2006
Gregor Richards wrote:Also, I wouldn't be particularly opposed to moving the 'D' to the end, I just think that makes the suffixing a but confusing *shrugs*I think there is a Windows standard that libraries with d suffix means "debugging" versions, so a D prefix might actually be better.Remember, the entire purpose of this is for a build tool (say, build) to conform to a standard. Only the build tool needs to figure out the flags for DMD/GDC, the person compiling should get the libraries "for free."I haven't gotten Build to work for me, so I am still using Make. To make things easier, I am using "gcc" to link - even for DMD. --anders
Sep 19 2006
Anders F Björklund wrote:Gregor Richards wrote:This is true, and I agree.Also, I wouldn't be particularly opposed to moving the 'D' to the end, I just think that makes the suffixing a but confusing *shrugs*I think there is a Windows standard that libraries with d suffix means "debugging" versions, so a D prefix might actually be better.I didn't use Build until recently, and now I wouldn't do without it. Build makes using D similar to using Java but without the recompilation issues--ie. it "just works". If Build could be made to work with different library designs in a clean and simple manner it would feel like cheating. SeanRemember, the entire purpose of this is for a build tool (say, build) to conform to a standard. Only the build tool needs to figure out the flags for DMD/GDC, the person compiling should get the libraries "for free."I haven't gotten Build to work for me, so I am still using Make. To make things easier, I am using "gcc" to link - even for DMD.
Sep 19 2006
Sean Kelly wrote:I didn't use Build until recently, and now I wouldn't do without it. Build makes using D similar to using Java but without the recompilation issues--ie. it "just works". If Build could be made to work with different library designs in a clean and simple manner it would feel like cheating.Build is like that cool power tool, that refuses to start when I try it. Make is like that old rusty grandpa tool, that eventually does the job. I'll switch, eventually. Meanwhile, make will do the job just fine. ;-) --anders
Sep 19 2006
Gregor Richards wrote:Package names should be directly reflected in the file name of the library. Furthermore, the fact that this library is written in D should also be in the name of the library. To this end, the basic form is: libD.<package name>.<extension>Why does the fact that it uses D have to be reflected in the name ? I don't see any C++ libraries advertising that they are using that, so I still need to remember to link them to libstdc++ or get errors. Don't think it would be that bad to avoid prefixing them with "D.", and instead get similar link errors from Phobos (and the D runtime). So I would rather continue to name them as lib<name>.a/<name>.lib... --anders
Sep 19 2006
Gregor Richards wrote:A Modest Proposal for Standardization in Naming of D Libraries -------------------------------------------------------------- (by Gregor Richards)[...]If anything is confusing here, please respond, and I will attempt to clarify. It's all quite clear in my head :)Just to check I understood this, would these be standard names: /usr/include/d/sdl/* /usr/lib/libD.sdl.a /usr/include/d/gl/* /usr/lib/libD.gl.a /usr/include/d/wx/* /usr/lib/libD.wx.a For the modules under "sdl." and "gl." and "wx." , respectively ? --anders
Sep 19 2006
Anders F Björklund wrote:Gregor Richards wrote:That would be one means of dividing it. Mind you, I'm not inclusive of include files, they are not in the scope of the text. Furthermore, you could subdivide, if you wanted. For example, if you have this: /usr/include/d/sdl/amazingsdlplugina/* /usr/include/d/sdl/* You can do either one library: libD.sdl.<extension> Or more: libD.sdl.<extension> libD.sdl.amazingsdlplugina.<extension> , so long as a module can always be found in the most specific, extant library. - Gregor RichardsA Modest Proposal for Standardization in Naming of D Libraries -------------------------------------------------------------- (by Gregor Richards)[...]If anything is confusing here, please respond, and I will attempt to clarify. It's all quite clear in my head :)Just to check I understood this, would these be standard names: /usr/include/d/sdl/* /usr/lib/libD.sdl.a /usr/include/d/gl/* /usr/lib/libD.gl.a /usr/include/d/wx/* /usr/lib/libD.wx.a For the modules under "sdl." and "gl." and "wx." , respectively ? --anders
Sep 19 2006
Gregor Richards wrote:Anders F Björklund wrote:For D import files I settled on using /usr/import instead of /usr/include. It seemed to provide a cleaner separation than sticking C/C++/D files all in the same tree. SeanGregor Richards wrote:That would be one means of dividing it. Mind you, I'm not inclusive of include files, they are not in the scope of the text. Furthermore, you could subdivide, if you wanted. For example, if you have this: /usr/include/d/sdl/amazingsdlplugina/* /usr/include/d/sdl/*A Modest Proposal for Standardization in Naming of D Libraries -------------------------------------------------------------- (by Gregor Richards)[...]If anything is confusing here, please respond, and I will attempt to clarify. It's all quite clear in my head :)Just to check I understood this, would these be standard names: /usr/include/d/sdl/* /usr/lib/libD.sdl.a /usr/include/d/gl/* /usr/lib/libD.gl.a /usr/include/d/wx/* /usr/lib/libD.wx.a For the modules under "sdl." and "gl." and "wx." , respectively ? --anders
Sep 19 2006
Sean Kelly wrote:For D import files I settled on using /usr/import instead of /usr/include. It seemed to provide a cleaner separation than sticking C/C++/D files all in the same tree.Sure, but /usr/import is not in the Filesystem Hierarchy Standard (FHS). And I don't really have a problem with combining C/C++/D together. But if it is ever important, "import" can be a symlink over to "include/d". --anders
Sep 19 2006
Gregor Richards wrote:That would be one means of dividing it. Mind you, I'm not inclusive of include files, they are not in the scope of the text.I was just trying to make it more concrete, since those are the libraries that I am using for SDL, OpenGL and wxWidgets. In the RPM packages, the includes and libs travel together.Furthermore, you could subdivide, if you wanted.I really don't :-) A minor problem is that wxD is now two libraries (wxc/wxd), but I guess those objects could be stuffed into a libD.wx.a --anders
Sep 19 2006
Gregor Richards wrote:Please respond with any opinions.Sounds good. -- Lars Ivar Igesund blog at http://larsivi.net DSource & #D: larsivi
Sep 19 2006
Since there seems to be a lot of confusion as per the purpose of this library naming convention, I've added the following section: Purpose As there is much misunderstanding of the purpose of this library naming convention, it's outlined here: * The Library Naming Convention has no bearing whatsoever over the names or arrangement of source files. * The Library Naming Convention does not intend to replace build-style 'build everything from source with include paths' building, it intends to be a superior alternative. * The Library Naming Convention is a convention. That is, no one is required to conform to it, but (were it to be accepted) it would be inconveniencing not to. * The Library Naming Convention allows the source maintainer to choose what packages to make into libraries, with what level of division, etc. It has few rules over what is and what isn't in given libraries, as its purpose is the naming of libraries, not the content. o The only rule is that any module must be in the most specifically-named library corresponding to that module. That is, if you have libD.a.b.so.0 and libD.a.so.0, the module a.b.c should be in libD.a.b.so.0, not libD.a.so.0. Doing otherwise is fairly ridiculous anyway. http://www.prowiki.org/wiki4d/wiki.cgi?LibraryNamingConvention
Sep 20 2006
Gregor Richards wrote:o The only rule is that any module must be in the most specifically-named library corresponding to that module. That is, if you have libD.a.b.so.0 and libD.a.so.0, the module a.b.c should be in libD.a.b.so.0, not libD.a.so.0. Doing otherwise is fairly ridiculous anyway.Why so? Why should a given library be splited across multiple so/dll files according to package, instead of being compiled in just one file? -- Bruno Medeiros - MSc in CS/E student http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D
Sep 23 2006
Bruno Medeiros wrote:Gregor Richards wrote:You totally missed the point, if the library _is_ split, then there should be a standardized way to decide which part a module is in. -- Lars Ivar Igesund blog at http://larsivi.net DSource & #D: larsivio The only rule is that any module must be in the most specifically-named library corresponding to that module. That is, if you have libD.a.b.so.0 and libD.a.so.0, the module a.b.c should be in libD.a.b.so.0, not libD.a.so.0. Doing otherwise is fairly ridiculous anyway.Why so? Why should a given library be splited across multiple so/dll files according to package, instead of being compiled in just one file?
Sep 23 2006
Lars Ivar Igesund wrote:Bruno Medeiros wrote:I see. Well, But then I think it is a bad code "convention" to have subdivisions (packages) are not at the same level. That means that one should either have: a.* // a lib with 'a' and all subpackages and modules or have: // three related libs, each with it's subpackages and modules. a.b.* a.c.* a.d.* This means that in the second option, package 'a' cannot have sub-modules, only sub-packages. -- Bruno Medeiros - MSc in CS/E student http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#DGregor Richards wrote:You totally missed the point, if the library _is_ split, then there should be a standardized way to decide which part a module is in.o The only rule is that any module must be in the most specifically-named library corresponding to that module. That is, if you have libD.a.b.so.0 and libD.a.so.0, the module a.b.c should be in libD.a.b.so.0, not libD.a.so.0. Doing otherwise is fairly ridiculous anyway.Why so? Why should a given library be splited across multiple so/dll files according to package, instead of being compiled in just one file?
Sep 26 2006
Bruno Medeiros wrote:Lars Ivar Igesund wrote:No it doesn't - the rule is that a module is in the /most specific/ library. So, if you wanted to split out a.b, a.c and a.d, but had a module a.mod, there would be a library libD.a.{so,a,lib,whatever}, but it would only contain the module a.mod, no subpackages. A library is not assumed to be fully inclusive - it is assumed to be inclusive of everything that isn't in a more specifically-named library. - Gregor Richards PS: Yes, I thought this all through :)Bruno Medeiros wrote:I see. Well, But then I think it is a bad code "convention" to have subdivisions (packages) are not at the same level. That means that one should either have: a.* // a lib with 'a' and all subpackages and modules or have: // three related libs, each with it's subpackages and modules. a.b.* a.c.* a.d.* This means that in the second option, package 'a' cannot have sub-modules, only sub-packages.Gregor Richards wrote:You totally missed the point, if the library _is_ split, then there should be a standardized way to decide which part a module is in.o The only rule is that any module must be in the most specifically-named library corresponding to that module. That is, if you have libD.a.b.so.0 and libD.a.so.0, the module a.b.c should be in libD.a.b.so.0, not libD.a.so.0. Doing otherwise is fairly ridiculous anyway.Why so? Why should a given library be splited across multiple so/dll files according to package, instead of being compiled in just one file?
Sep 26 2006
Gregor Richards wrote:Bruno Medeiros wrote:It doesn't have to in *your* rule. But I was not describing your rule, I was describing mine. That is, I was saying that I would prefer a rule where a lib named "foo.{so|a|lib|whatever}" would mean that the lib "foo" contains all sub-packages of 'foo', not just the sub-modules. Since that lib is almost surely dependent on that packages. No, it is just not dependent one those packages, it is *made by* those packages. And what if those packages change between versions, can you imagine the confusion? Consider these two versions of the same lib: lib.foo.1.so lib.foo.bar.1.so lib.foo.gok.1.so lib.foo.2.so lib.foo.bar.2.so lib.foo.newgok.2.so lib.foo.zap.2.so Now see how they look sorted in a lib dir: lib.foo.1.so lib.foo.2.so lib.foo.bar.1.so lib.foo.bar.2.so lib.foo.gok.1.so lib.foo.newgok.2.so lib.foo.zap.2.so It's quite a mess. -- Bruno Medeiros - MSc in CS/E student http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#DLars Ivar Igesund wrote:No it doesn't - the rule is that a module is in the /most specific/ library. So, if you wanted to split out a.b, a.c and a.d, but had a module a.mod, there would be a library libD.a.{so,a,lib,whatever}, but it would only contain the module a.mod, no subpackages. A library is not assumed to be fully inclusive - it is assumed to be inclusive of everything that isn't in a more specifically-named library. - Gregor Richards PS: Yes, I thought this all through :)Bruno Medeiros wrote:I see. Well, But then I think it is a bad code "convention" to have subdivisions (packages) are not at the same level. That means that one should either have: a.* // a lib with 'a' and all subpackages and modules or have: // three related libs, each with it's subpackages and modules. a.b.* a.c.* a.d.* This means that in the second option, package 'a' cannot have sub-modules, only sub-packages.Gregor Richards wrote:You totally missed the point, if the library _is_ split, then there should be a standardized way to decide which part a module is in.o The only rule is that any module must be in the most specifically-named library corresponding to that module. That is, if you have libD.a.b.so.0 and libD.a.so.0, the module a.b.c should be in libD.a.b.so.0, not libD.a.so.0. Doing otherwise is fairly ridiculous anyway.Why so? Why should a given library be splited across multiple so/dll files according to package, instead of being compiled in just one file?
Sep 28 2006
Bruno Medeiros wrote:It doesn't have to in *your* rule. But I was not describing your rule, I was describing mine. That is, I was saying that I would prefer a rule where a lib named "foo.{so|a|lib|whatever}" would mean that the lib "foo" contains all sub-packages of 'foo', not just the sub-modules.That was my original thought. But that doesn't work.Since that lib is almost surely dependent on that packages.In the general case, module a.b.c is dependent on module a.c, NOT the reverse. Dependency within a tree usually goes up or across the tree, not down. Across is simply interlibrary dependency, up is where our point of contention is.No, it is just not dependent one those packages, it is *made by* those packages. And what if those packages change between versions, can you imagine the confusion? Consider these two versions of the same lib: lib.foo.1.so lib.foo.bar.1.so lib.foo.gok.1.soThis is not the standard way of naming UNIX library versions, but OK, continuing ...lib.foo.2.so lib.foo.bar.2.so lib.foo.newgok.2.so lib.foo.zap.2.so Now see how they look sorted in a lib dir: lib.foo.1.so lib.foo.2.so lib.foo.bar.1.so lib.foo.bar.2.so lib.foo.gok.1.so lib.foo.newgok.2.so lib.foo.zap.2.soMakes sense to me.It's quite a mess.I see no mess. - Gregor Richards
Sep 28 2006
Gregor Richards wrote:Bruno Medeiros wrote:Wouldn't work? Why not?It doesn't have to in *your* rule. But I was not describing your rule, I was describing mine. That is, I was saying that I would prefer a rule where a lib named "foo.{so|a|lib|whatever}" would mean that the lib "foo" contains all sub-packages of 'foo', not just the sub-modules.That was my original thought. But that doesn't work.I'm uncertain of that. Ultimately it depends on how one organizes his code structure and modules, but I think an organization where modules are usually dependent on submodules/subpackages(and not parent ones) is more logical. (In any case it doesn't affect my point about the lib file)Since that lib is almost surely dependent on that packages.In the general case, module a.b.c is dependent on module a.c, NOT the reverse. Dependency within a tree usually goes up or across the tree, not down. Across is simply interlibrary dependency, up is where our point of contention is.Well then, we've hit a subjective point(opinion), so not much to argue then. (I like the files as clean and tidy as possible) -- Bruno Medeiros - MSc in CS/E student http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#DNow see how they look sorted in a lib dir: lib.foo.1.so lib.foo.2.so lib.foo.bar.1.so lib.foo.bar.2.so lib.foo.gok.1.so lib.foo.newgok.2.so lib.foo.zap.2.soMakes sense to me.It's quite a mess.I see no mess. - Gregor Richards
Oct 02 2006
I like the idea of having this naming schema. I want to add the suggestion of using the unique package naming with reversed domain names. Like e.g. mango from dsource.org Package: org.dsource.mango in Version 1.2.3 and the resulting lib should be in /usr/lib/ lib: libD.org.dsource.mango.so.1.2.3 lnk: libD.org.dsource.mango.so.1.2 lnk: libD.org.dsource.mango.so.1 lnk: libD.org.dsource.mango.so I think the include directory hierarchy is close related to this. I would suggest to generate a header tree like this: in /usr/include/d/ dir: org.dsource.mango.1.2.3 lnk: org.dsource.mango.1.2 lnk: org.dsource.mango.1 lnk: org.dsource.mango src: org.dsource.mango.1.2.3/org/dsource/mango/.... Perhaps, in future, it is possible to pack and deploy the headers in some kind of jar file. Frank
Sep 23 2006
On Mon, 18 Sep 2006 21:10:46 -0700, Gregor Richards wrote:Since there aren't many libraries for D (that is, builds that produce .so files or the ilk)In fact, are there any? I vaguely remember reading that D does not yet support the creation of .so files. Is that still the case? Anyhow, let's break Gregor's proposal into two parts: (a) creating shared libraries (DLL in Windows, .so in Linux, ? in other *nix) (b) using shared libraries in the build process I will defer (a) until I know how to do it :-) With (b) I'm willing to add this feature to Build but with idea that failure to locate a library from a package name is not a failure to build. I'm not convinced that Gregor's idea that every (top-level) package must also be a library is a valid idea. I can imagine people still wanting to do static linking of object files. -- Derek Parnell Melbourne, Australia "Down with mediocrity!"
Sep 23 2006
Derek Parnell wrote:On Mon, 18 Sep 2006 21:10:46 -0700, Gregor Richards wrote:This last paragraph makes me think that I've miscommunicated my concept *urk* 1) Failure to locate an appropriately-named library is /not/ a failure, just a failure in that subsystem (if indeed the symbol can't be found anywhere, that'd be a failure). This would probably be a report-but-continue type warning. 2) My creation of libraries is slightly more flexible than what you're suggesting, but less flexible than what you'd like I think. Any /package/ can be a library (whether top-level or not), only making libraries of top-level packages is just a /default/. The only reason for making this restriction is, to not place a reasonable restriction on what modules are contained in a library makes the naming exponentially complex or impossible. Urk. As per making .so files, DMD can't do it but GDC can. For DMD to do it, it would need the equivalent of an -fPIC option, to produce position-independent code - the linking part is easy. Of course, since either can create .a files, its implementable so long as DMD users are willing to surrender the ability to make .so files until it's capable. If build did this all in its automatic processing, the source maintainer wouldn't need to care whether the compiler being used is capable of producing libraries of any sort. (Actually, DMD may be producing position independent code already for all I know, in which case it's perfectly capable, just need -L-shared - unfortunately, this is hard to test, since much software will "just work" even if it's not made position independent) Furthermore, this proposal is not shared-library specific, it just shows its greatest benefit there (since the reliability in naming is carried through every conceivable stage of making/running a program). - Gregor RichardsSince there aren't many libraries for D (that is, builds that produce .so files or the ilk)In fact, are there any? I vaguely remember reading that D does not yet support the creation of .so files. Is that still the case? Anyhow, let's break Gregor's proposal into two parts: (a) creating shared libraries (DLL in Windows, .so in Linux, ? in other *nix) (b) using shared libraries in the build process I will defer (a) until I know how to do it :-) With (b) I'm willing to add this feature to Build but with idea that failure to locate a library from a package name is not a failure to build. I'm not convinced that Gregor's idea that every (top-level) package must also be a library is a valid idea. I can imagine people still wanting to do static linking of object files.
Sep 24 2006
Gregor Richards wrote:Derek Parnell wrote:According to DMD's docs, it has an -fPIC flag. I see no compelling reason why it shouldn't be able to create .so files, given that. To make a .so file (just a guess): dmd <source files> -fPIC -L-shared -L-Wl-soname=libfoo.so.0 -oflibfoo.so.0.0.0 (In case you don't get the discrepency in naming, it's all about how ELF .so files are versioned: the first number after .so is essentially the ABI version, so it should only change when non-backwards-compatible changes are made. The name that programs keep in them has one number. The next is generally used for backwards-compatible ABI changes, the next for other changes) If you are willing to implement this in build, I'd be glad to extend my help, especially on getting all the UNIX stuff named/installed as would be traditional for a UNIX system. (And of course I've also offered to make the changes myself, but I'm under the distinctive impression that you'd prefer to make big changes like that yourself - perfectly understandable) - Gregor RichardsOn Mon, 18 Sep 2006 21:10:46 -0700, Gregor Richards wrote:This last paragraph makes me think that I've miscommunicated my concept *urk* 1) Failure to locate an appropriately-named library is /not/ a failure, just a failure in that subsystem (if indeed the symbol can't be found anywhere, that'd be a failure). This would probably be a report-but-continue type warning. 2) My creation of libraries is slightly more flexible than what you're suggesting, but less flexible than what you'd like I think. Any /package/ can be a library (whether top-level or not), only making libraries of top-level packages is just a /default/. The only reason for making this restriction is, to not place a reasonable restriction on what modules are contained in a library makes the naming exponentially complex or impossible. Urk. As per making .so files, DMD can't do it but GDC can. For DMD to do it, it would need the equivalent of an -fPIC option, to produce position-independent code - the linking part is easy. Of course, since either can create .a files, its implementable so long as DMD users are willing to surrender the ability to make .so files until it's capable. If build did this all in its automatic processing, the source maintainer wouldn't need to care whether the compiler being used is capable of producing libraries of any sort. (Actually, DMD may be producing position independent code already for all I know, in which case it's perfectly capable, just need -L-shared - unfortunately, this is hard to test, since much software will "just work" even if it's not made position independent) Furthermore, this proposal is not shared-library specific, it just shows its greatest benefit there (since the reliability in naming is carried through every conceivable stage of making/running a program). - Gregor RichardsSince there aren't many libraries for D (that is, builds that produce .so files or the ilk)In fact, are there any? I vaguely remember reading that D does not yet support the creation of .so files. Is that still the case? Anyhow, let's break Gregor's proposal into two parts: (a) creating shared libraries (DLL in Windows, .so in Linux, ? in other *nix) (b) using shared libraries in the build process I will defer (a) until I know how to do it :-) With (b) I'm willing to add this feature to Build but with idea that failure to locate a library from a package name is not a failure to build. I'm not convinced that Gregor's idea that every (top-level) package must also be a library is a valid idea. I can imagine people still wanting to do static linking of object files.
Sep 24 2006
On Sun, 24 Sep 2006 02:24:06 -0700, Gregor Richards wrote:Derek Parnell wrote:Oh, I wouldn't blame my failure to understand on your ability to communicate ;-)On Mon, 18 Sep 2006 21:10:46 -0700, Gregor Richards wrote:This last paragraph makes me think that I've miscommunicated my concept *urk*Since there aren't many libraries for D (that is, builds that produce .so files or the ilk)In fact, are there any? I vaguely remember reading that D does not yet support the creation of .so files. Is that still the case? Anyhow, let's break Gregor's proposal into two parts: (a) creating shared libraries (DLL in Windows, .so in Linux, ? in other *nix) (b) using shared libraries in the build process I will defer (a) until I know how to do it :-) With (b) I'm willing to add this feature to Build but with idea that failure to locate a library from a package name is not a failure to build. I'm not convinced that Gregor's idea that every (top-level) package must also be a library is a valid idea. I can imagine people still wanting to do static linking of object files.1) Failure to locate an appropriately-named library is /not/ a failure, just a failure in that subsystem (if indeed the symbol can't be found anywhere, that'd be a failure). This would probably be a report-but-continue type warning.You still seem to be saying that the *only* place that module ought to reside in is in libraries. Whereas I maintain that they *could* also reside in object files (.o/.obj) files. So if Build doesn't find a library that matches the import name, why can't it use the object file that maps to the import name - and if the object file doesn't exist then why can't I have Build compile the module's source to create it? The fact that the library doesn't exist is not a failure then - or must there be a switch to say that only libraries are acceptable for this build process? Example: import a.b.c; Looks for libD.a.b.so libD.a.b.a libD.a.so libD.a.a a.b.c.o and places the first one it comes across onto the linker command line. If it finds none of these, it will compile a.b.c.d to form an object file. And then looks for a.b.c.di a.b.c.d to see if the source file is more recent than the 'object' file found earlier. So a further complication is what should Build do if the module source files are more recent than the matching library? Currently, it recompiles the source files and uses the resulting object files in the linkage. -- Derek (skype: derek.j.parnell) Melbourne, Australia "Down with mediocrity!" 25/09/2006 9:41:49 AM
Sep 24 2006
Derek Parnell wrote:On Sun, 24 Sep 2006 02:24:06 -0700, Gregor Richards wrote:Aha! This is a core misunderstanding. All of this library stuff is supposed to come well after failing to find it in an object file - the order in my head is: source files, object files, specified libraries, automatically-discovered libraries. That is, it should FIRST look for source, then objects, and only if failing in both of those should it look at libraries - libraries are solely for things that are NOT included with the source!Derek Parnell wrote:Oh, I wouldn't blame my failure to understand on your ability to communicate ;-)On Mon, 18 Sep 2006 21:10:46 -0700, Gregor Richards wrote:This last paragraph makes me think that I've miscommunicated my concept *urk*Since there aren't many libraries for D (that is, builds that produce .so files or the ilk)In fact, are there any? I vaguely remember reading that D does not yet support the creation of .so files. Is that still the case? Anyhow, let's break Gregor's proposal into two parts: (a) creating shared libraries (DLL in Windows, .so in Linux, ? in other *nix) (b) using shared libraries in the build process I will defer (a) until I know how to do it :-) With (b) I'm willing to add this feature to Build but with idea that failure to locate a library from a package name is not a failure to build. I'm not convinced that Gregor's idea that every (top-level) package must also be a library is a valid idea. I can imagine people still wanting to do static linking of object files.1) Failure to locate an appropriately-named library is /not/ a failure, just a failure in that subsystem (if indeed the symbol can't be found anywhere, that'd be a failure). This would probably be a report-but-continue type warning.You still seem to be saying that the *only* place that module ought to reside in is in libraries.Whereas I maintain that they *could* also reside in object files (.o/.obj) files. So if Build doesn't find a library that matches the import name, why can't it use the object file that maps to the import name - and if the object file doesn't exist then why can't I have Build compile the module's source to create it? The fact that the library doesn't exist is not a failure then - or must there be a switch to say that only libraries are acceptable for this build process? Example: import a.b.c; Looks for libD.a.b.so libD.a.b.a libD.a.so libD.a.a a.b.c.o and places the first one it comes across onto the linker command line. If it finds none of these, it will compile a.b.c.d to form an object file. And then looks for a.b.c.di a.b.c.d to see if the source file is more recent than the 'object' file found earlier. So a further complication is what should Build do if the module source files are more recent than the matching library? Currently, it recompiles the source files and uses the resulting object files in the linkage.The timing on source/object files has no reason to change - the only issue is with conflicts, and in my opinion source should /always/ trump libraries on conflicts. - Gregor Richards
Sep 24 2006
On Sun, 24 Sep 2006 17:14:06 -0700, Gregor Richards wrote:Derek Parnell wrote:Ok! So the example should be more like ... Example: import a.b.c; Looks for a.b.c.o libD.a.b.so libD.a.b.a libD.a.so libD.a.a and places the first one it comes across onto the linker command line. If it finds none of these, or finds that the source is more recent, it will compile a.b.c.d to form an object file. -- Derek (skype: derek.j.parnell) Melbourne, Australia "Down with mediocrity!" 25/09/2006 10:17:17 AMYou still seem to be saying that the *only* place that module ought to reside in is in libraries.Aha! This is a core misunderstanding. All of this library stuff is supposed to come well after failing to find it in an object file - the order in my head is: source files, object files, specified libraries, automatically-discovered libraries. That is, it should FIRST look for source, then objects, and only if failing in both of those should it look at libraries - libraries are solely for things that are NOT included with the source!
Sep 24 2006
Derek Parnell wrote:On Sun, 24 Sep 2006 17:14:06 -0700, Gregor Richards wrote:Exactly! Using libraries isn't supposed to /replace/ using source or already-compiled object files, it's supposed to be an adjunct - in many cases, it's just inconvenient to use object files or source, and of course there's the possibility of proprietary software in which a library is clearly the superior method. Obviously, if you included the source by some means, you intend to use them - otherwise, libraries are probably more useful. - Gregor RichardsDerek Parnell wrote:Ok! So the example should be more like ... Example: import a.b.c; Looks for a.b.c.o libD.a.b.so libD.a.b.a libD.a.so libD.a.a and places the first one it comes across onto the linker command line. If it finds none of these, or finds that the source is more recent, it will compile a.b.c.d to form an object file.You still seem to be saying that the *only* place that module ought to reside in is in libraries.Aha! This is a core misunderstanding. All of this library stuff is supposed to come well after failing to find it in an object file - the order in my head is: source files, object files, specified libraries, automatically-discovered libraries. That is, it should FIRST look for source, then objects, and only if failing in both of those should it look at libraries - libraries are solely for things that are NOT included with the source!
Sep 24 2006
Example: import a.b.c; Looks for a.b.c.oI think that DMD looks for a/b/c.o (a\b\c.obj on windows).
Sep 24 2006
On Sun, 24 Sep 2006 23:17:22 -0300, Miles wrote:Oops. Of course your right. -- Derek (skype: derek.j.parnell) Melbourne, Australia "Down with mediocrity!" 25/09/2006 12:21:45 PMExample: import a.b.c; Looks for a.b.c.oI think that DMD looks for a/b/c.o (a\b\c.obj on windows).
Sep 24 2006
Derek Parnell wrote:So a further complication is what should Build do if the module source files are more recent than the matching library? Currently, it recompiles the source files and uses the resulting object files in the linkage.Shouldn't a module be recompiled if its object file is older than the module or any of its dependencies? If -inline is specified at any rate. Sean
Sep 24 2006