## digitalmars.D - Suggestion: Object filenames should be fully-qualified module names

Kirk McDonald <kirklin.mcdonald gmail.com> writes:
I originally heard this idea proposed by Gregor Richards in #d, and I
think it should become DMD's default behavior. If it is not the default
behavior, then it should at least be available as an option.

Perhaps my biggest grievance with both the DMD and GDC compilers is
their handling of object files. DMD's default behavior is to dump all
object files into the current directory. If the -od option is specified,
the object files will be placed into the specified directory instead. If
-op is specified, the object files are placed alongside the original
source files.

The default behavior and using -od on its own both fail if any two
source files in the project have the same name, even if they are in
different packages. Using -op by itself is unappealing for two reasons:

1) It is not unreasonable to expect a system to place libraries in
directories to which the user does not have write access. Placing object
files alongside the source files would therefore fail.

2) It pollutes the source directories with object files. I much prefer
keeping my object files somewhere to the side, in a designated "build"
directory. This makes keeping projects in version control much easier,
as I can simply exclude the one directory to keep object files out of
version control.

Specifying both -op and -od causes things to get a little more
interesting. Take the following:

test.d
testpkg
test.d

// test.d
module test;
import testpkg.test : foo;
void main() {
foo();
}

// testpkg/test.d
module testpkg.test;
import std.stdio : writefln;
void foo() { writefln("foo"); }

If we compile with this:

$dmd test.d testpkg/test.d -op -odbuild The "build" directory has the following structure: build test.obj testpkg test.obj This is all and well. However, if we compile like this:$ dmd test.d /path/to/testpkg/test.d -op -odbuild

Then DMD doesn't know what to do, and it places testpkg/test.obj
alongside the source file. (More specifically, the full path is "joined"
to the path specified by -od, which works out to just being the full path.)

This ambiguity can be disposed of if an object file's name is its
fully-qualified module name. If this were true, then we could just say

$dmd test.d testpkg/test.d -odbuild and the result would be the build directory looking like this: build test.obj testpkg.test.obj I find this very clean and simple. Since the compiler fails anyway if two modules have the same name, there should not ever be overlaps in object file names with this scheme. The -op option could probably be safely deprecated. As someone pointed out in #d, this would fail on NTFS if the module's fully-qualified name exceeds 255 characters. Though I cannot recall ever using a module whose name even approached that limit, this should be solved in most cases by truncating the filename at the start. (Hopefully, the last 255 characters are unique.) If the object file would fail to be unique even then, it can probably be safely declared the coder's fault for using a stupid naming scheme. -- Kirk McDonald Pyd: Wrapping Python with D http://pyd.dsource.org  Jan 20 2007 Johan Granberg <lijat.meREM OVE.gmail.com> writes: Kirk McDonald wrote: I originally heard this idea proposed by Gregor Richards in #d, and I think it should become DMD's default behavior. If it is not the default behavior, then it should at least be available as an option. Perhaps my biggest grievance with both the DMD and GDC compilers is their handling of object files. DMD's default behavior is to dump all object files into the current directory. If the -od option is specified, the object files will be placed into the specified directory instead. If -op is specified, the object files are placed alongside the original source files. The default behavior and using -od on its own both fail if any two source files in the project have the same name, even if they are in different packages. Using -op by itself is unappealing for two reasons: 1) It is not unreasonable to expect a system to place libraries in directories to which the user does not have write access. Placing object files alongside the source files would therefore fail. 2) It pollutes the source directories with object files. I much prefer keeping my object files somewhere to the side, in a designated "build" directory. This makes keeping projects in version control much easier, as I can simply exclude the one directory to keep object files out of version control. Specifying both -op and -od causes things to get a little more interesting. Take the following: test.d testpkg test.d // test.d module test; import testpkg.test : foo; void main() { foo(); } // testpkg/test.d module testpkg.test; import std.stdio : writefln; void foo() { writefln("foo"); } If we compile with this:$ dmd test.d testpkg/test.d -op -odbuild

The "build" directory has the following structure:

build
test.obj
testpkg
test.obj

This is all and well. However, if we compile like this:

$dmd test.d /path/to/testpkg/test.d -op -odbuild Then DMD doesn't know what to do, and it places testpkg/test.obj alongside the source file. (More specifically, the full path is "joined" to the path specified by -od, which works out to just being the full path.) This ambiguity can be disposed of if an object file's name is its fully-qualified module name. If this were true, then we could just say$ dmd test.d testpkg/test.d -odbuild

and the result would be the build directory looking like this:

build
test.obj
testpkg.test.obj

I find this very clean and simple. Since the compiler fails anyway if
two modules have the same name, there should not ever be overlaps in
object file names with this scheme. The -op option could probably be
safely deprecated.

As someone pointed out in #d, this would fail on NTFS if the module's
fully-qualified name exceeds 255 characters. Though I cannot recall ever
using a module whose name even approached that limit, this should be
solved in most cases by truncating the filename at the start.
(Hopefully, the last 255 characters are unique.) If the object file
would fail to be unique even then, it can probably be safely declared
the coder's fault for using a stupid naming scheme.

I very much agree that something must be done and this solution is as good
as any.

Jan 20 2007
"Rioshin an'Harthen" <rharth75 hotmail.com> writes:
"Kirk McDonald" <kirklin.mcdonald gmail.com> wrote:
This ambiguity can be disposed of if an object file's name is its
fully-qualified module name. If this were true, then we could just say

$dmd test.d testpkg/test.d -odbuild and the result would be the build directory looking like this: build test.obj testpkg.test.obj I find this very clean and simple. Since the compiler fails anyway if two modules have the same name, there should not ever be overlaps in object file names with this scheme. The -op option could probably be safely deprecated. As someone pointed out in #d, this would fail on NTFS if the module's fully-qualified name exceeds 255 characters. Though I cannot recall ever using a module whose name even approached that limit, this should be solved in most cases by truncating the filename at the start. (Hopefully, the last 255 characters are unique.) If the object file would fail to be unique even then, it can probably be safely declared the coder's fault for using a stupid naming scheme. Votes++  Jan 20 2007 "Chris Miller" <chris dprogramming.com> writes: On Sat, 20 Jan 2007 07:17:45 -0500, Kirk McDonald <kirklin.mcdonald gmail.com> wrote: [snip] This ambiguity can be disposed of if an object file's name is its fully-qualified module name. If this were true, then we could just say$ dmd test.d testpkg/test.d -odbuild

and the result would be the build directory looking like this:

build
test.obj
testpkg.test.obj

I find this very clean and simple. Since the compiler fails anyway if
two modules have the same name, there should not ever be overlaps in
object file names with this scheme. The -op option could probably be
safely deprecated.

Agreed, this is a much better way of handling it.

As someone pointed out in #d, this would fail on NTFS if the module's
fully-qualified name exceeds 255 characters. Though I cannot recall ever
using a module whose name even approached that limit, this should be
solved in most cases by truncating the filename at the start.
(Hopefully, the last 255 characters are unique.) If the object file
would fail to be unique even then, it can probably be safely declared
the coder's fault for using a stupid naming scheme.

I don't think it's an actual limitation of NTFS, but rather most programs
use buffers for file paths of MAX_PATH characters (260), and to use more
requires special handling (using some path prefix). I believe the linker
would be one of those programs that can't handle larger paths.

As a compromise, perhaps N characters from both the head and tail of the
fully-qualified module name could be used, such as 75 chars from the
beginning and 75 from the end, and then it still leaves almost half the
range (MAX_PATH chars) available for storing the path that leads up to the
module.

For example:
C:\dmd\src\phobos\std\D\is\a\systems\programming\language\Its\focus\is\on\combining\the\power\and\high\performance\of\C\and\Cpp\with\the\programmer\productivity\of\modern\languages\like\Ruby\and\Python\Special\attention\is\given\to\the\needs\of\quality\assurance\documentation\management\portability\and\reliability.d
Would shrink down to:
std.D.is.a.systems.programming.language.Its.focus.is.on.combining.the.powers.of.quality.assurance.documentation.management.portability.and.reliability.obj
and still have room to store the directory it's saved in (e.g.
C:\dmd\src\phobos on the front).
;)

Jan 20 2007
Chris Nicholson-Sauls <ibisbasenji gmail.com> writes:
Kirk McDonald wrote:
I originally heard this idea proposed by Gregor Richards in #d, and I
think it should become DMD's default behavior. If it is not the default
behavior, then it should at least be available as an option.

Perhaps my biggest grievance with both the DMD and GDC compilers is
their handling of object files. DMD's default behavior is to dump all
object files into the current directory. If the -od option is specified,
the object files will be placed into the specified directory instead. If
-op is specified, the object files are placed alongside the original
source files.

The default behavior and using -od on its own both fail if any two
source files in the project have the same name, even if they are in
different packages. Using -op by itself is unappealing for two reasons:

1) It is not unreasonable to expect a system to place libraries in
directories to which the user does not have write access. Placing object
files alongside the source files would therefore fail.

2) It pollutes the source directories with object files. I much prefer
keeping my object files somewhere to the side, in a designated "build"
directory. This makes keeping projects in version control much easier,
as I can simply exclude the one directory to keep object files out of
version control.

Specifying both -op and -od causes things to get a little more
interesting. Take the following:

test.d
testpkg
test.d

// test.d
module test;
import testpkg.test : foo;
void main() {
foo();
}

// testpkg/test.d
module testpkg.test;
import std.stdio : writefln;
void foo() { writefln("foo"); }

If we compile with this:

$dmd test.d testpkg/test.d -op -odbuild The "build" directory has the following structure: build test.obj testpkg test.obj This is all and well. However, if we compile like this:$ dmd test.d /path/to/testpkg/test.d -op -odbuild

Then DMD doesn't know what to do, and it places testpkg/test.obj
alongside the source file. (More specifically, the full path is "joined"
to the path specified by -od, which works out to just being the full path.)

This ambiguity can be disposed of if an object file's name is its
fully-qualified module name. If this were true, then we could just say

$dmd test.d testpkg/test.d -odbuild and the result would be the build directory looking like this: build test.obj testpkg.test.obj I find this very clean and simple. Since the compiler fails anyway if two modules have the same name, there should not ever be overlaps in object file names with this scheme. The -op option could probably be safely deprecated. As someone pointed out in #d, this would fail on NTFS if the module's fully-qualified name exceeds 255 characters. Though I cannot recall ever using a module whose name even approached that limit, this should be solved in most cases by truncating the filename at the start. (Hopefully, the last 255 characters are unique.) If the object file would fail to be unique even then, it can probably be safely declared the coder's fault for using a stupid naming scheme. Personally, I would love it. (I already use a little utility to do this to DDoc output, changing pkg/Module.html to pkg.Module.html.) -- Chris Nicholson-Sauls  Jan 20 2007 =?ISO-8859-1?Q?Lu=EDs_Marques?= <luismarques+spam gmail.com> writes: Kirk McDonald wrote: Perhaps my biggest grievance with both the DMD and GDC compilers is their handling of object files. DMD's default behavior is to dump all object files into the current directory. If the -od option is specified, the object files will be placed into the specified directory instead. If -op is specified, the object files are placed alongside the original source files. I have found the default behavior quite senseless too. A big approval to changing this from me! :) -- Luís Marques  Jan 20 2007 Gregor Richards <Richards codu.org> writes: Kirk McDonald wrote: I originally heard this idea proposed by Gregor Richards in #d, and I think it should become DMD's default behavior. If it is not the default behavior, then it should at least be available as an option. Bizarrely, I find myself agreeing with this ;) - Gregor Richards  Jan 20 2007 Bradley Smith <digitalmars-com baysmith.com> writes: Have you seen issue 492 (http://d.puremagic.com/issues/show_bug.cgi?id=492)? Kirk McDonald wrote: I originally heard this idea proposed by Gregor Richards in #d, and I think it should become DMD's default behavior. If it is not the default behavior, then it should at least be available as an option. Perhaps my biggest grievance with both the DMD and GDC compilers is their handling of object files. DMD's default behavior is to dump all object files into the current directory. If the -od option is specified, the object files will be placed into the specified directory instead. If -op is specified, the object files are placed alongside the original source files. The default behavior and using -od on its own both fail if any two source files in the project have the same name, even if they are in different packages. Using -op by itself is unappealing for two reasons: 1) It is not unreasonable to expect a system to place libraries in directories to which the user does not have write access. Placing object files alongside the source files would therefore fail. 2) It pollutes the source directories with object files. I much prefer keeping my object files somewhere to the side, in a designated "build" directory. This makes keeping projects in version control much easier, as I can simply exclude the one directory to keep object files out of version control. Specifying both -op and -od causes things to get a little more interesting. Take the following: test.d testpkg test.d // test.d module test; import testpkg.test : foo; void main() { foo(); } // testpkg/test.d module testpkg.test; import std.stdio : writefln; void foo() { writefln("foo"); } If we compile with this:$ dmd test.d testpkg/test.d -op -odbuild

The "build" directory has the following structure:

build
test.obj
testpkg
test.obj

This is all and well. However, if we compile like this:

$dmd test.d /path/to/testpkg/test.d -op -odbuild Then DMD doesn't know what to do, and it places testpkg/test.obj alongside the source file. (More specifically, the full path is "joined" to the path specified by -od, which works out to just being the full path.) This ambiguity can be disposed of if an object file's name is its fully-qualified module name. If this were true, then we could just say$ dmd test.d testpkg/test.d -odbuild

and the result would be the build directory looking like this:

build
test.obj
testpkg.test.obj

I find this very clean and simple. Since the compiler fails anyway if
two modules have the same name, there should not ever be overlaps in
object file names with this scheme. The -op option could probably be
safely deprecated.

As someone pointed out in #d, this would fail on NTFS if the module's
fully-qualified name exceeds 255 characters. Though I cannot recall ever
using a module whose name even approached that limit, this should be
solved in most cases by truncating the filename at the start.
(Hopefully, the last 255 characters are unique.) If the object file
would fail to be unique even then, it can probably be safely declared
the coder's fault for using a stupid naming scheme.


Jan 20 2007
Kirk McDonald <kirklin.mcdonald gmail.com> writes:
Bradley Smith wrote:
Have you seen issue 492
(http://d.puremagic.com/issues/show_bug.cgi?id=492)?

No, I hadn't. It's basically the same suggestion, only with using path
seperators instead of dots. This might have the advantage of not running
into any theoretical length limits on filenames. Still, being able to
(sanely) dump object files into a flat directory is also nice.

Either one of these would be good.

--
Kirk McDonald
Pyd: Wrapping Python with D
http://pyd.dsource.org

Jan 20 2007
Gregor Richards <Richards codu.org> writes:
Kirk McDonald wrote:
Have you seen issue 492
(http://d.puremagic.com/issues/show_bug.cgi?id=492)?

No, I hadn't. It's basically the same suggestion, only with using path
seperators instead of dots. This might have the advantage of not running
into any theoretical length limits on filenames. Still, being able to
(sanely) dump object files into a flat directory is also nice.

Either one of these would be good.

Part of why I like my suggestion of using fully-qualified dot-names is
how .a files work on UNIX. It's a directory-free archive. So you'll have
foo.o and foo.o and foo.o :-P

Yes, it can work like this, but it makes extracting .a files a huge PITA.

- Gregor Richards

Jan 20 2007
Gregor Richards <Richards codu.org> writes:
Gregor Richards wrote:
Kirk McDonald wrote:
Have you seen issue 492
(http://d.puremagic.com/issues/show_bug.cgi?id=492)?

No, I hadn't. It's basically the same suggestion, only with using path
running into any theoretical length limits on filenames. Still, being
able to (sanely) dump object files into a flat directory is also nice.

Either one of these would be good.

Part of why I like my suggestion of using fully-qualified dot-names is
how .a files work on UNIX. It's a directory-free archive. So you'll have
foo.o and foo.o and foo.o :-P

Yes, it can work like this, but it makes extracting .a files a huge PITA.

- Gregor Richards

(AFAIK, .lib files can't be extracted at all, so this doesn't translate
to DMD/Windows at all)

- Gregor Richards

Jan 20 2007
just jeff <jeffrparsons optusnet.com.au> writes:
Votes++ for either this or support for building object files to a mirror
of the source tree. Both sound like good solutions to me. :)

Jan 20 2007
Kirk McDonald <kirklin.mcdonald gmail.com> writes:
just jeff wrote:
Votes++ for either this or support for building object files to a mirror
of the source tree. Both sound like good solutions to me. :)

There's a problem with mirroring the source tree. In effect, this is
what combining -op and -od does now. The problem is: What if you have
source files from outside your own project's directory? What if you have
to go all the way to the root before finding a directory that they have
in common? There's no particularly sensible way to solve this. Basing
object file names on the module names, rather than however the source
files are organized, is the only solution I can think of.

--
Kirk McDonald
Pyd: Wrapping Python with D
http://pyd.dsource.org

Jan 20 2007
Bill Baxter <dnewsgroup billbaxter.com> writes:
Yes please!
I've spent the last week sending mail back and forth with some poor soul
who was trying to use my Luigi GUI lib, but couldn't seem to link it for
some reason.

Finally today I tracked it down to the only difference between my
working build file and his not working one being his use of the -od
flag.  I didn't know what -od was for, but after figuring it out, I
remembered this thread.  -od flag and .obj files were silently being
overwritten with other obj files of the same name.

EVIL!

Isn't Kirk's proposal also the way Java compilers do it?  I seem to
recall a bunch of foo.bar.baz.class files appearing last time I played
with Java.

--bb

Kirk McDonald wrote:
I originally heard this idea proposed by Gregor Richards in #d, and I
think it should become DMD's default behavior. If it is not the default
behavior, then it should at least be available as an option.

Perhaps my biggest grievance with both the DMD and GDC compilers is
their handling of object files. DMD's default behavior is to dump all
object files into the current directory. If the -od option is specified,
the object files will be placed into the specified directory instead. If
-op is specified, the object files are placed alongside the original
source files.

The default behavior and using -od on its own both fail if any two
source files in the project have the same name, even if they are in
different packages. Using -op by itself is unappealing for two reasons:

1) It is not unreasonable to expect a system to place libraries in
directories to which the user does not have write access. Placing object
files alongside the source files would therefore fail.

2) It pollutes the source directories with object files. I much prefer
keeping my object files somewhere to the side, in a designated "build"
directory. This makes keeping projects in version control much easier,
as I can simply exclude the one directory to keep object files out of
version control.

Specifying both -op and -od causes things to get a little more
interesting. Take the following:

test.d
testpkg
test.d

// test.d
module test;
import testpkg.test : foo;
void main() {
foo();
}

// testpkg/test.d
module testpkg.test;
import std.stdio : writefln;
void foo() { writefln("foo"); }

If we compile with this:

$dmd test.d testpkg/test.d -op -odbuild The "build" directory has the following structure: build test.obj testpkg test.obj This is all and well. However, if we compile like this:$ dmd test.d /path/to/testpkg/test.d -op -odbuild

Then DMD doesn't know what to do, and it places testpkg/test.obj
alongside the source file. (More specifically, the full path is "joined"
to the path specified by -od, which works out to just being the full path.)

This ambiguity can be disposed of if an object file's name is its
fully-qualified module name. If this were true, then we could just say

$dmd test.d testpkg/test.d -odbuild and the result would be the build directory looking like this: build test.obj testpkg.test.obj I find this very clean and simple. Since the compiler fails anyway if two modules have the same name, there should not ever be overlaps in object file names with this scheme. The -op option could probably be safely deprecated. As someone pointed out in #d, this would fail on NTFS if the module's fully-qualified name exceeds 255 characters. Though I cannot recall ever using a module whose name even approached that limit, this should be solved in most cases by truncating the filename at the start. (Hopefully, the last 255 characters are unique.) If the object file would fail to be unique even then, it can probably be safely declared the coder's fault for using a stupid naming scheme.  Jan 25 2007 Bradley Smith <digitalmars-com baysmith.com> writes: Bill Baxter wrote: Isn't Kirk's proposal also the way Java compilers do it? I seem to recall a bunch of foo.bar.baz.class files appearing last time I played with Java. Java uses directories. It would be foo/bar/baz.class.  Jan 26 2007 Bill Baxter <dnewsgroup billbaxter.com> writes: Bradley Smith wrote: Bill Baxter wrote: > Isn't Kirk's proposal also the way Java compilers do it? I seem to recall a bunch of foo.bar.baz.class files appearing last time I played with Java. Java uses directories. It would be foo/bar/baz.class. What does it do with inner classes then? Maybe that's what I was thinking of. --bb  Jan 26 2007 Bradley Smith <digitalmars-com baysmith.com> writes: Bill Baxter wrote: Bradley Smith wrote: Bill Baxter wrote: > Isn't Kirk's proposal also the way Java compilers do it? I seem to recall a bunch of foo.bar.baz.class files appearing last time I played with Java. Java uses directories. It would be foo/bar/baz.class. What does it do with inner classes then? Maybe that's what I was thinking of. Something like foo/bar/baz$inner.class.

Jan 28 2007