www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - module thoughts

reply "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
in the D docs, it says that "one cause for slow compile times in C/C++ is
recompiling tens of thousands of lines of header files every time" or
something along those lines.

if i'm not mistaken, it has to parse any imported modules into a symbol
table.  granted, it's not compiling them, but it's still parsing *every*
imported module *every time*.

it seems a little counterintuituve, though i'm not sure what the alternative
would be.


secondly, say you're using a batch file to make your program, and in it you
have something along the lines of:

\dmd\bin\dmd myfile.d mymodule.d -c

this will make 2 object files, one for the main file and one for the module.
if you then want D to just link the object rather than recompiling the
module, you then must change the command line:

\dmd\bin\dmd myfile.d mymodule.obj

it "imports" the module from the main file, thus parsing the symbols, and
then links to the module.

every time you change the module, you would have to recompile it into an
obj, unless you want to do this:

\dmd\bin\dmd myfile.d mymodule.d

thus recompiling both files every time, making things easier but slower.

is there any way to automatically detect which modules need to be rebuilt?
i suppose an IDE would help here ;)
Jul 12 2004
next sibling parent "Walter" <newshound digitalmars.com> writes:
"Jarrett Billingsley" <kb3ctd2 yahoo.com> wrote in message
news:ccvjhq$13t9$1 digitaldaemon.com...
 in the D docs, it says that "one cause for slow compile times in C/C++ is
 recompiling tens of thousands of lines of header files every time" or
 something along those lines.

 if i'm not mistaken, it has to parse any imported modules into a symbol
 table.  granted, it's not compiling them, but it's still parsing *every*
 imported module *every time*.

 it seems a little counterintuituve, though i'm not sure what the

 would be.

True if you compile each module separately. But you can lump them all on one command line, and then they only get scanned once eacy.
 secondly, say you're using a batch file to make your program, and in it

 have something along the lines of:

 \dmd\bin\dmd myfile.d mymodule.d -c

 this will make 2 object files, one for the main file and one for the

 if you then want D to just link the object rather than recompiling the
 module, you then must change the command line:

 \dmd\bin\dmd myfile.d mymodule.obj

 it "imports" the module from the main file, thus parsing the symbols, and
 then links to the module.

 every time you change the module, you would have to recompile it into an
 obj, unless you want to do this:

 \dmd\bin\dmd myfile.d mymodule.d

 thus recompiling both files every time, making things easier but slower.

 is there any way to automatically detect which modules need to be rebuilt?
 i suppose an IDE would help here ;)

Using makefiles is the typical solution. www.digitalmars.com/ctg/make.exe
Jul 12 2004
prev sibling next sibling parent Lars Ivar Igesund <larsivar igesund.net> writes:
Jarrett Billingsley wrote:

 thus recompiling both files every time, making things easier but slower.

Actually, I believe it's still faster in most cases. If you have very many files, the one file at a time approach might be better if some dependency checking is done making sure only changed files are recompiled. The dependency checking slows down the process though. On linux, due to slower linking, the situation might be quite different. Lars Ivar Igesund
Jul 13 2004
prev sibling next sibling parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <ccvjhq$13t9$1 digitaldaemon.com>, Jarrett Billingsley says...
in the D docs, it says that "one cause for slow compile times in C/C++ is
recompiling tens of thousands of lines of header files every time" or
something along those lines.

if i'm not mistaken, it has to parse any imported modules into a symbol
table.  granted, it's not compiling them, but it's still parsing *every*
imported module *every time*.

Walter says that the parse process is much, much faster than compilation, and so shouldn't worry anyone. I discovered, during the course of putting etc.unicode.unicode together, however, that this isn't quite as true as we'd like to believe. etc.unicode.unicode is a rather unusual beast, of course, consisting of hundreds of source-files, some containing const array literals thousands of lines long. For something like this, even /parsing/ them is nightmarishly slow. Fortunately, I have now persuaded my robot to create stripped headers, which now of course compile like lightning.
is there any way to automatically detect which modules need to be rebuilt?

Most people use "make" for this - or some other system which achieves the same effect. Me, I have a custom build script written in PHP which figures out what to compile by recursively parsing import statements, starting from the file containing main(). It works out all the dependencies, and what needs to be rebuilt and what doesn't, and then gets on with it. As an added bonus it also figures out what libraries to link with. (It's pretty neat actually, because a new file becomes part of my project simply be being mentioned in an import statement - no messing about with makefiles needed). If you're not comfortable with writing your own scripts, and you're not comfortable with make, there are other alternatives, which I'm sure other people will suggest. Arcane Jill
Jul 13 2004
next sibling parent reply "David Barrett" <dbarrett quinthar.com> writes:
"Arcane Jill" <Arcane_member pathlink.com> wrote in message
news:cd04t2$241v$1 digitaldaemon.com...
 For something like this, even /parsing/ them is nightmarishly slow.

 I have now persuaded my robot to create stripped headers, which now of

 compile like lightning.

Hm, that's disconcerting that you needed to effectively "undo" one of D's primary features ("no header files!") to make D compile a complex app. However, it's cool that header files are treated as an "optimization" in D, rather than a necessity (as in C/C++). Is there any way to automate this process -- perhaps flag certain D modules (possibly a #pragma in the file?) so the compiler automatically performs this optimization? Second best, can we package a header stripping tool with DMD so it can be available to everyone? -david
Jul 13 2004
parent Arcane Jill <Arcane_member pathlink.com> writes:
In article <cd115g$m7m$1 digitaldaemon.com>, David Barrett says...

Hm, that's disconcerting that you needed to effectively "undo" one of D's
primary features ("no header files!") to make D compile a complex app.

It's not a complex app. Actually, it's a very simple bunch of table lookups. It's just the sheer size and number of tables that slow things down, so header files in this case just shorten compilation times, that's all.
However, it's cool that header files are treated as an "optimization" in D,
rather than a necessity (as in C/C++).

Yeah, I like that too.
Is there any way to automate this process -- perhaps flag certain D modules
(possibly a #pragma in the file?) so the compiler automatically performs
this optimization?

Without clairvoyance, I don't see how it can. But then, I've never written a compiler. I guess Walter would know if it's feasable.
Second best, can we package a header stripping tool with
DMD so it can be available to everyone?

Walter says that would be an easy thing to create, but so far no-one's volunteered to make it. (You want the job?). The people most interested in this would be closed-source developers who want to hide their source code. I'm very much an open-source advocate, myself, so I don't think I'd be particularly interested in such a tool. (The Unicode thing is robot-generated, so header files just get created alongside the full source files). Arcane Jill
Jul 13 2004
prev sibling parent "Walter" <newshound digitalmars.com> writes:
"Arcane Jill" <Arcane_member pathlink.com> wrote in message
news:cd04t2$241v$1 digitaldaemon.com...
 etc.unicode.unicode is a rather unusual beast, of course, consisting of

 of source-files, some containing const array literals thousands of lines

 For something like this, even /parsing/ them is nightmarishly slow.

 I have now persuaded my robot to create stripped headers, which now of

 compile like lightning.

If you can give me a canonical test case for this, I'll look and see if the lexer has any stupid bottlenecks in it that can be fixed.
Jul 13 2004
prev sibling parent "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
thank you all :)  i'll look into make.
Jul 13 2004