www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - The annoying D build system

reply Jacob Carlborg <doob me.com> writes:
Is it just me that thinks the build system of D (DMD, Phobos, druntime, 
and so on) is really annoying. I'm mostly thinking of the directory 
structure that the makefiles expect.

For example, when building Phobos it will also, automatically, build 
druntime, which it expects to find in the parent directory.

Today I tried to build the dlang.org web site. The first thing that hit 
me was that it tried to build a complete release out of DMD by default. 
Sure that can be consider my fault since I didn't check what targets the 
makefile would run by default. But this makefile will also expect DMD, 
Phobos and so on to be in the parent directory. It will also expect that 
its own folder is named "d-programming-language.org". I renamed that to 
"dlang.org" since it's shorter and that's what the site is called these 
days, but that completely broke the makefile. The makefile is looking 
explicitly for a folder named "d-programming-language.org", i.e. 
"../d-programming-language.org/std.ddoc".

I think it's really annoying, and slightly scary, that the build system 
breaks out of its own directory, doing things in the parent directory 
and expects a certain parent directory structure.

-- 
/Jacob Carlborg
Nov 19 2012
next sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 11/19/12, Jacob Carlborg <doob me.com> wrote:
 Is it just me that thinks the build system of D (DMD, Phobos, druntime,
 and so on) is really annoying. I'm mostly thinking of the directory
 structure that the makefiles expect.

Well on win32 I have two directories on my drive, "dmd" and "dmd-git". "dmd" is an official release, while "dmd-git" is git-head. The dir structure is: dmd-git\dmd2\src\dmd <- git-head dmd-git\dmd2\src\druntime <- git-head dmd-git\dmd2\src\phobos <- git-head dmd-git\dmd2\windows\ ... <- binaries, etc "dmd-git" is in the PATH env variable before "dmd" so it always overrides it. When I want to switch between using git-head and an official release I append an underscore to "dmd-git" (a shortcut key can do the trick using autohotkey). It's the simplest system I could think of..
Nov 19 2012
parent reply Jacob Carlborg <doob me.com> writes:
On 2012-11-19 11:34, Andrej Mitrovic wrote:

 Well on win32 I have two directories on my drive, "dmd" and "dmd-git".
 "dmd" is an official release, while "dmd-git" is git-head. The dir
 structure is:

 dmd-git\dmd2\src\dmd  <- git-head
 dmd-git\dmd2\src\druntime <- git-head
 dmd-git\dmd2\src\phobos <- git-head
 dmd-git\dmd2\windows\  ... <- binaries, etc

 "dmd-git" is in the PATH env variable before "dmd" so it always
 overrides it. When I want to switch between using git-head and an
 official release I append an underscore to "dmd-git" (a shortcut key
 can do the trick using autohotkey).

 It's the simplest system I could think of..

I think you misunderstood what I was trying to say, or I am misunderstanding you. -- /Jacob Carlborg
Nov 19 2012
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-11-20 09:21, Andrej Mitrovic wrote:

 I'm showing the path of least resistance for how to build either
 dmd/druntime/phobos/dlang-org

The build systems for the repositories are still breaking out of their top level directory.
 They are buggy on Windows. The last time I've used them Git started
 showing thousands of error messages, crashed and erased my repo and
 commit history, but one of the last few commits were left in the
 reflog so I managed to recover my files. It also copied the submodule
 repo directory into randomly selected folders in an upper directory.

 Maybe it's just an msysgit thing, but I'm never touching submodules again.

I haven't had any problems with submodules so far. I don't really understand why they wouldn't work as good on Windows as the other platforms. -- /Jacob Carlborg
Nov 20 2012
prev sibling parent =?ISO-8859-1?Q?Alex_R=F8nne_Petersen?= <alex lycus.org> writes:
On 20-11-2012 09:21, Andrej Mitrovic wrote:
 On 11/20/12, Jacob Carlborg <doob me.com> wrote:
 I think you misunderstood what I was trying to say, or I am
 misunderstanding you.

I'm showing the path of least resistance for how to build either dmd/druntime/phobos/dlang-org
 One solution that instantly pops in my mind is using git submodules.

They are buggy on Windows. The last time I've used them Git started showing thousands of error messages, crashed and erased my repo and commit history, but one of the last few commits were left in the reflog so I managed to recover my files. It also copied the submodule repo directory into randomly selected folders in an upper directory. Maybe it's just an msysgit thing, but I'm never touching submodules again.

Er... what? I certainly cannot say I've had the issues you describe. Submodules have always worked flawlessly on Windows for me. -- Alex Rønne Petersen alex lycus.org http://lycus.org
Nov 20 2012
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 11/19/12, Andrej Mitrovic <andrej.mitrovich gmail.com> wrote:
 dmd-git\dmd2\src\dmd  <- git-head
 dmd-git\dmd2\src\druntime <- git-head
 dmd-git\dmd2\src\phobos <- git-head
 dmd-git\dmd2\windows\  ... <- binaries, etc

Also: dmd-git\dmd2\src\d-programming-language.org
Nov 19 2012
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 11/19/12 3:29 AM, Jacob Carlborg wrote:
 Is it just me that thinks the build system of D (DMD, Phobos, druntime,
 and so on) is really annoying. I'm mostly thinking of the directory
 structure that the makefiles expect.

 For example, when building Phobos it will also, automatically, build
 druntime, which it expects to find in the parent directory.

That's because phobos is dependent on druntime.
 Today I tried to build the dlang.org web site. The first thing that hit
 me was that it tried to build a complete release out of DMD by default.

That is because the current site docs must include the current druntime and phobos docs. In turn, those must be built with the currently released compiler. To automate all that, the build makes sure the latest released compiler is built and then the docs for the latest released libs are built. Before that we were in the situation that updating the site was putting whatever was in the Phobos head (or worse, what was on the machine the site was built on).
 Sure that can be consider my fault since I didn't check what targets the
 makefile would run by default. But this makefile will also expect DMD,
 Phobos and so on to be in the parent directory. It will also expect that
 its own folder is named "d-programming-language.org". I renamed that to
 "dlang.org" since it's shorter and that's what the site is called these
 days, but that completely broke the makefile. The makefile is looking
 explicitly for a folder named "d-programming-language.org", i.e.
 "../d-programming-language.org/std.ddoc".

That's the name of the github project.
 I think it's really annoying, and slightly scary, that the build system
 breaks out of its own directory, doing things in the parent directory
 and expects a certain parent directory structure.

The makefiles have evolved organically out of clear and simple necessities. Also, they could use some love because there was more focus on solving specific automation issues, rather than on a globally nice system. So if you have ideas on how to improve the above - great! Just don't fall for Chesterton's fence fallacy. Andrei
Nov 19 2012
parent Jacob Carlborg <doob me.com> writes:
On 2012-11-19 21:00, Andrei Alexandrescu wrote:

 That's the name of the github project.

Yeah, but I haven't seen any other project that is actually dependent on the name of its own folder.
 So if you have ideas on how to improve the above - great! Just don't
 fall for Chesterton's fence fallacy.

One solution that instantly pops in my mind is using git submodules. Create a new repository, say "dlang", add the other repositories as git submoduels to "dlang". Then the build system can do whatever it wants in the "dlang" directory and its sub directories, but it shouldn't escape the "dlang" directory. Since Make is one of the worst thing I know, I'm not very enthusiastic about improving the build system. -- /Jacob Carlborg
Nov 19 2012
prev sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 11/20/12, Jacob Carlborg <doob me.com> wrote:
 I think you misunderstood what I was trying to say, or I am
 misunderstanding you.

I'm showing the path of least resistance for how to build either dmd/druntime/phobos/dlang-org
 One solution that instantly pops in my mind is using git submodules.

They are buggy on Windows. The last time I've used them Git started showing thousands of error messages, crashed and erased my repo and commit history, but one of the last few commits were left in the reflog so I managed to recover my files. It also copied the submodule repo directory into randomly selected folders in an upper directory. Maybe it's just an msysgit thing, but I'm never touching submodules again.
Nov 20 2012