www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Building is slow!

reply Ivan Trombley <itrombley dot-borg.org> writes:
When DUB bulds the gtk-d library, it takes a long time. This is 
mostly because it's only using one processor. It hasn't been such 
a big deal on Linux but I'm building my app on Windows right now 
and it been building gtk-d for the last half hour! Is there any 
way to make DUB use more processors?
Dec 01 2017
next sibling parent reply rjframe <dlang ryanjframe.com> writes:
On Sat, 02 Dec 2017 00:24:12 +0000, Ivan Trombley wrote:

 When DUB bulds the gtk-d library, it takes a long time. This is mostly
 because it's only using one processor. It hasn't been such a big deal on
 Linux but I'm building my app on Windows right now and it been building
 gtk-d for the last half hour! Is there any way to make DUB use more
 processors?
There is a --parallel flag: `dub build --parallel`. The help string says it "Runs multiple compiler instances in parallel, if possible."
Dec 01 2017
parent reply Ivan Trombley <itrombley dot-borg.org> writes:
On Saturday, 2 December 2017 at 03:08:10 UTC, rjframe wrote:
 There is a --parallel flag: `dub build --parallel`.

 The help string says it "Runs multiple compiler instances in 
 parallel, if possible."
Thanks, I'll give that a try. After an hour, I pressed CTRL+C, shut it down and went home.
Dec 02 2017
next sibling parent tetyys <tetyys tetyys.com> writes:
Something must be definitely wrong if compilation takes half an 
hour, unless you're trying to compile it on a toaster. An 
infinite loop?
Dec 02 2017
prev sibling parent reply Arek <arychlinski gmail.com> writes:
On Saturday, 2 December 2017 at 09:50:32 UTC, Ivan Trombley wrote:
 On Saturday, 2 December 2017 at 03:08:10 UTC, rjframe wrote:
 There is a --parallel flag: `dub build --parallel`.

 The help string says it "Runs multiple compiler instances in 
 parallel, if possible."
Thanks, I'll give that a try. After an hour, I pressed CTRL+C, shut it down and went home.
By default, when you run `dub build` building is done in mode 'separate', what means all source files are compiled at once in one dmd instance. In case of big project it can consume a lot of memory and sometimes swaps intensively. Also "--parallel" doesn't change anything, because there is only one process of the compiler so it runs on one core only. You can try `dub build --build-mode=single-file --parallel`. It will execute separate instance of compiler for each source file. If --parallel is given, dub will launch several instances of dmd in parallel.
Dec 02 2017
parent reply Ivan Trombley <itrombley dot-borg.org> writes:
On Saturday, 2 December 2017 at 14:34:58 UTC, Arek wrote:
 You can try `dub build --build-mode=single-file --parallel`. It 
 will execute separate instance of compiler for each source 
 file. If --parallel is given, dub will launch several instances 
 of dmd in parallel.
I get the error: Error processing arguments: BuildMode does not have a member named 'single-file' On Saturday, 2 December 2017 at 22:54:56 UTC, Indigo wrote:
 I haven't tried gtkd in a while but when I did it built in 
 seconds... I was not using dub though but the build script that 
 comes with it.
My project has "dependencies": { "gtk-d": "3.7.2" } To build release, it takes more than three minutes to build gtk-d on Linux (Ryzen 7 1700 processor).
Dec 02 2017
next sibling parent reply Ivan Trombley <itrombley dot-borg.org> writes:
On Saturday, 2 December 2017 at 23:26:20 UTC, Ivan Trombley wrote:
 On Saturday, 2 December 2017 at 14:34:58 UTC, Arek wrote:
 You can try `dub build --build-mode=single-file --parallel`. 
 It will execute separate instance of compiler for each source 
 file. If --parallel is given, dub will launch several 
 instances of dmd in parallel.
I get the error: Error processing arguments: BuildMode does not have a member named 'single-file'
--build-mode=singleFile seems to work. This shaved off 1/3 of the time to compile gtk-d in release mode on Linux, so I'm going to give this a try on Windows.
Dec 02 2017
parent reply Ivan Trombley <itrombley dot-borg.org> writes:
There are issues with using "--build-mode=singleFile --parallel". 
On Windows I get errors saying that it can't write out some 
intermediate files (it looks like the file names may be too long 
for Windows) and on Linux, it makes the executable at least 3 MB 
larger in release mode. Also, it doesn't always seem to make 
building faster. On a 2 core i7 machine, it actually takes nearly 
twice as long to build.
Dec 05 2017
parent Arek <arychlinski gmail.com> writes:
On Tuesday, 5 December 2017 at 19:28:12 UTC, Ivan Trombley wrote:
 There are issues with using "--build-mode=singleFile 
 --parallel". On Windows I get errors saying that it can't write 
 out some intermediate files (it looks like the file names may 
 be too long for Windows) and on Linux, it makes the executable 
 at least 3 MB larger in release mode. Also, it doesn't always 
 seem to make building faster. On a 2 core i7 machine, it 
 actually takes nearly twice as long to build.
On my old i7-950, simple demo (cairo_clock) compiles (in release mode) below 4 minutes and uses 1.8GB of RAM (linux, one core used). Rebuild is done in about 3 seconds. I think Windows may introduce some problems in case of parallel access to files (especially if something is deleted). But I have no idea why your builds last so long. :/
Dec 05 2017
prev sibling parent Arek <arychlinski gmail.com> writes:
On Saturday, 2 December 2017 at 23:26:20 UTC, Ivan Trombley wrote:
 On Saturday, 2 December 2017 at 14:34:58 UTC, Arek wrote:
 You can try `dub build --build-mode=single-file --parallel`. 
 It will execute separate instance of compiler for each source 
 file. If --parallel is given, dub will launch several 
 instances of dmd in parallel.
I get the error: Error processing arguments: BuildMode does not have a member named 'single-file'
Sorry, my mistake. `--build-mode=singleFile`
 On Saturday, 2 December 2017 at 22:54:56 UTC, Indigo wrote:
 I haven't tried gtkd in a while but when I did it built in 
 seconds... I was not using dub though but the build script 
 that comes with it.
My project has "dependencies": { "gtk-d": "3.7.2" } To build release, it takes more than three minutes to build gtk-d on Linux (Ryzen 7 1700 processor).
The only reason for really long builds I know, was a lack of memory and swapping.
Dec 03 2017
prev sibling next sibling parent Indigo <Indigo Go.go> writes:
On Saturday, 2 December 2017 at 00:24:12 UTC, Ivan Trombley wrote:
 When DUB bulds the gtk-d library, it takes a long time. This is 
 mostly because it's only using one processor. It hasn't been 
 such a big deal on Linux but I'm building my app on Windows 
 right now and it been building gtk-d for the last half hour! Is 
 there any way to make DUB use more processors?
I haven't tried gtkd in a while but when I did it built in seconds... I was not using dub though but the build script that comes with it.
Dec 02 2017
prev sibling parent reply Ivan Trombley <itrombley dot-borg.org> writes:
I want to revisit this issue.

Building 64 bit on Linux, release or debug, is fast. However, 
building 64 bit release on Windows 10 is super slow. I have a 
cross platform app that uses gtk-d. Today, I updated DMD to 
2.079.1 and the gtk-d lib to 3.8.0. When I performed a debug 
build on Windows 10, it only took a few seconds to build gtk-d. I 
attempted to build release but canceled the build after a couple 
of hours. I tried building 32 bit release on Windows and while it 
took a lot longer than debug, it still completed in a reasonable 
amount of time (it wouldn't link, though, probably because I'm 
missing some 32 libraries).

Does anyone have any idea why 64 bit release builds on Windows 
would take so long?
Apr 16 2018
next sibling parent Daniel Kozak <kozzi11 gmail.com> writes:
Try disable your antivirus

On Mon, Apr 16, 2018 at 10:27 PM, Ivan Trombley via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 I want to revisit this issue.

 Building 64 bit on Linux, release or debug, is fast. However, building 64
 bit release on Windows 10 is super slow. I have a cross platform app that
 uses gtk-d. Today, I updated DMD to 2.079.1 and the gtk-d lib to 3.8.0.
 When I performed a debug build on Windows 10, it only took a few seconds to
 build gtk-d. I attempted to build release but canceled the build after a
 couple of hours. I tried building 32 bit release on Windows and while it
 took a lot longer than debug, it still completed in a reasonable amount of
 time (it wouldn't link, though, probably because I'm missing some 32
 libraries).

 Does anyone have any idea why 64 bit release builds on Windows would take
 so long?
Apr 17 2018
prev sibling parent Dlang User <dlang.user gmx.com> writes:
On 4/16/2018 3:27 PM, Ivan Trombley wrote:
 I want to revisit this issue.
 
 Building 64 bit on Linux, release or debug, is fast. However, building 
 64 bit release on Windows 10 is super slow. I have a cross platform app 
 that uses gtk-d. Today, I updated DMD to 2.079.1 and the gtk-d lib to 
 3.8.0. When I performed a debug build on Windows 10, it only took a few 
 seconds to build gtk-d. I attempted to build release but canceled the 
 build after a couple of hours. I tried building 32 bit release on 
 Windows and while it took a lot longer than debug, it still completed in 
 a reasonable amount of time (it wouldn't link, though, probably because 
 I'm missing some 32 libraries).
 
 Does anyone have any idea why 64 bit release builds on Windows would 
 take so long?
 
I have no idea why, but I can confirm that I am seeing something similar on Windows 10. I have tried it both with anti-virus enabled and disabled, with the same results. Compiler Version: 2.079.1 (-m64 switch set in the sc.ini file) Using a gtkd project (tried two versions 3.8.0 and 3.8.1) Attempts: 1. plain build It succeeds with this output (build time is 21 seconds, including fetching): C:\DProj\gtktest>dub build -b plain --force Fetching gtk-d 3.8.1 (getting selected version)... Performing "plain" build using C:\D\dmd2\windows\bin\dmd.exe for x86_64. gtk-d:gtkd 3.8.1: building configuration "library"... gtk-d:gstreamer 3.8.1: building configuration "library"... gtk-d:peas 3.8.1: building configuration "library"... gtk-d:sv 3.8.1: building configuration "library"... gtk-d:vte 3.8.1: building configuration "library"... gtktest ~master: building configuration "application"... Linking... 2. debug build It succeeds with a linker warning (build time is 23 seconds, already fetched): C:\DProj\gtktest>dub build -b debug --force Performing "debug" build using C:\D\dmd2\windows\bin\dmd.exe for x86_64. gtk-d:gtkd 3.8.1: building configuration "library"... gtk-d:gstreamer 3.8.1: building configuration "library"... gtk-d:peas 3.8.1: building configuration "library"... gtk-d:sv 3.8.1: building configuration "library"... gtk-d:vte 3.8.1: building configuration "library"... gtktest ~master: building configuration "application"... Linking... gtkd-3.lib(functions.obj) : warning LNK4255: library contain multiple objects of the same name; linking object as if no debug info gtkd-3.lib(functions.obj) : warning LNK4255: library contain multiple objects of the same name; linking object as if no debug info gtkd-3.lib(functions.obj) : warning LNK4255: library contain multiple objects of the same name; linking object as if no debug info gtkd-3.lib(functions.obj) : warning LNK4255: library contain multiple objects of the same name; linking object as if no debug info gtkd-3.lib(functions.obj) : warning LNK4255: library contain multiple objects of the same name; linking object as if no debug info gtkd-3.lib(functions.obj) : warning LNK4255: library contain multiple objects of the same name; linking object as if no debug info gtkd-3.lib(functions.obj) : warning LNK4255: library contain multiple objects of the same name; linking object as if no debug info gtkd-3.lib(functions.obj) : warning LNK4255: library contain multiple objects of the same name; linking object as if no debug info gtkd-3.lib(ActionIF.obj) : warning LNK4255: library contain multiple objects of the same name; linking object as if no debug info 3. release build This hangs (I killed it after 10 minutes): C:\DProj\gtktest>dub build -b release --force Performing "release" build using C:\D\dmd2\windows\bin\dmd.exe for x86_64. gtk-d:gtkd 3.8.1: building configuration "library"...
Apr 18 2018