www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Building is slow!

reply Ivan Trombley <itrombley dot-borg.org> writes:
When DUB bulds the gtk-d library, it takes a long time. This is 
mostly because it's only using one processor. It hasn't been such 
a big deal on Linux but I'm building my app on Windows right now 
and it been building gtk-d for the last half hour! Is there any 
way to make DUB use more processors?
Dec 01
next sibling parent reply rjframe <dlang ryanjframe.com> writes:
On Sat, 02 Dec 2017 00:24:12 +0000, Ivan Trombley wrote:

 When DUB bulds the gtk-d library, it takes a long time. This is mostly
 because it's only using one processor. It hasn't been such a big deal on
 Linux but I'm building my app on Windows right now and it been building
 gtk-d for the last half hour! Is there any way to make DUB use more
 processors?
There is a --parallel flag: `dub build --parallel`. The help string says it "Runs multiple compiler instances in parallel, if possible."
Dec 01
parent reply Ivan Trombley <itrombley dot-borg.org> writes:
On Saturday, 2 December 2017 at 03:08:10 UTC, rjframe wrote:
 There is a --parallel flag: `dub build --parallel`.

 The help string says it "Runs multiple compiler instances in 
 parallel, if possible."
Thanks, I'll give that a try. After an hour, I pressed CTRL+C, shut it down and went home.
Dec 02
next sibling parent tetyys <tetyys tetyys.com> writes:
Something must be definitely wrong if compilation takes half an 
hour, unless you're trying to compile it on a toaster. An 
infinite loop?
Dec 02
prev sibling parent reply Arek <arychlinski gmail.com> writes:
On Saturday, 2 December 2017 at 09:50:32 UTC, Ivan Trombley wrote:
 On Saturday, 2 December 2017 at 03:08:10 UTC, rjframe wrote:
 There is a --parallel flag: `dub build --parallel`.

 The help string says it "Runs multiple compiler instances in 
 parallel, if possible."
Thanks, I'll give that a try. After an hour, I pressed CTRL+C, shut it down and went home.
By default, when you run `dub build` building is done in mode 'separate', what means all source files are compiled at once in one dmd instance. In case of big project it can consume a lot of memory and sometimes swaps intensively. Also "--parallel" doesn't change anything, because there is only one process of the compiler so it runs on one core only. You can try `dub build --build-mode=single-file --parallel`. It will execute separate instance of compiler for each source file. If --parallel is given, dub will launch several instances of dmd in parallel.
Dec 02
parent reply Ivan Trombley <itrombley dot-borg.org> writes:
On Saturday, 2 December 2017 at 14:34:58 UTC, Arek wrote:
 You can try `dub build --build-mode=single-file --parallel`. It 
 will execute separate instance of compiler for each source 
 file. If --parallel is given, dub will launch several instances 
 of dmd in parallel.
I get the error: Error processing arguments: BuildMode does not have a member named 'single-file' On Saturday, 2 December 2017 at 22:54:56 UTC, Indigo wrote:
 I haven't tried gtkd in a while but when I did it built in 
 seconds... I was not using dub though but the build script that 
 comes with it.
My project has "dependencies": { "gtk-d": "3.7.2" } To build release, it takes more than three minutes to build gtk-d on Linux (Ryzen 7 1700 processor).
Dec 02
next sibling parent reply Ivan Trombley <itrombley dot-borg.org> writes:
On Saturday, 2 December 2017 at 23:26:20 UTC, Ivan Trombley wrote:
 On Saturday, 2 December 2017 at 14:34:58 UTC, Arek wrote:
 You can try `dub build --build-mode=single-file --parallel`. 
 It will execute separate instance of compiler for each source 
 file. If --parallel is given, dub will launch several 
 instances of dmd in parallel.
I get the error: Error processing arguments: BuildMode does not have a member named 'single-file'
--build-mode=singleFile seems to work. This shaved off 1/3 of the time to compile gtk-d in release mode on Linux, so I'm going to give this a try on Windows.
Dec 02
parent reply Ivan Trombley <itrombley dot-borg.org> writes:
There are issues with using "--build-mode=singleFile --parallel". 
On Windows I get errors saying that it can't write out some 
intermediate files (it looks like the file names may be too long 
for Windows) and on Linux, it makes the executable at least 3 MB 
larger in release mode. Also, it doesn't always seem to make 
building faster. On a 2 core i7 machine, it actually takes nearly 
twice as long to build.
Dec 05
parent Arek <arychlinski gmail.com> writes:
On Tuesday, 5 December 2017 at 19:28:12 UTC, Ivan Trombley wrote:
 There are issues with using "--build-mode=singleFile 
 --parallel". On Windows I get errors saying that it can't write 
 out some intermediate files (it looks like the file names may 
 be too long for Windows) and on Linux, it makes the executable 
 at least 3 MB larger in release mode. Also, it doesn't always 
 seem to make building faster. On a 2 core i7 machine, it 
 actually takes nearly twice as long to build.
On my old i7-950, simple demo (cairo_clock) compiles (in release mode) below 4 minutes and uses 1.8GB of RAM (linux, one core used). Rebuild is done in about 3 seconds. I think Windows may introduce some problems in case of parallel access to files (especially if something is deleted). But I have no idea why your builds last so long. :/
Dec 05
prev sibling parent Arek <arychlinski gmail.com> writes:
On Saturday, 2 December 2017 at 23:26:20 UTC, Ivan Trombley wrote:
 On Saturday, 2 December 2017 at 14:34:58 UTC, Arek wrote:
 You can try `dub build --build-mode=single-file --parallel`. 
 It will execute separate instance of compiler for each source 
 file. If --parallel is given, dub will launch several 
 instances of dmd in parallel.
I get the error: Error processing arguments: BuildMode does not have a member named 'single-file'
Sorry, my mistake. `--build-mode=singleFile`
 On Saturday, 2 December 2017 at 22:54:56 UTC, Indigo wrote:
 I haven't tried gtkd in a while but when I did it built in 
 seconds... I was not using dub though but the build script 
 that comes with it.
My project has "dependencies": { "gtk-d": "3.7.2" } To build release, it takes more than three minutes to build gtk-d on Linux (Ryzen 7 1700 processor).
The only reason for really long builds I know, was a lack of memory and swapping.
Dec 03
prev sibling parent Indigo <Indigo Go.go> writes:
On Saturday, 2 December 2017 at 00:24:12 UTC, Ivan Trombley wrote:
 When DUB bulds the gtk-d library, it takes a long time. This is 
 mostly because it's only using one processor. It hasn't been 
 such a big deal on Linux but I'm building my app on Windows 
 right now and it been building gtk-d for the last half hour! Is 
 there any way to make DUB use more processors?
I haven't tried gtkd in a while but when I did it built in seconds... I was not using dub though but the build script that comes with it.
Dec 02