www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.ldc - [WiP] Creating a snap package of LDC

reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
Hello all,

I decided that it was time to try creating snap packages of 
various core D programs, and I thought I'd start with LDC since 
its very standard cmake-based build and install makes for 
super-easy integration with the snapcraft package-creation system.

The TL;DR is that creating the _package_ is super-easy, but there 
are a handful of interesting integration issues that it would be 
good to have your advice on.

For cross-reference, here's the discussion on the snapcraft 
mailing list where I raise the issues I encountered:
https://lists.ubuntu.com/archives/snapcraft/2016-August/000860.html

... and I'm writing the current email in the wake of feedback 
received there.  Anyway, first let me describe what I've done, 
before raising any questions.

The basics of how to create an LDC snap package are super easy.  
Simply create a file called `snapcraft.yaml` in an empty 
directory, containing the following contents:

-----------------------------------------
name: ldc2
version: "1.0.0"
summary: D compiler with LLVM backend
description: LDC is a compiler for the D programming Language.
              It is based on the latest DMD frontend and uses LLVM 
as backend.
confinement: strict

apps:
   ldc2:
     command: ldc2
     plugs: [home]
   ldmd2:
     command: ldmd2
     plugs: [home]

parts:
   ldc:
     plugin: cmake
     source: git://github.com/ldc-developers/ldc.git
     source-tag: v1.0.0
     build-packages:
     - ldc
     - llvm-dev
     - libconfig++-dev
     - libcurl4-gnutls-dev
     - libedit-dev
     - zlib1g-dev
-----------------------------------------

As you can see, this metadata describes the apps to be included 
in the snap package, together with the build plugin required 
(cmake is supported out of the box), and where to get the source 
(git is again out-of-the-box supported and you simply need to 
specify the version tag to check out).

Note the listing of `ldc` as a build dependency: this will use 
the Ubuntu-packaged ldc 0.17.1.  Note also the ["home"] plugs: 
this ensures that the applications in the containerized snap will 
still be able to access the user's home directory (pretty 
important if you want to build stuff).

Given all this, on any Ubuntu 16.04 system, from the same 
directory as the `snapcraft.yaml` file, just run:

     snapcraft

The snapcraft program will install any necessary build 
dependencies, check out the code, build it, and install the 
results into a `stage` directory.  This is then further processed 
to create a `prime` directory creating the actual contents that 
will be in the snap package; and finally, the snap package itself 
is generated.  Assuming you're running this on a 64-bit system, 
it will be named something like:

     ldc2_1.0.0_amd64.snap

This can then be installed (on Ubuntu 16.04) with:

     sudo snap install ldc2_1.0.0_amd64.snap

Snaps are installed (read-only) into the `/snap/` top-level 
folder; in this case, you can find the contents in 
`/snap/ldc2/current/` (`current` is a symlink to the particular 
snap that is currently in use).  Everything you'd expect to find 
in a normal LDC install will be in there: the `bin/` directory 
containing `ldc2` and `ldmd2`; the `etc/` directory containing 
`bash_completion.d` and `ldc2.conf`; the `include/` dir with the 
usual contents; the `lib/` dir containing static libraries of 
druntime and phobos (with and without debug symbols); and so on.

You should now be able to run `ldc2 --version` and see the 
appropriate info.

So far so good!  Unfortunately, now we have to start dealing with 
the integration issues.

One issue is to do with how the commands are exposed: the actual 
commands you can run will be called `ldc2` and `ldc2.ldmd2` (as 
snappy effectively 'namespaces' apps that do not share a name 
with the snap package).  This is solvable but will require some 
updates to `snapd` itself, so no need to discuss, just to be 
aware that this is the current situation.

Next, the `ldc2.conf` file's -I and -L path definitions are 
problematic, possibly because of the containerized way that 
snapcraft builds the snap:

     switches = [
         "-I/include/d/ldc",
         "-I/include/d",
         "-L-L/lib",
         "-defaultlib=phobos2-ldc,druntime-ldc",
         "-debuglib=phobos2-ldc-debug,druntime-ldc-debug"
     ];

This leads me to my first question: is it possible to specify the 
paths here relative to the location of the directory where the 
`ldc2` binary resides?  I tried using the dmd-style:

     -I% P%/../include/d/ldc

... etc., but that didn't seem to work (I don't know if is % P% 
in any way standard, or whether it's something dmd-specific...?).

Manually specifying the exact path on the command line (or 
manually editing the generated files before finalizing the snap 
package) can be used to work around this.

Finally, the snap-packaged LDC will currently fail at the linker 
stage, because (as I understand it) the snap package can't access 
regular system resources, only stuff that is in other snaps, and 
the core system snap doesn't include stuff like gcc.  So, 
building falls over with an error:

     Error: failed to locate gcc

There are a few long-term potential fixes here, including 
creating a snap that exposes GCC or a linker, but in the short 
term, I've been advised that probably the easiest way to create a 
reliable standalone LDC package is to bundle the linker into the 
snap.

So, my question is, what would be the minimum requirements for a 
linker to accompany ldc2, and how would I ensure that the built 
ldc2 tries to use _that_ linker rather than some other one?

Thanks & best wishes,

      -- Joe
Aug 28 2016
next sibling parent reply Dicebot <public dicebot.lv> writes:
On Sunday, 28 August 2016 at 11:58:00 UTC, Joseph Rushton 
Wakeling wrote:

 This leads me to my first question: is it possible to specify 
 the paths here relative to the location of the directory where 
 the `ldc2` binary resides?


AFAIK contents of config file "switches" variable in ldc is 
simply prepended to command-line flag list with no special 
processing.
Aug 28 2016
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Sunday, 28 August 2016 at 12:37:17 UTC, Dicebot wrote:

 On Sunday, 28 August 2016 at 11:58:00 UTC, Joseph Rushton 
 Wakeling wrote:

 This leads me to my first question: is it possible to specify 
 the paths here relative to the location of the directory where 
 the `ldc2` binary resides?


 AFAIK contents of config file "switches" variable in ldc is 
 simply prepended to command-line flag list with no special 
 processing.


Yes, that I understand.  The problem is that the contents of that 
file are wrong ;-)  I can edit it part way through the snap 
packaging process, but what I'd really like is an automatable 
solution.

The issue here is that, as I understand it, a snap package can 
wind up being installed in

/snap/snapname/current/

or /home/<user>/snap/snapname/current/

... so the precise path can't be known at package creation time 
but only at install time.

However, if the paths in `ldc2.conf` could be specified relative 
to the path where the `ldc2` binary lives, that might provide an 
easy workaround.
Aug 28 2016
next sibling parent reply Dicebot <public dicebot.lv> writes:
On Sunday, 28 August 2016 at 12:41:44 UTC, Joseph Rushton 
Wakeling wrote:

 On Sunday, 28 August 2016 at 12:37:17 UTC, Dicebot wrote:

 On Sunday, 28 August 2016 at 11:58:00 UTC, Joseph Rushton 
 Wakeling wrote:

 This leads me to my first question: is it possible to specify 
 the paths here relative to the location of the directory 
 where the `ldc2` binary resides?


 AFAIK contents of config file "switches" variable in ldc is 
 simply prepended to command-line flag list with no special 
 processing.


 Yes, that I understand.  The problem is that the contents of 
 that file are wrong ;-)  I can edit it part way through the 
 snap packaging process, but what I'd really like is an 
 automatable solution.


Providing correct ldc2.conf is normally a responsibility of a 
packager for the target system thus it doesn't seem wrong to me. 
You may want to check how the config file is defined in tarball 
provided by github releases though as I suppose it is intended to 
be usable without any installation.
Aug 28 2016
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Sunday, 28 August 2016 at 12:48:55 UTC, Dicebot wrote:

 Providing correct ldc2.conf is normally a responsibility of a 
 packager for the target system thus it doesn't seem wrong to me.


Well, yes... and hence these questions, because as the packager 
I'm trying to understand my options, so that I _can_ take 
responsibility for what's in the package being correct. ;-)

Note the constraint that while the path relative to the snap 
install location is well defined, the absolute install path isn't.


 You may want to check how the config file is defined in tarball 
 provided by github releases though as I suppose it is intended 
 to be usable without any installation.


Hmm, interesting. The ldc2.conf file includes what looks like a 
wildcard for the compiler location, but passing that wildcard 
manually via -I flags doesn't work. I'll re-check later (had to 
step out for a bit) but it looks like it might be intended to be 
manually replaced.
Aug 28 2016
parent reply kinke <noone nowhere.com> writes:
On Sunday, 28 August 2016 at 13:28:07 UTC, Joseph Rushton 
Wakeling wrote:

 Hmm, interesting. The ldc2.conf file includes what looks like a 
 wildcard for the compiler location, but passing that wildcard 
 manually via -I flags doesn't work. I'll re-check later (had to 
 step out for a bit) but it looks like it might be intended to 
 be manually replaced.


I think we only replace that special placeholder in the ldc2.conf 
contents, not generally in the command line.
Aug 28 2016
parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Sunday, 28 August 2016 at 13:31:42 UTC, kinke wrote:

 On Sunday, 28 August 2016 at 13:28:07 UTC, Joseph Rushton 
 Wakeling wrote:

 Hmm, interesting. The ldc2.conf file includes what looks like 
 a wildcard for the compiler location, but passing that 
 wildcard manually via -I flags doesn't work. I'll re-check 
 later (had to step out for a bit) but it looks like it might 
 be intended to be manually replaced.


 I think we only replace that special placeholder in the 
 ldc2.conf contents, not generally in the command line.


OK, cool. I'm out for the rest of the afternoon but I'll try to 
tweak the package this way later this evening & report back how 
it goes.

Meanwhile, thanks for the help!
Aug 28 2016
prev sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Sunday, 28 August 2016 at 12:41:44 UTC, Joseph Rushton 
Wakeling wrote:

 However, if the paths in `ldc2.conf` could be specified 
 relative to the path where the `ldc2` binary lives, that might 
 provide an easy workaround.


To be more precise: this is exactly what dmd uses in its dmd.conf 
file, e.g.:

[Environment32]
DFLAGS=-I% P%/../../src/phobos -I% P%/../../src/druntime/import 
-L-L% P%/../lib32 -L--export-dynamic

[Environment64]
DFLAGS=-I% P%/../../src/phobos -I% P%/../../src/druntime/import 
-L-L% P%/../lib64 -L--export-dynamic

However, would I be right in assuming that % P% is a dmd-specific 
wildcard that the compiler knows to replace with the compiler's 
path location?

I ask because I've tried using

   -I% P%/../include/d/ldc -I% P%/../include/d

manually on the command line with ldc2 and ldmd2 and it doesn't 
work, whereas manually specifying the _full_ path does.

Is there any way of providing a similarly 
relative-to-compiler-path -I option to ldc2 and/or ldmd2 ... ?
Aug 28 2016
parent reply kinke <noone nowhere.com> writes:
On Sunday, 28 August 2016 at 12:56:17 UTC, Joseph Rushton 
Wakeling wrote:

 Is there any way of providing a similarly 
 relative-to-compiler-path -I option to ldc2 and/or ldmd2 ... ?


Of course; the Windows package is already portable.
`%%ldcbinarypath%%` is replaced by the directory containing the 
ldc2 executable. So `-I%%ldcbinarypath%%/../include/d` etc. is 
the way to go.
Aug 28 2016
next sibling parent David Nadlinger via digitalmars-d-ldc <digitalmars-d-ldc puremagic.com> writes:
On 28 Aug 2016, at 14:28, kinke via digitalmars-d-ldc wrote:

 On Sunday, 28 August 2016 at 12:56:17 UTC, Joseph Rushton Wakeling 
 wrote:

 Is there any way of providing a similarly relative-to-compiler-path 
 -I option to ldc2 and/or ldmd2 ... ?


 Of course; the Windows package is already portable.


Not only the Windows packages, by the way – binary releases were 
portable pretty much ever since I first set them up in their current 
form.

  — David
Aug 28 2016
prev sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Sunday, 28 August 2016 at 13:28:33 UTC, kinke wrote:

 On Sunday, 28 August 2016 at 12:56:17 UTC, Joseph Rushton 
 Wakeling wrote:

 Is there any way of providing a similarly 
 relative-to-compiler-path -I option to ldc2 and/or ldmd2 ... ?


 Of course; the Windows package is already portable.
 `%%ldcbinarypath%%` is replaced by the directory containing the 
 ldc2 executable. So `-I%%ldcbinarypath%%/../include/d` etc. is 
 the way to go.


OK, great, this works.  So, at a minimum, I can manually tweak 
things to guarantee that the final snap package will have a 
workable ldc2.conf.

Is there any way to get the build process of LDC to auto-generate 
the file with these relative paths including the 
%%ldcbinarypath%% wildcard ... ?  e.g. a cmake or make flag?
Aug 28 2016
parent reply kinke <noone nowhere.com> writes:
On Sunday, 28 August 2016 at 19:47:57 UTC, Joseph Rushton 
Wakeling wrote:

 Is there any way to get the build process of LDC to 
 auto-generate the file with these relative paths including the 
 %%ldcbinarypath%% wildcard ... ?  e.g. a cmake or make flag?


A single line in PowerShell suffices to replace a well-known 
build dir with the placeholder in the ldc2.conf file:
https://github.com/ldc-developers/ldc/blob/master/appveyor.yml#L144


 This is another binary generated as part of the LDC build?


Yes (ninja target: ldc-profdata, included by the default target).
Aug 28 2016
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Sunday, 28 August 2016 at 20:15:28 UTC, kinke wrote:

 On Sunday, 28 August 2016 at 19:47:57 UTC, Joseph Rushton 
 Wakeling wrote:

 Is there any way to get the build process of LDC to 
 auto-generate the file with these relative paths including the 
 %%ldcbinarypath%% wildcard ... ?  e.g. a cmake or make flag?


 A single line in PowerShell suffices to replace a well-known 
 build dir with the placeholder in the ldc2.conf file:
 https://github.com/ldc-developers/ldc/blob/master/appveyor.yml#L144


Yea, and similarly I can use `sed` just fine on Linux.  I was 
just curious if there was a 'standard' way to do this that didn't 
involve search'n'replace.


 This is another binary generated as part of the LDC build?


 Yes (ninja target: ldc-profdata, included by the default 
 target).


Erm ... I've just done a completely fresh build of LDC v1.0.0 
(freshly checked out from git), and there's no sign of an 
ldc-profdata binary.  Nor does make seem to know anything about 
and ldc-profdata target.

What am I missing? :-\
Aug 28 2016
parent reply Johan Engelen <j j.nl> writes:
On Sunday, 28 August 2016 at 20:42:12 UTC, Joseph Rushton 
Wakeling wrote:

 
 Erm ... I've just done a completely fresh build of LDC v1.0.0 
 (freshly checked out from git), and there's no sign of an 
 ldc-profdata binary.  Nor does make seem to know anything about 
 and ldc-profdata target.

 What am I missing? :-\


Sorry, my fault. It's not part of 1.0.0.

-Johan
Aug 28 2016
parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Sunday, 28 August 2016 at 22:13:06 UTC, Johan Engelen wrote:

 On Sunday, 28 August 2016 at 20:42:12 UTC, Joseph Rushton 
 Wakeling wrote:

 
 Erm ... I've just done a completely fresh build of LDC v1.0.0 
 (freshly checked out from git), and there's no sign of an 
 ldc-profdata binary.  Nor does make seem to know anything 
 about and ldc-profdata target.

 What am I missing? :-\


 Sorry, my fault. It's not part of 1.0.0.


No worries.  I'll make sure that it gets added to the package 
definition once 1.1.0 is out ;-)
Aug 28 2016
prev sibling next sibling parent reply Johan Engelen <j j.nl> writes:
On Sunday, 28 August 2016 at 11:58:00 UTC, Joseph Rushton 
Wakeling wrote:

 
 apps:
   ldc2:
     command: ldc2
     plugs: [home]
   ldmd2:
     command: ldmd2
     plugs: [home]


Don't know if it's relevant here, but `ldc-profdata` should also 
be part of the package :)

-Johan
Aug 28 2016
parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Sunday, 28 August 2016 at 17:21:59 UTC, Johan Engelen wrote:

 On Sunday, 28 August 2016 at 11:58:00 UTC, Joseph Rushton 
 Wakeling wrote:

 
 apps:
   ldc2:
     command: ldc2
     plugs: [home]
   ldmd2:
     command: ldmd2
     plugs: [home]


 Don't know if it's relevant here, but `ldc-profdata` should 
 also be part of the package :)


This is another binary generated as part of the LDC build?  I ask 
because I can't find it among the binaries generated.  Is it 
something new?
Aug 28 2016
prev sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
Thanks to everyone's help and advice, this remains the last major 
hurdle to getting a snap package completed:

On Sunday, 28 August 2016 at 11:58:00 UTC, Joseph Rushton 
Wakeling wrote:

 Finally, the snap-packaged LDC will currently fail at the 
 linker stage, because (as I understand it) the snap package 
 can't access regular system resources, only stuff that is in 
 other snaps, and the core system snap doesn't include stuff 
 like gcc.  So, building falls over with an error:

     Error: failed to locate gcc

 There are a few long-term potential fixes here, including 
 creating a snap that exposes GCC or a linker, but in the short 
 term, I've been advised that probably the easiest way to create 
 a reliable standalone LDC package is to bundle the linker into 
 the snap.

 So, my question is, what would be the minimum requirements for 
 a linker to accompany ldc2, and how would I ensure that the 
 built ldc2 tries to use _that_ linker rather than some other 
 one?


... could anyone advise ... ?
Aug 28 2016
next sibling parent reply David Nadlinger via digitalmars-d-ldc <digitalmars-d-ldc puremagic.com> writes:
On 28 Aug 2016, at 21:12, Joseph Rushton Wakeling via digitalmars-d-ldc 
wrote:

 So, my question is, what would be the minimum requirements for a 
 linker to accompany ldc2 […]


 ... could anyone advise ... ?


You need a GCC-compatible linker driver, which links in the C standard 
libraries. (The latter is the reason why (g)cc is used instead of ld.) 
You might need to pull in the full GCC package if there is no other way 
to depend on a C toolchain.


 and how would I ensure that the built ldc2 tries to use _that_ linker 
 rather than some other one?




You can set the CC environment variable when invoking LDC (or use the 
hidden "-gcc" option). Otherwise, we use llvm::findProgramByName("gcc"), 
which searches the PATH.

  — David
Aug 28 2016
parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Sunday, 28 August 2016 at 20:27:21 UTC, David Nadlinger wrote:

 You need a GCC-compatible linker driver, which links in the C 
 standard libraries. (The latter is the reason why (g)cc is used 
 instead of ld.) You might need to pull in the full GCC package 
 if there is no other way to depend on a C toolchain.


Ouch!  But while that might be nasty to deal with short-term, in 
the long run it'll probably be possible to interact with a 
separate gcc snap.


 and how would I ensure that the built ldc2 tries to use 
 _that_ linker rather than some other one?




 You can set the CC environment variable when invoking LDC (or 
 use the hidden "-gcc" option). Otherwise, we use 
 llvm::findProgramByName("gcc"), which searches the PATH.


OK, thanks.  I'll see what I can come up with package-wise ...
Aug 28 2016
prev sibling parent reply kinke <noone nowhere.com> writes:
On Sunday, 28 August 2016 at 20:12:44 UTC, Joseph Rushton 
Wakeling wrote:

 how would I ensure that the built ldc2 tries to use _that_ 
 linker rather than some other one?




There's a hidden `-gcc=<pathToGcc>` switch supported by LDC. So I 
guess specifying that one in the default switches in the 
ldc2.conf file does the trick (e.g., `-gcc=%%ldcbinarypath/gcc`).
Not sure about the requirements, but shipping with both 32/64 bit 
libs would surely be nice. See 
https://github.com/ldc-developers/ldc/blob/master/driver/linker.cpp for some
default libs and switches passed to the linker.

And yes, ldc-profdata is pretty new.
Aug 28 2016
next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Sunday, 28 August 2016 at 20:30:49 UTC, kinke wrote:

 There's a hidden `-gcc=<pathToGcc>` switch supported by LDC. So 
 I guess specifying that one in the default switches in the 
 ldc2.conf file does the trick (e.g., 
 `-gcc=%%ldcbinarypath/gcc`).


Thanks, I'll look into that. :-)


 Not sure about the requirements, but shipping with both 32/64 
 bit libs would surely be nice. See 
 https://github.com/ldc-developers/ldc/blob/master/driver/linker.cpp for some
default libs and switches passed to the linker.


I agree.  Probably the first iteration of the package will just 
provide libs for the architecture it was built for, but a 
definite aim would be to have libs for multiple different 
architectures.


 And yes, ldc-profdata is pretty new.


I guess it's going to land in a stable release with v1.1 ... ?
Aug 28 2016
prev sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Sunday, 28 August 2016 at 20:30:49 UTC, kinke wrote:

 On Sunday, 28 August 2016 at 20:12:44 UTC, Joseph Rushton 
 Wakeling wrote:

 how would I ensure that the built ldc2 tries to use _that_ 
 linker rather than some other one?




 There's a hidden `-gcc=<pathToGcc>` switch supported by LDC. So 
 I guess specifying that one in the default switches in the 
 ldc2.conf file does the trick (e.g., 
 `-gcc=%%ldcbinarypath/gcc`).


Is there any way of also specifying flags to pass to gcc?  I ask 
because just including them in the -gcc flag doesn't work, e.g.:

    -gcc=ldc2.gcc --sysroot=/snap/ldc2/x1/bin/..

... results in a gcc call where the --sysroot flag is placed in a 
position that gcc doesn't accept.

I've tried specifying `-L=...` but that's translated into a gcc 
call with `-Xlinker ...` rather than just being handed to gcc 
directly.
Aug 30 2016
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Tuesday, 30 August 2016 at 23:22:12 UTC, Joseph Rushton 
Wakeling wrote:

 Is there any way of also specifying flags to pass to gcc?  I 
 ask because just including them in the -gcc flag doesn't work, 
 e.g.:

    -gcc=ldc2.gcc --sysroot=/snap/ldc2/x1/bin/..

 ... results in a gcc call where the --sysroot flag is placed in 
 a position that gcc doesn't accept.


Actually, I'm wrong about this: I think that LDC is checking to 
see if a file called `ldc2.gcc --sysroot=/snap/ldc2/x1/bin/..` 
exists, and since it doesn't find one (!), it exits with an error 
message.

Anyway, same question -- is there any way of including flags to 
pass to gcc, or is it just the gcc command itself that can be 
specified?
Aug 31 2016
parent reply David Nadlinger via digitalmars-d-ldc <digitalmars-d-ldc puremagic.com> writes:
On 31 Aug 2016, at 20:37, Joseph Rushton Wakeling via digitalmars-d-ldc 
wrote:

 Anyway, same question -- is there any way of including flags to pass 
 to gcc, or is it just the gcc command itself that can be specified?


Unfortunately, there is currently no way for doing that as far as I can 
see.

Such functionality would also be helpful for passing on GCC-style flags 
picked up from pkg-config et al. – see e.g. 
https://github.com/ldc-developers/ldc/pull/1594, although we'd need 
widespread host compiler support before we could rely on something like 
that.

For your particular use case, however – in a controlled and very 
specific situation –, wouldn't a small wrapper around the GCC binary 
to supplant the necessary arguments work just as well? If GCC doesn't 
work without any additional arguments, this is arguably a bug anyway.

  – David
Aug 31 2016
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Wednesday, 31 August 2016 at 20:24:44 UTC, David Nadlinger 
wrote:

 For your particular use case, however – in a controlled and 
 very specific situation –, wouldn't a small wrapper around the 
 GCC binary to supplant the necessary arguments work just as 
 well? If GCC doesn't work without any additional arguments, 
 this is arguably a bug anyway.


It's pretty much what I've been trying to do, but the choice of 
wrapper doesn't seem to be respected.

Is there anything problematic with this ldc2.conf that you can 
see?  Any reason why it might not be respecting the `-gcc` flag 
included in the config?
https://gist.github.com/WebDrake/229645efeca14fa54b0b1c82bcbb6477
Aug 31 2016
parent reply David Nadlinger via digitalmars-d-ldc <digitalmars-d-ldc puremagic.com> writes:
On 31 Aug 2016, at 22:24, Joseph Rushton Wakeling via digitalmars-d-ldc 
wrote:

 Is there anything problematic with this ldc2.conf that you can see?  
 Any reason why it might not be respecting the `-gcc` flag included in 
 the config?
 https://gist.github.com/WebDrake/229645efeca14fa54b0b1c82bcbb6477


LDC will still go through the usual path lookup routines for the 
argument in order to deal with PATH lookup (see gen/programs.cpp). In 
what might be regarded as a bit of a quirk, as a result the -gcc 
argument and CC global are effectively just ignored silently if they 
cannot be resolved successfully.

Are you sure the binary specified is executable, on the PATH, and so on?

  — David
Aug 31 2016
next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Wednesday, 31 August 2016 at 23:16:04 UTC, David Nadlinger 
wrote:

 LDC will still go through the usual path lookup routines for 
 the argument in order to deal with PATH lookup (see 
 gen/programs.cpp). In what might be regarded as a bit of a 
 quirk, as a result the -gcc argument and CC global are 
 effectively just ignored silently if they cannot be resolved 
 successfully.

 Are you sure the binary specified is executable, on the PATH, 
 and so on?


Yea, that sounds a distinct possibility.  I was suspecting all 
along that LDC was somehow failing "silently" and falling back to 
some default, so it's good to have it confirmed that it's what is 
likely happening here.

Thanks very much -- I _think_ this is probably the last piece of 
the puzzle (famous last words, I know:-).
Aug 31 2016
prev sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Wednesday, 31 August 2016 at 23:16:04 UTC, David Nadlinger 
wrote:

 LDC will still go through the usual path lookup routines for 
 the argument in order to deal with PATH lookup (see 
 gen/programs.cpp). In what might be regarded as a bit of a 
 quirk, as a result the -gcc argument and CC global are 
 effectively just ignored silently if they cannot be resolved 
 successfully.

 Are you sure the binary specified is executable, on the PATH, 
 and so on?


Just to confirm: will the %%ldcbinarypath%% wildcard be respected 
when passed to the `-gcc` flag?  As in e.g.

     -gcc=%%ldcbinarypath%%/../usr/bin/gcc

... ?
Sep 01 2016
next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Thursday, 1 September 2016 at 20:43:48 UTC, Joseph Rushton 
Wakeling wrote:

 On Wednesday, 31 August 2016 at 23:16:04 UTC, David Nadlinger 
 wrote:

 LDC will still go through the usual path lookup routines for 
 the argument in order to deal with PATH lookup (see 
 gen/programs.cpp). In what might be regarded as a bit of a 
 quirk, as a result the -gcc argument and CC global are 
 effectively just ignored silently if they cannot be resolved 
 successfully.

 Are you sure the binary specified is executable, on the PATH, 
 and so on?


 Just to confirm: will the %%ldcbinarypath%% wildcard be 
 respected when passed to the `-gcc` flag?  As in e.g.

     -gcc=%%ldcbinarypath%%/../usr/bin/gcc

 ... ?


To be specific: I mean when placed in the `ldc2.conf` file.
Sep 01 2016
prev sibling parent reply David Nadlinger via digitalmars-d-ldc <digitalmars-d-ldc puremagic.com> writes:
On 1 Sep 2016, at 21:43, Joseph Rushton Wakeling via digitalmars-d-ldc 
wrote:

 Just to confirm: will the %%ldcbinarypath%% wildcard be respected when 
 passed to the `-gcc` flag?


See gen/configfile.cpp: %%ldcbinarypath%% will be replaced with the 
corresponding value when extracting the switches from the config file, 
so yes, it will also be respected for -gcc.

This, of course, does not apply when -gcc (or any other switch) is 
directly passed on the command line.

  — David
Sep 01 2016
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Thursday, 1 September 2016 at 20:49:48 UTC, David Nadlinger 
wrote:

 On 1 Sep 2016, at 21:43, Joseph Rushton Wakeling via 
 digitalmars-d-ldc wrote:

 Just to confirm: will the %%ldcbinarypath%% wildcard be 
 respected when passed to the `-gcc` flag?


 See gen/configfile.cpp: %%ldcbinarypath%% will be replaced with 
 the corresponding value when extracting the switches from the 
 config file, so yes, it will also be respected for -gcc.

 This, of course, does not apply when -gcc (or any other switch) 
 is directly passed on the command line.


Sure; this is what I expected, just wanted to verify.

Anyway, I've encountered something weird with this (unless I'm 
missing something stupid).

First, is there _anything_ wrong per se with the following config 
file that would stop the -gcc flag getting through?

// This configuration file uses libconfig.
// See http://www.hyperrealm.com/libconfig/ for syntax details.

// The default group is required
default:
{
     // 'switches' holds array of string that are appends to the 
command line
     // arguments before they are parsed.
     switches = [
         "-I%%ldcbinarypath%%/../include/d/ldc",
         "-I%%ldcbinarypath%%/../include/d",
         "-L-L%%ldcbinarypath%%/../lib",
         "-defaultlib=phobos2-ldc,druntime-ldc",
         "-debuglib=phobos2-ldc-debug,druntime-ldc-debug",
         "-gcc=/snap/ldc2/x1/command-gcc.wrapper"
     ];
};

I ask because including the above in the snap package doesn't 
work, but if I _manually_ pass the -gcc flag after installing the 
snap:

     ldc2.ldmd2 -gcc=/snap/ldc2/x1/command-gcc.wrapper myprogram.d

... then this works.

That would suggest there's something about the snap-package 
containment that is making the path resolution mess up when it's 
specified inside the snap, but which is able to resolve the path 
correctly when it's given the full (from the outside system 
level) path to an executable that the containerized LDC can 
access.
Sep 01 2016
parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Thursday, 1 September 2016 at 21:14:37 UTC, Joseph Rushton 
Wakeling wrote:

 Anyway, I've encountered something weird with this (unless I'm 
 missing something stupid).


OK, I _was_ missing something stupid.  I'd included an extra -gcc 
call courtesy of the snapcraft.yaml file.

Having -gcc=%%ldcbinarypath%%/../command-gcc.wrapper provide by 
the config file _does_ work.  I'll follow up later with a more 
extensive summary of things and a working example of the snap 
package.
Sep 01 2016