www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.ldc - Windows --> arm-linux cross compiler and lld

reply Mike Franklin <slavo5150 yahoo.com> writes:
I'm exploring the idea of creating an LDC Windows (host) to 
arm-linux-gnueabihf (target) cross-compiler.  However, it appears 
LDC still expects GCC to be present for linking, even on Windows.

Does LDC still have a hard dependency on GCC for linking?

I suppose even without GCC, itself, LDC would still need the 
libraries that GCC pulls in automatically (libc, etc.).  Is there 
any plan to remove the dependency on the GNU toolchain altogether 
sometime in the future?  Does LLVM have its own implementations 
of libc or whatever else LDC requires from the GNU toolchain?

Thanks,
Mike
Apr 08 2018
next sibling parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Monday, 9 April 2018 at 01:04:51 UTC, Mike Franklin wrote:

 I suppose even without GCC, itself, LDC would still need the 
 libraries that GCC pulls in automatically (libc, etc.).  Is 
 there any plan to remove the dependency on the GNU toolchain 
 altogether sometime in the future?  Does LLVM have its own 
 implementations of libc or whatever else LDC requires from the 
 GNU toolchain?
Well, actually, I suppose libc, libm, libpthread, and whatever else is needed to create a binary is actually provided by the target, and not by the GNU toolchain. I suppose I could copy the necessary .a/.o/.so files to the Windows host, but is there some way to have ldc invoke lld.exe on the host, or will I have to resort to -c (separate compile and link) to avoid having to provision a windows GCC toolchain?
Apr 08 2018
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On Monday, 9 April 2018 at 01:45:38 UTC, Mike Franklin wrote:

 Well, actually, I suppose libc, libm, libpthread, and whatever 
 else is needed to create a binary is actually provided by the 
 target, and not by the GNU toolchain.  I suppose I could copy 
 the necessary .a/.o/.so files to the Windows host, but is there 
 some way to have ldc invoke lld.exe on the host, or will I have 
 to resort to -c (separate compile and link) to avoid having to 
 provision a windows GCC toolchain?
You can link with Clang instead of GCC by setting the `CC` environment variable. Clang is already a cross-compiler. Using the `-isysroot` you can specify the root directory where it tries to find libraries, includes and binaries. -- /Jacob Carlborg
Apr 08 2018
parent Jacob Carlborg <doob me.com> writes:
On Monday, 9 April 2018 at 06:59:43 UTC, Jacob Carlborg wrote:

 You can link with Clang instead of GCC by setting the `CC` 
 environment variable. Clang is already a cross-compiler. Using 
 the `-isysroot` you can specify the root directory where it 
 tries to find libraries, includes and binaries.
You can also have a look at ELLCC [1], a cross-compiler built on Clang. [1] http://ellcc.org -- /Jacob Carlborg
Apr 09 2018
prev sibling parent Kagamin <spam here.lot> writes:
On Monday, 9 April 2018 at 01:45:38 UTC, Mike Franklin wrote:
 Well, actually, I suppose libc, libm, libpthread, and whatever 
 else is needed to create a binary is actually provided by the 
 target, and not by the GNU toolchain.  I suppose I could copy 
 the necessary .a/.o/.so files to the Windows host, but is there 
 some way to have ldc invoke lld.exe on the host, or will I have 
 to resort to -c (separate compile and link) to avoid having to 
 provision a windows GCC toolchain?
You can write a small program (in D) named "gcc", that will forward arguments to lld.
Apr 11 2018
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On Monday, 9 April 2018 at 01:04:51 UTC, Mike Franklin wrote:
 I'm exploring the idea of creating an LDC Windows (host) to 
 arm-linux-gnueabihf (target) cross-compiler.  However, it 
 appears LDC still expects GCC to be present for linking, even 
 on Windows.
You can specify `-link-internally`, which at least works when not cross-compiling on Windows. There's also the `-linker` flag which allows you to specify which linker to use, but it still invokes GCC. You can use the `CC` environment variable to override the C compiler that is invoked for linking.
 Does LDC still have a hard dependency on GCC for linking?

 I suppose even without GCC, itself, LDC would still need the 
 libraries that GCC pulls in automatically (libc, etc.).  Is 
 there any plan to remove the dependency on the GNU toolchain 
 altogether sometime in the future?  Does LLVM have its own 
 implementations of libc or whatever else LDC requires from the 
 GNU toolchain?
LLVM contains replacements for most of the GNU toolchain, although it does not have its own implementation of libc. musl can be used as a replacement but only on Linux. -- /Jacob Carlborg
Apr 08 2018
prev sibling parent reply kinke <kinke libero.it> writes:
On Monday, 9 April 2018 at 01:04:51 UTC, Mike Franklin wrote:
 I'm exploring the idea of creating an LDC Windows (host) to 
 arm-linux-gnueabihf (target) cross-compiler.  However, it 
 appears LDC still expects GCC to be present for linking, even 
 on Windows.

 Does LDC still have a hard dependency on GCC for linking?
For non-Windows targets, we expect a gcc-compatible linker driver, i.e., gcc or clang, and as Jacob pointed out, you can choose between them via CC env variable (and a hidden -gcc option IIRC). A while ago, I integrated LLD into LDC itself. It's currently only able to link MSVC binaries, so `-link-internally` only works for MSVC targets. There's a PR in which I extended it to ELF and Mach-O targets too; it seems to be working after a quick test (if you have the required libs obviously), but the needed linker flags normally added by gcc makes the command-line explode: https://github.com/ldc-developers/ldc/pull/2203#issuecomment-339167131 The command-line flags could be added once to the config file though if your environment is stable.
 I suppose I could copy the necessary .a/.o/.so files to the 
 Windows host
Yep, that's what I'd do.
Apr 09 2018
parent reply Mike Franklin <slavo5150 yahoo.com> writes:
Thanks Jacob and kinke for the advice.

I've been able to piece together a Frankenstein toolchain using 
LLVM 6, ldc2-1.8.0-windows-x64, and ldc2-1.8.0-linux-armhf

I compile using ldc2-1.8.0-windows-x64:
ldc2 -c main.d

I create a samba share on my ARM device use it as sysroot from 
the Windows host at \\192.168.0.26\root.

I pull the `lib` folder out of ldc2-1.8.0-linux-armhf for the 
following linking task.

I link with something like this (pieced together from my notes; 
may not be perfect):
ld.lld.exe -static -m armelf_linux_eabi -o main
-sysroot \\192.168.0.26\root
\\192.168.0.26\root\usr\lib\arm-linux-gnueabihf\crt1.o
\\192.168.0.26\root\usr\lib\arm-linux-gnueabihf\crti.o
\\192.168.0.26\root\usr\lib\arm-linux-gnueabihf\crtbegin.o
-L \\192.168.0.26\root\usr\lib\arm-linux-gnueabihf
-L \\192.168.0.26\root\usr\lib\gcc\arm-linux-gnueabihf\4.9.2
main.o
\\192.168.0.26\root\usr\lib\arm-linux-gnueabihf\crtend.o
\\192.168.0.26\root\usr\lib\arm-linux-gnueabihf\crtn.o
-L C:\ldc2-1.8.0-linux-armhf\lib -lphobos2-ldc -ldruntime-ldc
-lrt -ldl -lpthread -lm -lc -lgcc -lgcc_eh

It works.

I couldn't get linking with clang or separate dynamic linking to 
work; my binary ends up throwing a segmentation fault, but I 
suspect I could probably troubleshoot and work out what I need to 
do eventually.

 A while ago, I integrated LLD into LDC itself. It's currently 
 only able to link MSVC binaries, so `-link-internally` only 
 works for MSVC targets.
 There's a PR in which I extended it to ELF and Mach-O targets 
 too; it seems to be working after a quick test (if you have the 
 required libs obviously), but the needed linker flags normally 
 added by gcc makes the command-line explode: 
 https://github.com/ldc-developers/ldc/pull/2203#issuecomment-339167131

 The command-line flags could be added once to the config file 
 though if your environment is stable.
That looks really nice. I see you had to patch lld; is that going upstream? How does clang know how to generate its linker command? Whatever it is, can ldc be made do the same? Mike
Apr 10 2018
next sibling parent Mike Franklin <slavo5150 yahoo.com> writes:
On Wednesday, 11 April 2018 at 00:39:23 UTC, Mike Franklin wrote:

 I compile using ldc2-1.8.0-windows-x64:
 ldc2 -c main.d
Oops! forgot the most important part; the triple. It should be something like this: ldc2 -triple=arm-linux-gnueabihf -c main.d Mike
Apr 10 2018
prev sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Wednesday, 11 April 2018 at 00:39:23 UTC, Mike Franklin wrote:
 How does clang know how to generate its linker command?
I don't think it does, it's probably hard-coded based on the target, though you can usually override various elements with flags. Since every target has its own selection of flags and object files it needs, we delegate linking to the C (cross-)compiler for that target. You can always choose your own linker for ldc with the -linker= flag and do everything manually if you want though.
 Whatever it is, can ldc be made do the same?
Yes, but it would be an impossible task to do it for any non-trivial set of platforms. It's _much_ easier to free-ride off the C compiler which has already been configured for the target.
Apr 12 2018
parent reply Joakim <dlang joakim.fea.st> writes:
On Friday, 13 April 2018 at 03:56:24 UTC, Joakim wrote:
 On Wednesday, 11 April 2018 at 00:39:23 UTC, Mike Franklin 
 wrote:
 How does clang know how to generate its linker command?
I don't think it does, it's probably hard-coded based on the target, though you can usually override various elements with flags. Since every target has its own selection of flags and object files it needs, we delegate linking to the C (cross-)compiler for that target. You can always choose your own linker for ldc with the -linker= flag and do everything manually if you want though.
 Whatever it is, can ldc be made do the same?
Yes, but it would be an impossible task to do it for any non-trivial set of platforms. It's _much_ easier to free-ride off the C compiler which has already been configured for the target.
Btw, you may be interested in this thread about using the llvm linker, lld, which delves into a lot of the issues involved with not delegating to the C compiler: https://github.com/ldc-developers/ldc/issues/2028
Apr 16 2018
parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Tuesday, 17 April 2018 at 06:53:59 UTC, Joakim wrote:

 Btw, you may be interested in this thread about using the llvm 
 linker, lld, which delves into a lot of the issues involved 
 with not delegating to the C compiler:

 https://github.com/ldc-developers/ldc/issues/2028
Thanks, it was an interesting read. IMO, however, all of this dependency and C (C runtime, C standard library, C compiler, etc...) is all superficial. With a little more due diligence in the compiler and runtime implementations, none of it would be necessary. I illustrate with the following example. It is a gross oversimplification but demonstrates that D doesn't need C...any of it. ---object.d module object; alias immutable(char)[] string; private long __d_sys_write(long arg1, in void* arg2, long arg3) { long result; asm { mov RAX, 1; mov RDI, arg1; mov RSI, arg2; mov RDX, arg3; syscall; } return result; } void write(string text) { __d_sys_write(2, text.ptr, text.length); } private void __d_sys_exit(long arg1) { asm { mov RAX, 60; mov RDI, arg1; syscall; } } extern void main(); private extern(C) void _start() { main(); __d_sys_exit(0); } ---main.d module main; void main() { write("Hello, World\n"); } $dmd -c -lib -conf= object.d main.d -of=main.o $ld main.o -o main $size main text data bss dec hex filename 176 0 0 176 b0 main $main Hello, World I admit that without leveraging C (actually gcc, even llvm still needs the crt files from gcc) D would be assuming more work, but I think doing away with the C compiler and going directly to the linker is a step in the right direction. Mike
Apr 17 2018
parent reply Jacob Carlborg <doob me.com> writes:
On 2018-04-17 10:55, Mike Franklin wrote:

 IMO, however, all of this dependency and C (C runtime, C standard 
 library, C compiler, etc...) is all superficial.  With a little more due 
 diligence in the compiler and runtime implementations, none of it would 
 be necessary.
I agree. That's one case where Go shines. It makes cross-compiling so much simpler. -- /Jacob Carlborg
Apr 18 2018
parent reply kinke <noone nowhere.com> writes:
On Wednesday, 18 April 2018 at 19:08:50 UTC, Jacob Carlborg wrote:
 On 2018-04-17 10:55, Mike Franklin wrote:

 IMO, however, all of this dependency and C (C runtime, C 
 standard library, C compiler, etc...) is all superficial.  
 With a little more due diligence in the compiler and runtime 
 implementations, none of it would be necessary.
I agree. That's one case where Go shines. It makes cross-compiling so much simpler.
I don't see a huge problem for the compiler (except for quite a bunch of LLVM 'intrinsics' boiling down to a C function call, so LLVM expects some core C libs to be linked in as well - so these C symbols would need to be provided by a D lib). Basing druntime on the C runtime layer, abstracting away many OS differences, makes things considerably simpler though; with the current manpower, I see no way to get rid of it and re-implement everything in D, while still supporting all platforms (and new platforms to come with as little hassle as possible). OTOH, libunwind (or for MSVC, the Visual C++ EH personality function used by LDC), libpthread, the math functions (which I'd really like to see all in pure D), I/O, memory management... Considering that we'll be stuck with core.stdc anyway, getting rid of the C dependencies won't be feasible for a very long time IMO. A cross-compiler with integrated cross-linking ability (and a tool to cross-compile druntime and Phobos), shipping with ready-made C libs and a preconfigured config file in a Docker image, working just out of the box should be a great experience for the user (and the Docker image generation easily automatable for us devs). I've never used Go, so how can that experience be further improved?
Apr 18 2018
next sibling parent kinke <noone nowhere.com> writes:
On Wednesday, 18 April 2018 at 21:07:01 UTC, kinke wrote:
 A cross-compiler with integrated cross-linking ability (and a 
 tool to cross-compile druntime and Phobos), shipping with 
 ready-made C libs and a preconfigured config file in a Docker 
 image, working just out of the box should be a great experience 
 for the user (and the Docker image generation easily 
 automatable for us devs).
It'd probably be quite cool to merge all prebuilt packages into a single universal Docker image (augmented by each platform's C libs & object files and the ldc2.conf config file extended by a section for each target triple). That would currently allow to freely cross-compile to 32/64-bit Windows/Linux/macOS and 32-bit ARM, simply depending on the used `-mtriple=...`. Legal aspects would need to be clarified first though; just saying that something like this wouldn't actually be that hard to do & automate.
Apr 18 2018
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2018-04-18 23:07, kinke wrote:

 I don't see a huge problem for the compiler (except for quite a bunch of 
 LLVM 'intrinsics' boiling down to a C function call, so LLVM expects 
 some core C libs to be linked in as well - so these C symbols would need 
 to be provided by a D lib).
 
 Basing druntime on the C runtime layer, abstracting away many OS 
 differences, makes things considerably simpler though; with the current 
 manpower, I see no way to get rid of it and re-implement everything in 
 D, while still supporting all platforms (and new platforms to come with 
 as little hassle as possible). OTOH, libunwind (or for MSVC, the Visual 
 C++ EH personality function used by LDC), libpthread, the math functions 
 (which I'd really like to see all in pure D), I/O, memory management...
Originally Tango, which druntime is based, only used syscalls and (as far as I know) a C main function and everything necessary for that. But druntime has evolved quite a lot since then with new features and more platforms.
 I've never used Go, so how can that experience be further improved?
The standard Go compiler is a cross-compiler. They use a custom linker which is a cross-linker as well. In Go you cannot directly link to a C function in the same way as you can in D. Go also requires (unless you want to mess with the whole reason to use Go) that every call is non-blocking. That means they need to reimplement most things and cannot rely on the C standard library. Therefore they don't and instead only using syscalls as the most basic layer. Cross-compiling using Go is as simple as: GOOS=linux GOARCH=arm go build main.go Since the compiler and linker are cross-tools and they're not relaying on any platform SDK it just works if you have the standard Go package. No issues with trying to find a cross-linker, no issues of the SDK cannot be redistributed for macOS or Windows. It will even generate statically linked binaries on macOS which the C standard library doesn't even support. -- /Jacob Carlborg
Apr 19 2018