www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Cross compilers; GDC/LDC

reply Manu <turkeyman gmail.com> writes:
Hi people.

So I've been AWOL a while, and I'll still be intermittent for some weeks
yet, but I'm trying to work on std.simd and also my dconf talks in any gaps
I can.
However, in order to do a thorough job, I'm missing some tools. I wonder if
anyone can help me out...?

I need to do some rather strenuous testing of SIMD code gen in D, to make
sure I don't make mistakes and/or lie.

This requires testing against as many compilers+architectures as I can.
I have barely any internet access to speak of, I don't really know how, and
I also don't run linux, so I can't practically build myself a suite of
toolchains to work with >_<

Ideally, I would really like all these toolchains:
 GDC: x86, x64, ARM, PPC, MIPS, SH4 <- common toolchains
 LDC: x86, x64, ARM, PPC <- used by apple

I wonder if anyone with know-how building GDC or LDC for windows could help
me out by producing as many of these in binary as possible?

As a side note, I think it would be really valuable to keep a cache of such
windows binaries somewhere on the net anyway, so anyone working on foreign
platforms can get to work and actually focus on platform support, without
wasting all their time learning how to build these toolchains.

I think Daniel Green who usually builds the windows GDC toolchains actually
builds them from linux? (which seems weird to me)

Cheers people!
Mar 03 2013
next sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Sun, 3 Mar 2013 23:33:27 +1000
schrieb Manu <turkeyman gmail.com>:

 Hi people.
 
 So I've been AWOL a while, and I'll still be intermittent for some
 weeks yet, but I'm trying to work on std.simd and also my dconf talks
 in any gaps I can.
 However, in order to do a thorough job, I'm missing some tools. I
 wonder if anyone can help me out...?
 
 I need to do some rather strenuous testing of SIMD code gen in D, to
 make sure I don't make mistakes and/or lie.
 
 This requires testing against as many compilers+architectures as I
 can. I have barely any internet access to speak of, I don't really
 know how, and I also don't run linux, so I can't practically build
 myself a suite of toolchains to work with >_<
 
 Ideally, I would really like all these toolchains:
  GDC: x86, x64, ARM, PPC, MIPS, SH4 <- common toolchains
  LDC: x86, x64, ARM, PPC <- used by apple
 
 I wonder if anyone with know-how building GDC or LDC for windows
 could help me out by producing as many of these in binary as possible?
 
 As a side note, I think it would be really valuable to keep a cache
 of such windows binaries somewhere on the net anyway, so anyone
 working on foreign platforms can get to work and actually focus on
 platform support, without wasting all their time learning how to
 build these toolchains.
 
 I think Daniel Green who usually builds the windows GDC toolchains
 actually builds them from linux? (which seems weird to me)
 
 Cheers people!
 

I could build the gdc toolchains, but I need some more information: What OS should the toolchains target? Linux, windows or "bare-metal"? (Especially the x86 toolchains) Do you need libdruntime/libphobos or is a plain compiler enough? I'm not sure if druntime compiles for SH4, MIPS and PPC.
Mar 03 2013
next sibling parent reply Manu <turkeyman gmail.com> writes:
x86 and x64 are already available. I've seen ARM builds kicking around too
which I've already done some work with. I'm not sure if they're up to date,
but it's probably fine for me to use.

Linux or bare metal + no library for the others will suit me now.
Whatever's easiest.

Google seem to have a build script for building their Android ARM and MIPS
toolchains:
http://developer.android.com/tools/sdk/ndk/index.html
I wonder if the GDC frontend can be dropped in to their scripts easily?

I guess the most important one in that case is PPC, since VMX is really
standard, it's a very common arch, and there's no existing toolchains for
it.

I remember talk of a build-server that automatically rebuilds binaries of
many toolchains each time a new release of D comes out some time back, did
that lead anywhere?

In the future, it'd be great to have a more complete set of offerings
linked from http://dlang.org/download.html


On 4 March 2013 03:20, Johannes Pfau <nospam example.com> wrote:

 Am Sun, 3 Mar 2013 23:33:27 +1000
 schrieb Manu <turkeyman gmail.com>:

 Hi people.

 So I've been AWOL a while, and I'll still be intermittent for some
 weeks yet, but I'm trying to work on std.simd and also my dconf talks
 in any gaps I can.
 However, in order to do a thorough job, I'm missing some tools. I
 wonder if anyone can help me out...?

 I need to do some rather strenuous testing of SIMD code gen in D, to
 make sure I don't make mistakes and/or lie.

 This requires testing against as many compilers+architectures as I
 can. I have barely any internet access to speak of, I don't really
 know how, and I also don't run linux, so I can't practically build
 myself a suite of toolchains to work with >_<

 Ideally, I would really like all these toolchains:
  GDC: x86, x64, ARM, PPC, MIPS, SH4 <- common toolchains
  LDC: x86, x64, ARM, PPC <- used by apple

 I wonder if anyone with know-how building GDC or LDC for windows
 could help me out by producing as many of these in binary as possible?

 As a side note, I think it would be really valuable to keep a cache
 of such windows binaries somewhere on the net anyway, so anyone
 working on foreign platforms can get to work and actually focus on
 platform support, without wasting all their time learning how to
 build these toolchains.

 I think Daniel Green who usually builds the windows GDC toolchains
 actually builds them from linux? (which seems weird to me)

 Cheers people!

I could build the gdc toolchains, but I need some more information: What OS should the toolchains target? Linux, windows or "bare-metal"? (Especially the x86 toolchains) Do you need libdruntime/libphobos or is a plain compiler enough? I'm not sure if druntime compiles for SH4, MIPS and PPC.

Mar 03 2013
parent reply Johannes Pfau <nospam example.com> writes:
Am Mon, 4 Mar 2013 10:36:45 +1000
schrieb Manu <turkeyman gmail.com>:

 x86 and x64 are already available. I've seen ARM builds kicking
 around too which I've already done some work with. I'm not sure if
 they're up to date, but it's probably fine for me to use.

I included a new ARM build. Unfortunately ARM druntime broke when upgrading to 2.062. We already had working cross-compilers with phobos for ARM (raspberry pi).
 
 Linux or bare metal + no library for the others will suit me now.
 Whatever's easiest.

OK, then linux
 
 Google seem to have a build script for building their Android ARM and
 MIPS toolchains:
 http://developer.android.com/tools/sdk/ndk/index.html
 I wonder if the GDC frontend can be dropped in to their scripts
 easily?

Unless you actually need android support building a crosstool-NG toolchain is easier.
 I guess the most important one in that case is PPC, since VMX is
 really standard, it's a very common arch, and there's no existing
 toolchains for it.
 
 I remember talk of a build-server that automatically rebuilds
 binaries of many toolchains each time a new release of D comes out
 some time back, did that lead anywhere?
 
 In the future, it'd be great to have a more complete set of offerings
 linked from http://dlang.org/download.html

Once we have working crosstool-NG configurations updating the cross compilers is trivial. I should set up a qemu image though so others can easily build the packages as well. However I think a bare compiler without druntime is only useful for very few people. I've uploaded the toolchains now. I verified that they can compile a simple program, but I don't have hardware to test if the result actually works. Also attached a simple hello world. I guess you know this already but you have to compile with -nophoboslib and make sure that gdc can find the object.di file. The included object.di was just copied from my local installation. It's not really used but the frontend does some sanity checks on object.di now so an empty file doesn't work anymore. Also make sure _Dmodule_ref is defined as in the test.d example file. Some information about the toolchains: GCC 4.7.2 GDC b8f5c22b0e7afa7e68a287ed788597e783540063 (2.062 frontend) Endianness: Whatever was the default in ct-NG (All version identifiers from http://dlang.org/version.html should work.) It's sometimes possible to change the endianness without recompiling gcc (e.g. for MIPS -EB and -EL flags.). I'm not sure if this works though as the C libraries are compiled with default endianness. All toolchains target 32bit processors. If you need 64bit targets just tell me. Multilib is not enabled. http://www.mediafire.com/?b8pgyvbotbgpd And here's an old release for Raspbian with druntime & phobos: http://www.mediafire.com/?00d05h1gox9c50h
Mar 04 2013
parent reply Manu <turkeyman gmail.com> writes:
Wowzer! That's awesome! :)

So, I don't understand this whole crosstools-NG thing, how did you do that
so easily?

So, aside from doing these tests I'm working on now, let's say I were
working on some games console projects, how hard is it to integrate the
console patches (like the ones linked in my prior email) into this build
system to make actual working compilers for those platforms?
This seems like an amazing tool! But it'd be really awesome if it
maintained targets for games consoles too...


On 4 March 2013 22:57, Johannes Pfau <nospam example.com> wrote:

 Am Mon, 4 Mar 2013 10:36:45 +1000
 schrieb Manu <turkeyman gmail.com>:

 x86 and x64 are already available. I've seen ARM builds kicking
 around too which I've already done some work with. I'm not sure if
 they're up to date, but it's probably fine for me to use.

I included a new ARM build. Unfortunately ARM druntime broke when upgrading to 2.062. We already had working cross-compilers with phobos for ARM (raspberry pi).
 Linux or bare metal + no library for the others will suit me now.
 Whatever's easiest.

OK, then linux
 Google seem to have a build script for building their Android ARM and
 MIPS toolchains:
 http://developer.android.com/tools/sdk/ndk/index.html
 I wonder if the GDC frontend can be dropped in to their scripts
 easily?

Unless you actually need android support building a crosstool-NG toolchain is easier.
 I guess the most important one in that case is PPC, since VMX is
 really standard, it's a very common arch, and there's no existing
 toolchains for it.

 I remember talk of a build-server that automatically rebuilds
 binaries of many toolchains each time a new release of D comes out
 some time back, did that lead anywhere?

 In the future, it'd be great to have a more complete set of offerings
 linked from http://dlang.org/download.html

Once we have working crosstool-NG configurations updating the cross compilers is trivial. I should set up a qemu image though so others can easily build the packages as well. However I think a bare compiler without druntime is only useful for very few people. I've uploaded the toolchains now. I verified that they can compile a simple program, but I don't have hardware to test if the result actually works. Also attached a simple hello world. I guess you know this already but you have to compile with -nophoboslib and make sure that gdc can find the object.di file. The included object.di was just copied from my local installation. It's not really used but the frontend does some sanity checks on object.di now so an empty file doesn't work anymore. Also make sure _Dmodule_ref is defined as in the test.d example file. Some information about the toolchains: GCC 4.7.2 GDC b8f5c22b0e7afa7e68a287ed788597e783540063 (2.062 frontend) Endianness: Whatever was the default in ct-NG (All version identifiers from http://dlang.org/version.html should work.) It's sometimes possible to change the endianness without recompiling gcc (e.g. for MIPS -EB and -EL flags.). I'm not sure if this works though as the C libraries are compiled with default endianness. All toolchains target 32bit processors. If you need 64bit targets just tell me. Multilib is not enabled. http://www.mediafire.com/?b8pgyvbotbgpd And here's an old release for Raspbian with druntime & phobos: http://www.mediafire.com/?00d05h1gox9c50h

Mar 04 2013
parent Johannes Pfau <nospam example.com> writes:
Am Tue, 5 Mar 2013 00:16:08 +1000
schrieb Manu <turkeyman gmail.com>:

 Wowzer! That's awesome! :)
 
 So, I don't understand this whole crosstools-NG thing, how did you do
 that so easily?
 
 So, aside from doing these tests I'm working on now, let's say I were
 working on some games console projects, how hard is it to integrate
 the console patches (like the ones linked in my prior email) into
 this build system to make actual working compilers for those
 platforms? This seems like an amazing tool! But it'd be really
 awesome if it maintained targets for games consoles too...
 

Crosstool-NG is a set of scripts & patches to built gcc cross compilers. It includes patches for common build problems, a nice GUI to configure the build and allows to build complicated cross compiler configurations (like building a toolchain which targets C (e.g. ARM), runs on B (e.g. Windows) and is compiled on B (e.g. Linux)). (The important part is that it's maintained and used by many people so it has patches for most of the problems you can run into. For example I ran into this problem when building devkitPro: http://crosstool-ng.org/hg/crosstool-ng/rev/372245834071) It can also build additional useful tools and libraries (like gdb, strace, ...) Integrating D is very easy, here are some instructions: http://gdcproject.org/wiki/Cross%20Compiler/crosstool-NG Adding console toolchains might be a little more difficult. Adding the GCC/binutils patches is simple (just copy them into the right folder), but you have to be careful making sure that gcc is built with exactly the same options as the current console toolchain (certainly doable). I'm not sure how good the newlib support is in crosstool-NG though, adding rules for newlib might be some extra work. But if I was a devkitPro developer I'd try to move the toolchain to crosstool-NG as it really simplifies building cross compilers.
Mar 06 2013
prev sibling parent Iain Buclaw <ibuclaw ubuntu.com> writes:
On Mar 4, 2013 12:38 AM, "Manu" <turkeyman gmail.com> wrote:
 x86 and x64 are already available. I've seen ARM builds kicking around

too which I've already done some work with. I'm not sure if they're up to date, but it's probably fine for me to use.
 Linux or bare metal + no library for the others will suit me now.

Whatever's easiest.
 Google seem to have a build script for building their Android ARM and

MIPS toolchains:
 http://developer.android.com/tools/sdk/ndk/index.html
 I wonder if the GDC frontend can be dropped in to their scripts easily?

 I guess the most important one in that case is PPC, since VMX is really

standard, it's a very common arch, and there's no existing toolchains for it.
 I remember talk of a build-server that automatically rebuilds binaries of

many toolchains each time a new release of D comes out some time back, did that lead anywhere?
 In the future, it'd be great to have a more complete set of offerings

linked from http://dlang.org/download.html
 On 4 March 2013 03:20, Johannes Pfau <nospam example.com> wrote:
 Am Sun, 3 Mar 2013 23:33:27 +1000
 schrieb Manu <turkeyman gmail.com>:

 Hi people.

 So I've been AWOL a while, and I'll still be intermittent for some
 weeks yet, but I'm trying to work on std.simd and also my dconf talks
 in any gaps I can.
 However, in order to do a thorough job, I'm missing some tools. I
 wonder if anyone can help me out...?

 I need to do some rather strenuous testing of SIMD code gen in D, to
 make sure I don't make mistakes and/or lie.

 This requires testing against as many compilers+architectures as I
 can. I have barely any internet access to speak of, I don't really
 know how, and I also don't run linux, so I can't practically build
 myself a suite of toolchains to work with >_<

 Ideally, I would really like all these toolchains:
  GDC: x86, x64, ARM, PPC, MIPS, SH4 <- common toolchains
  LDC: x86, x64, ARM, PPC <- used by apple

 I wonder if anyone with know-how building GDC or LDC for windows
 could help me out by producing as many of these in binary as possible?

 As a side note, I think it would be really valuable to keep a cache
 of such windows binaries somewhere on the net anyway, so anyone
 working on foreign platforms can get to work and actually focus on
 platform support, without wasting all their time learning how to
 build these toolchains.

 I think Daniel Green who usually builds the windows GDC toolchains
 actually builds them from linux? (which seems weird to me)

 Cheers people!

I could build the gdc toolchains, but I need some more information: What OS should the toolchains target? Linux, windows or "bare-metal"? (Especially the x86 toolchains) Do you need libdruntime/libphobos or is a plain compiler enough? I'm not sure if druntime compiles for SH4, MIPS and PPC.


I did try testing this with QEMU, however the only reliable target that can build gcc is ARM under emulation. The best method of doing this would be to have physical kit either donated or bought through donations that can set-up as port boxes for anyone who wants to port their D application over to architecture X. Regards -- Iain Buclaw *(p < e ? p++ : p) = (c & 0x0f) + '0';
Mar 04 2013
prev sibling parent reply "David Nadlinger" <see klickverbot.at> writes:
On Sunday, 3 March 2013 at 13:57:15 UTC, Manu wrote:
 Ideally, I would really like all these toolchains:
  GDC: x86, x64, ARM, PPC, MIPS, SH4 <- common toolchains
  LDC: x86, x64, ARM, PPC <- used by apple

 I wonder if anyone with know-how building GDC or LDC for 
 windows could help
 me out by producing as many of these in binary as possible?

My dev laptop is being repaired right now, but I hope I'll be able to upload a recent MinGW build of LDC by the end of the week. I could build LLVM with all targets enabled, so you could at least do some basic codegen tests by manually specifying the target triple. See http://wiki.dlang.org/LDC for information on the ARM/PPC status. You might want to contact Kai directly, maybe he can easily provide you with a "real" PPC64 cross-compiler.
 As a side note, I think it would be really valuable to keep a 
 cache of such
 windows binaries somewhere on the net anyway, so anyone working 
 on foreign
 platforms can get to work and actually focus on platform 
 support, without
 wasting all their time learning how to build these toolchains.

In case of LDC, the main reason such binaries don't already exist, at least for the bare compiler, is that no Windows binaries exist at all (a MinGW preview will hopefully be available with the next release). Keep in mind, though, that any serious porting effort will likely require changes to the compiler source (e.g. custom ABI transformations), so the benefit of binaries for this specific use case will probably be limited. David
Mar 04 2013
parent reply "Kai Nacke" <kai redstar.de> writes:
On Monday, 4 March 2013 at 13:29:34 UTC, David Nadlinger wrote:
 On Sunday, 3 March 2013 at 13:57:15 UTC, Manu wrote:
 Ideally, I would really like all these toolchains:
 GDC: x86, x64, ARM, PPC, MIPS, SH4 <- common toolchains
 LDC: x86, x64, ARM, PPC <- used by apple

See http://wiki.dlang.org/LDC for information on the ARM/PPC status. You might want to contact Kai directly, maybe he can easily provide you with a "real" PPC64 cross-compiler.

Hi Manu, I uploaded a Win64-version of LDC to http://www.redstar.de/ldc/LDC.zip (md5: 5ae2c1c4b2c6e94e81b930b668a02cc4). It contains this version and targets: LDC - the LLVM D compiler (trunk): based on DMD v2.061 and LLVM 3.3svn Default target: x86_64-pc-win32 Host CPU: corei7 http://dlang.org - http://wiki.dlang.org/LDC Registered Targets: arm - ARM mips - Mips mips64 - Mips64 [experimental] mips64el - Mips64el [experimental] mipsel - Mipsel ppc32 - PowerPC 32 ppc64 - PowerPC 64 thumb - Thumb x86 - 32-bit X86: Pentium-Pro and above x86-64 - 64-bit X86: EM64T and AMD64 Maybe this is useful for you. Kai
Mar 09 2013
parent Manu <turkeyman gmail.com> writes:
On 10 March 2013 03:58, Kai Nacke <kai redstar.de> wrote:

 On Monday, 4 March 2013 at 13:29:34 UTC, David Nadlinger wrote:

 On Sunday, 3 March 2013 at 13:57:15 UTC, Manu wrote:

 Ideally, I would really like all these toolchains:
 GDC: x86, x64, ARM, PPC, MIPS, SH4 <- common toolchains
 LDC: x86, x64, ARM, PPC <- used by apple

See http://wiki.dlang.org/LDC for information on the ARM/PPC status. You might want to contact Kai directly, maybe he can easily provide you with a "real" PPC64 cross-compiler.

Hi Manu, I uploaded a Win64-version of LDC to http://www.redstar.de/ldc/LDC.**zip<http://www.redstar.de/ldc/LDC.zip>(md5: 5ae2c1c4b2c6e94e81b930b668a02c **c4). It contains this version and targets: LDC - the LLVM D compiler (trunk): based on DMD v2.061 and LLVM 3.3svn Default target: x86_64-pc-win32 Host CPU: corei7 http://dlang.org - http://wiki.dlang.org/LDC Registered Targets: arm - ARM mips - Mips mips64 - Mips64 [experimental] mips64el - Mips64el [experimental] mipsel - Mipsel ppc32 - PowerPC 32 ppc64 - PowerPC 64 thumb - Thumb x86 - 32-bit X86: Pentium-Pro and above x86-64 - 64-bit X86: EM64T and AMD64 Maybe this is useful for you. Kai

Awesome Kai! Thanks so much! I think that covers the tools I need. Now I just need to find some down time to put it all together >_<
Mar 10 2013