digitalmars.D.ldc - MinGW-w64 runtime library inclusion (Clang-compiled) for proper Win64
- kink (73/73) Jun 03 2013 Hi guys,
- kink (5/9) Jun 03 2013 Btw, DMD v2.063 prints similar garbage in x64 mode, just in case
- kink (9/9) Jun 03 2013 PS: For those who don't know: the MinGW C runtime is just an
- kink (8/11) Jun 06 2013 Discard that, I was erroneously using a Clang 3.3 build. It works
- David Nadlinger (18/18) Jun 12 2013 Hi,
- kink (22/26) Jun 14 2013 Hi David, from what I've seen, the MinGW-w64 CRT seems to be
- kink (3/5) Jun 14 2013 Damn, I should have checked first before making such a statement;
- kink (24/28) Jun 15 2013 I've just looked into the Clang sources; the "long double"
- kink (11/11) Jun 15 2013 Of course it is an ABI issue. Since MS doesn't support the x87
- bearophile (5/7) Jun 15 2013 See the D ABI:
- David Nadlinger (3/8) Jun 15 2013 … which doesn't fully apply to LDC. ;)
- bearophile (4/5) Jun 15 2013 What's the point of not following the ABI of D reals in LDC?
- David Nadlinger (7/10) Jun 15 2013 The question should be: What's the point of DMD not following the
- kink (3/6) Jun 16 2013 For 32-bit LLVM targets that is; sizeof(long double) == 16 bytes
- kink (8/8) Jun 16 2013 Okay, I finally got the x87 argument passing to work between
Hi guys,
first of all, a big fat thank-you to David & Kai for your
dedication, it's looking good and getting better!
I recently synced the LDC sources, compiled it on my Win8 x64
system using MSVC 2012 and latest LLVM 3.4 sources and ran a few
tests (the whole procedure has gotten even less painful, thank
you very much Kai!). I soon figured out that printing reals still
results in garbage, e.g.:
<code>
import std.stdio: writefln;
import std.conv: to;
int main()
{
real r = 2;
writefln("%%g (double): %g", cast(double) r);
writefln("%%g: %g", r);
string s = to!string(r);
writefln("to!string: %s", s);
s = "2.0";
r = to!real(s);
writefln("%s parsed as %g", s, cast(double) r);
return 0;
}
</code>
outputs something like:
%g (double): 2
%g: 3.95253e-323
to!string: 4.94066e-324
2.0 parsed as 2
The problem is that the underlying calls to MSVCRT's snprintf()
pass reals as unsupported 80-bit 'long doubles' (MSVCRT doesn't
support this type at all), and yes, I know the Wiki page mentions
broken reals support. ;)
So I was wondering about how to tackle this task, considering
Phobos' std.math strongly encourages using the real type. It's
nice that we exploit LLVM intrinsics, LLVM assembly, D assembly
and D workarounds in LDC, DRuntime and Phobos to circumvent some
issues caused by Microsoft's shameful missing C99 support in
their runtime (in general, math.h just being an example). But as
you know, there are still missing functions (=> linking issues),
broken implementations and bugs.
My idea was to include the MinGW(-w64) runtime library, since
afaik MinGW mainly addresses exactly Microsoft's lacking
compatibility. I really like LDC's ability to output
MSVC-compatible objects and the usage of the MS linker, i.e. the
pretty smooth integration into the Visual-Studio-ecosystem, and
don't want to use LDC based on MinGW(-w64) and its object-file
format.
I first tried to compile a few modules of the MinGW-w64-CRT
source using VC 2012. Obviously loads of issues with incompatible
keywords, assembly syntax etc. arised and the 'long double' type
simply maps to 'double' (you may very well call that cheating),
so there's no way to get that to work unless coding everything in
naked assembly.
Then I tried to compile a few modules with Clang (latest source &
latest LLVM, compiled painlessly with MSVC 2012 as well). It
seems to swallow MinGW's code quite well - nice. I succeeded in
linking a dummy DLL (compiled with Clang and linked with MSVCRT;
Clang is supposed to have troubles with Microsoft's C++ headers,
but the C headers should work) to a D executable (compiled with
LDC, of course). So imho with MinGW's runtime (or a trimmed-down
version thereof) as additional (default?) static/dynamic library
we'd have a nice fallback for missing functions.
Unfortunately, there are ABI issues, even a minimalistic
__declspec(dllexport) int foo(int a) { return a + 3; }
in the dummy library returns garbage in x64 mode, in both LDC and
MSVC executables - but works in x86 mode, at least when called
from a dummy MSVC executable (my LDC is on a strict diet and is
allowed to spit out x64 code only). So Clang apparently doesn't
support the Win64 ABI yet, *sigh*.
Anyway, I'd like to know what you think about including a (most
likely tailored) MinGW runtime library, compiled with Clang, with
LDC for Windows to obtain max compatibility.
Jun 03 2013
"kink" <noone nowhere.com> %g (double): 2 %g: 3.95253e-323 to!string: 4.94066e-324 2.0 parsed as 2Btw, DMD v2.063 prints similar garbage in x64 mode, just in case you were wondering. The huge amount of work required to circumvent all MSVCRT shortcomings in DRuntime and Phobos is probably why we won't see official Win64 support in DMD anytime soon, if ever.
Jun 03 2013
PS: For those who don't know: the MinGW C runtime is just an additional layer on top of Microsoft's Visual C Runtime, adding missing functionality. What we need would ideally be a clean D port of missing and relevant MinGW-runtime parts into DRuntime/Phobos, for both DMD and LDC on Win64. Since that's quite a task and especially because it is only needed for a single, yet the most popular OS, my proposal is to include relevant parts of the MinGW runtime layer directly with the Win64 D compiler, so that we have a solid fallback for the time being.
Jun 03 2013
Unfortunately, there are ABI issues, even a minimalistic
__declspec(dllexport) int foo(int a) { return a + 3; }
in the dummy library returns garbage in x64 mode.
Discard that, I was erroneously using a Clang 3.3 build. It works
using a very recent 3.4 build. A similar dummy function using
"long double" doesn't work though, but that's not really
surprising.
Once these ABI issues are fixed, I think the main task would
consist in emulating all included MinGW C system headers (not
present or incomplete in MSVC) by custom ones and getting rid of
superfluous modules - I guess that'd be a manageable effort.
Jun 06 2013
Hi, Sorry for my late response, but I haven't really looked at Win64 support in LDC at all, apart from some playing around with SEH quite some while back. Thus, I was hoping that Kai would answer first – maybe he has already something up the sleeves in this area? Anyway, I'd strongly argue that any such code should be shared between any D compilers targeting Win64/MSCRT, as you mentioned as well. And the current stance of pretty much everybody on the druntime/Phobos core team is that all code has to be Boost-licensed (or similar) to be considered for inclusion in the current I haven't checked what the exact licensing situation of the MinGW runtime is (apart from some parts which I know are public domain) but I imagine there might be potential problems in this area. Thus, I'd encourage you to primarily discuss this on the upstream (druntime/Phobos) development lists. David
Jun 12 2013
I haven't checked what the exact licensing situation of the MinGW runtime is (apart from some parts which I know are public domain) but I imagine there might be potential problems in this area.Hi David, from what I've seen, the MinGW-w64 CRT seems to be entirely public domain. I got the whole thing (latest v2.0.8) to compile; a few linking issues remain, working these out wouldn't be a problem. I chose to use the MinGW-w64 headers exclusively (no MS headers) to keep things simple and basically only had to adapt a few headers in a few places after I let Clang 3.4 run in a GCC-compatible way (cmdline options: -c -nostdinc -target x86_64-pc-win32 -Xclang -cxx-abi -Xclang microsoft -fno-ms-compatibility -fmsc-version=0). The thing is that Clang maps C "long double" to double as well, just like MSVC. I'm not sure if that's generally the case when targeting Windows or only if Clang itself has been compiled with MSVC and not with MinGW. GCC's __float80 does not seem to be supported either. So I think there's probably a good reason for the dropped x87 support; I don't know about how well it is still supported in x86 hardware nowadays and how slow it is compared to scalar SSE code. So maybe the best course of action would be to map D's real type to double as well, either as default setting in LDC for Windows or at least as option. I wouldn't miss the x87 type; I just want working reals support as it's used in Phobos' std.math all over the place.
Jun 14 2013
Hi David, from what I've seen, the MinGW-w64 CRT seems to be entirely public domain.Damn, I should have checked first before making such a statement; e.g., there are modules under the Zope Public License, copyrights by Sun, Lucent etc.
Jun 14 2013
The thing is that Clang maps C "long double" to double as well, just like MSVC. I'm not sure if that's generally the case when targeting Windows or only if Clang itself has been compiled with MSVC and not with MinGW.I've just looked into the Clang sources; the "long double" mapping depends on the target triple, in this case specifically the OS component. "win32" specifies a Visual Studio target, for which "long double" is remapped to double (doesn't happen for the corresponding MinGW target). I commented out the remapping in the constructor of the VisualStudioWindowsX86_64TargetInfo class; the "long double" x87 type is now supported. A dummy function __declspec(dllexport) long double foo(long double x) { return x + 2.5; } is now compiled to (dumpbin /disasm) push rbp mov rbp,rsp sub rsp,10h fld tbyte ptr [rbp+30h] fld st(0) fstp tbyte ptr [rbp-10h] fld dword ptr [180005698h] faddp st(1),st add rsp,10h pop rbp ret which looks quite okay. Calling it from an LDC-compiled exe still doesn't work though (returns NaN, probably an ABI issue). Still, it's an improvement. ;)
Jun 15 2013
Of course it is an ABI issue. Since MS doesn't support the x87 type, it obviously isn't mentioned in the Win64 ABI docs. Clang's ABI assumes it is passed by-val on the stack whereas in LDC we treat it as a struct > 64 bits and hence pass it by-ref (a pointer to a private copy for the callee, allocated by the caller; the Win64 way of passing larger structs by-val). Clang returns an x87 in the ST(0) x87 register and LDC expects the result there - perfect. I guess I'll have to modify Clang/LLVM further to pass x87s by-ref, that seems more appropriate to me. This shouldn't be a problem for Win32 though as x87s should be passed ordinarily by-val.
Jun 15 2013
kink:This shouldn't be a problem for Win32 though as x87s should be passed ordinarily by-val.See the D ABI: http://dlang.org/abi.html Bye, bearophile
Jun 15 2013
On Saturday, 15 June 2013 at 21:37:51 UTC, bearophile wrote:kink:… which doesn't fully apply to LDC. ;) DavidThis shouldn't be a problem for Win32 though as x87s should be passed ordinarily by-val.See the D ABI: http://dlang.org/abi.html
Jun 15 2013
David Nadlinger:… which doesn't fully apply to LDC. ;)What's the point of not following the ABI of D reals in LDC? Bye, bearophile
Jun 15 2013
On Saturday, 15 June 2013 at 22:39:17 UTC, bearophile wrote:David Nadlinger:The question should be: What's the point of DMD not following the standard C ABI on Win32? ;) In the case of reals, simple things should work, but DMD packs together 80-bit reals end-to-end (i.e. 10 byte spacing), whereas LLVM chooses 12 byte alignment. David… which doesn't fully apply to LDC. ;)What's the point of not following the ABI of D reals in LDC?
Jun 15 2013
In the case of reals, simple things should work, but DMD packs together 80-bit reals end-to-end (i.e. 10 byte spacing), whereas LLVM chooses 12 byte alignment.For 32-bit LLVM targets that is; sizeof(long double) == 16 bytes for the Win64 targets, so that the required 8-bytes alignment can be enforced.
Jun 16 2013
Okay, I finally got the x87 argument passing to work between Clang and LDC after inserting a few lines into Clang's WinX86_64ABIInfo::classify method. Calling foo(666) in D returns 668.5, as expected. As soon as I've fixed the remaining linking issues for MinGW-w64 CRT (including most modules and most likely all relevant ones), I'll post the required patches for Clang and the CRT so that others can experiment too.
Jun 16 2013
I've got to say that I'm not keen on working further on this. I
got rid of the linking issues, but the library is surely not 100%
functional. My changes to the MinGW-w64 CRT are a messy hack, and
that's because the codebase itself is a mess, very ugly and
partially from the 80s and 90s of the last century.
I'd really suggest mapping the real type to the double type for
Windows targets, ideally in all D compilers of course.
Here's at least the tiny patch for Clang to enable 80-bit long
double support on Win64:
diff --git a/lib/Basic/Targets.cpp b/lib/Basic/Targets.cpp
index 4968938..f6ac502 100644
--- a/lib/Basic/Targets.cpp
+++ b/lib/Basic/Targets.cpp
-3229,8 +3229,8 class VisualStudioWindowsX86_64TargetInfo :
public WindowsX86_64TargetInfo {
public:
VisualStudioWindowsX86_64TargetInfo(const std::string& triple)
: WindowsX86_64TargetInfo(triple) {
- LongDoubleWidth = LongDoubleAlign = 64;
- LongDoubleFormat = &llvm::APFloat::IEEEdouble;
+ //LongDoubleWidth = LongDoubleAlign = 64;
+ //LongDoubleFormat = &llvm::APFloat::IEEEdouble;
}
virtual void getTargetDefines(const LangOptions &Opts,
MacroBuilder &Builder) const {
diff --git a/lib/CodeGen/TargetInfo.cpp
b/lib/CodeGen/TargetInfo.cpp
index 4fa0c3b..9fb00a5 100644
--- a/lib/CodeGen/TargetInfo.cpp
+++ b/lib/CodeGen/TargetInfo.cpp
-2584,6 +2584,11 ABIArgInfo
WinX86_64ABIInfo::classify(QualType Ty, bool IsReturnType) const {
if (Ty->isPromotableIntegerType())
return ABIArgInfo::getExtend();
+ if (!IsReturnType && //
regular function parameter
+ Ty->isRealFloatingType() && Size > 64 && //
non-complex x87 type (80 bits)
+ getTarget().getTriple().getOS() == llvm::Triple::Win32) //
Win64 Visual Studio target
+ return ABIArgInfo::getIndirect(0, /*ByVal=*/false); //
TODO: is that the right ByVal setting?
+
return ABIArgInfo::getDirect();
}
Jun 22 2013
Hey guys, I didn't want to let my work be in vain - here's the tailored MinGW-w64 CRT: https://github.com/kinke/mingw-w64-crt The Clang patch is included. I cleaned up my changes and the whole source tree; I think the result is not extremely ugly and most of the code should work. ;)
Jun 29 2013