digitalmars.D.learn - Need help with Windows linkage ( DMD using ImportC)
- Carl Sturtivant (29/29) Mar 04 ```
- Carl Sturtivant (21/37) Mar 04 I forced linkage of these unused symbols as follows, but it would
- Gregor =?UTF-8?B?TcO8Y2ts?= (13/50) Mar 07 This looks like a combination of two issues:
- Carl Sturtivant (6/17) Mar 09 Onecore: not sure what I did to add that dependency. Two symbols
- Carl Sturtivant (51/57) Mar 10 Thanks for the explanation, on the strength of which I found a
- Carl Sturtivant (44/44) Mar 20 I found a way to make a solution for 64 bit Windows mechanically
``` C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.33.31629\bin\Hostx64\x64\link.exe ``` ``` DMD64 D Compiler v2.107.0 ``` I'm doing some experimentation with ImportC on Windows, and getting this when linking. ``` blah.obj: error LNK2019: unresolved external symbol _mul128 referenced in function MultiplyExtract128 blah.obj: error LNK2019: unresolved external symbol __shiftright128 referenced in function MultiplyExtract128 blah.obj: error LNK2019: unresolved external symbol _umul128 referenced in function UnsignedMultiplyExtract128 blah.obj: error LNK2019: unresolved external symbol __stosb referenced in function RtlSecureZeroMemory blah.obj: error LNK2019: unresolved external symbol __readgsqword referenced in function NtCurrentTeb blah.obj: error LNK2019: unresolved external symbol __imp_MapViewOfFileNuma2 referenced in function MapViewOfFile2 blah.obj: error LNK2019: unresolved external symbol __imp_CharUpperW referenced in function ua_CharUpperW ``` The details don't seem to matter much, windows headers #included in C files compiled in with ImportC bring these sorts of things in it seems, and I don't know how to link them, or exclude their declarations with a #define before I #include such.
Mar 04
On Monday, 4 March 2024 at 21:21:20 UTC, Carl Sturtivant wrote:``` blah.obj: error LNK2019: unresolved external symbol _mul128 referenced in function MultiplyExtract128 blah.obj: error LNK2019: unresolved external symbol __shiftright128 referenced in function MultiplyExtract128 blah.obj: error LNK2019: unresolved external symbol _umul128 referenced in function UnsignedMultiplyExtract128 blah.obj: error LNK2019: unresolved external symbol __stosb referenced in function RtlSecureZeroMemory blah.obj: error LNK2019: unresolved external symbol __readgsqword referenced in function NtCurrentTeb blah.obj: error LNK2019: unresolved external symbol __imp_MapViewOfFileNuma2 referenced in function MapViewOfFile2 blah.obj: error LNK2019: unresolved external symbol __imp_CharUpperW referenced in function ua_CharUpperW ```I forced linkage of these unused symbols as follows, but it would be nice to have a clean way to proceed. ```D extern(C) { int _InterlockedExchangeAdd(int* Addend, int Value) { return 0; }; long _InterlockedExchangeAdd64(long* Addend, long Value) { return 0; } void _mul128() {}; void __shiftright128() {}; void _umul128() {}; void __stosb() {}; void __readgsqword() {}; void __imp_MapViewOfFileNuma2() {}; void __imp_CharUpperW() {}; } ``` I got the D signatures of the first two so as to generate the correct linkage by using ImportC to translate the inclusion of `Windows.h` into a .di file, and searching.
Mar 04
On Tuesday, 5 March 2024 at 00:20:36 UTC, Carl Sturtivant wrote:On Monday, 4 March 2024 at 21:21:20 UTC, Carl Sturtivant wrote:This looks like a combination of two issues: 1. Missing import libraries for Win32 API functions. Anything starting with `__imp_` is a symbol that should be provided by a DLL import library. MapViewOfFileNuma2 for example is provided by onecore.lib in the Windows SDK, according to Microsoft documentation. 2. C code referring to MSVC-specific compiler intrinsics. At least InterlockedExchangeAdd, InterlockedExchangeAdd64 and _stosb are such intrinsics. This is harder to resolve. There are two ways forward here: either implement a shim function that replicates the intrinsic's functionality if possible or add support for these intrinsics to DMD.``` blah.obj: error LNK2019: unresolved external symbol _mul128 referenced in function MultiplyExtract128 blah.obj: error LNK2019: unresolved external symbol __shiftright128 referenced in function MultiplyExtract128 blah.obj: error LNK2019: unresolved external symbol _umul128 referenced in function UnsignedMultiplyExtract128 blah.obj: error LNK2019: unresolved external symbol __stosb referenced in function RtlSecureZeroMemory blah.obj: error LNK2019: unresolved external symbol __readgsqword referenced in function NtCurrentTeb blah.obj: error LNK2019: unresolved external symbol __imp_MapViewOfFileNuma2 referenced in function MapViewOfFile2 blah.obj: error LNK2019: unresolved external symbol __imp_CharUpperW referenced in function ua_CharUpperW ```I forced linkage of these unused symbols as follows, but it would be nice to have a clean way to proceed. ```D extern(C) { int _InterlockedExchangeAdd(int* Addend, int Value) { return 0; }; long _InterlockedExchangeAdd64(long* Addend, long Value) { return 0; } void _mul128() {}; void __shiftright128() {}; void _umul128() {}; void __stosb() {}; void __readgsqword() {}; void __imp_MapViewOfFileNuma2() {}; void __imp_CharUpperW() {}; } ``` I got the D signatures of the first two so as to generate the correct linkage by using ImportC to translate the inclusion of `Windows.h` into a .di file, and searching.
Mar 07
Carl Sturtivant <sturtivant gmail.com> On Thursday, 7 March 2024 at 18:14:32 UTC, Gregor Mückl wrote:1. Missing import libraries for Win32 API functions. Anything starting with `__imp_` is a symbol that should be provided by a DLL import library. MapViewOfFileNuma2 for example is provided by onecore.lib in the Windows SDK, according to Microsoft documentation.Onecore: not sure what I did to add that dependency. Two symbols there. Thanks for the __imp_ clue.2. C code referring to MSVC-specific compiler intrinsics. At least InterlockedExchangeAdd, InterlockedExchangeAdd64 and _stosb are such intrinsics. This is harder to resolve. There are two ways forward here: either implement a shim function that replicates the intrinsic's functionality if possible or add support for these intrinsics to DMD.Yes, not sure what the potential consequences are for my dirty replacement. Presumably DMD itself won't generate code using these, but ...
Mar 09
On Thursday, 7 March 2024 at 18:14:32 UTC, Gregor Mückl wrote:2. C code referring to MSVC-specific compiler intrinsics. At least InterlockedExchangeAdd, InterlockedExchangeAdd64 and _stosb are such intrinsics. This is harder to resolve. There are two ways forward here: either implement a shim function that replicates the intrinsic's functionality if possible or add support for these intrinsics to DMD.Thanks for the explanation, on the strength of which I found a way to deal with this correctly. I made `intrinsics1.c` that has `#include <intrin.h>` and contains an actual function for each missing intrinsic; e.g. for the missing __shiftright128 it has ```C unsigned __int64 D__shiftright128( unsigned __int64 LowPart, unsigned __int64 HighPart, unsigned char Shift ) { return __shiftright128(LowPart, HighPart, Shift); } ``` where I got the prototypes from this [list of intrinsics](https://learn.microsoft.com/en-us/cpp/intrinsics/alphabetical-listing-of-intrinsic-functions?view=msvc-170). Compiling this with MSVC `cl -c intrinsics1.c` produces a COFF object `intrinsics1.obj` containing an actual function to link to for each intrinsic. So this stage writes the code so we don't have to. As a matter of necessity, the names of the functions in `intrinsics1.c` representing the MSVC intrinsics are not the same as their actual names. By my convention above they are prefixed with "D". Now we could simply write an extern(C) D function in a module say `vcintrinsics.d`, that has exactly the intrinsic's name, and calls the "D" prefixed function linked from `intrinsics1.obj`, e.g. for the above example `vcintrinsics.d` could contain ```D extern(C): extern ulong D__shiftright128(ulong LowPart, ulong HighPart, ubyte Shift); ulong __shiftright128(ulong LowPart, ulong HighPart, ubyte Shift){ return D__shiftright128(LowPart, HighPart, Shift); } ``` As DMD doesn't know of these intrinsics, it won't complain about defining a function with exactly the same name as an intrinsic so as to implement that intrinsic as an actual function, solving the problem. `dmd -lib vcintrinsics.d intrinsics1.obj` then produces a library that resolves the linkage issue. An alternative not involving having two function calls to implement an intrinsic is to compile `intrinsics1.c` with MSVC into a DLL, and using a DEF file that renames the exports just like part of the solution [here](https://forum.dlang.org/post/ruupyklfkrvnjtkerhmx forum.dlang.org) make those functions available under their original names when linking to its import library. A lot of both of the above is boilerplate and can be automated. It would be even nicer if the MSVC tools could be persuaded to proceed in a similar way with DEF file renaming when building a static library, but I have not succeeded in making this happen. Anyone?
Mar 10
I found a way to make a solution for 64 bit Windows mechanically with many MSVC intrinsics, using only mingw64. Here's an [MSYS2](https://www.msys2.org/) bash script. ```bash gcc -E -P intrin.c -o vcintrinsics.c sed -i 's/extern __inline__ __attribute__((__always_inline__,__gnu_inline__))//g' vcintrinsics.c gcc -fPIC -shared -o vcintrinsics.dll vcintrinsics.c -Wl,--export-all-symbols -Wl,--output-def=vcintrinsics.def #lib -nologo -machine:x64 -def:vcintrinsics.def -out:vcintrinsics.lib dlltool -D vcintrinsics.dll -d vcintrinsics.def -l vcintrinsics.lib -m i386:x86-64 cp vcintrinsics.dll /c/D/dmd2/windows/bin64/ cp vcintrinsics.lib /c/D/dmd2/windows/lib64/ ``` The commented out line is using the MS librarian to do the same job as the mingw64 libtool. This script builds a dll containing 100+ intrinsics defined in mingw64 and makes an import library for DMD to use it from a developer command prompt. `intrin.c` contains only `#include <intrin.h>` and is preprocessed in the first line into `vcintrinsics.c` which if examined contains working definitions of many intrinsics but with `extern __inline__ __attribute__((__always_inline__,__gnu_inline__))` prefixing them, so they will not compile to actual library functions. The sed command strips those prefixes out, and the compilation makes a DLL containing the the intrinsics and a DEF file listing their names for linkage, ready for dlltool to assemble an import library suitable for DMD to link to so as to be able to dynamically link the DLL. And that's it; so far the result has just worked and I've had no linkage issues due to missing MSVC intrinsics since. I just put vcintrinsics.lib on the end of the dmd command line whenever I'm using real Windows headers in ImportC and the linker has always been satisfied so far. When I'm done, I can remove vcintrinsics from the dmd command line, and find out which intrinsics are actually needed for linking, and using the technique earlier in the thread above, satisfy those directly, so the result doesn't need vcintrinsics.dll. Hopefully DMD will soon know about these MSVC intrinsics but until then this is an effective way to more-or-less permanently work around the inevitable linkage problems.
Mar 20