digitalmars.D - Unilink - alternative linker for win32/64, DMD OMF extensions?
- Dmitry Olshansky <dmitry.olsh gmail.com> Jan 23 2011
- Andrej Mitrovic <andrej.mitrovich gmail.com> Jan 23 2011
- Dmitry Olshansky <dmitry.olsh gmail.com> Jan 23 2011
- Andrej Mitrovic <andrej.mitrovich gmail.com> Jan 23 2011
- Andrej Mitrovic <andrej.mitrovich gmail.com> Jan 23 2011
- Walter Bright <newshound2 digitalmars.com> Jan 23 2011
- Dmitry Olshansky <dmitry.olsh gmail.com> Jan 23 2011
- Walter Bright <newshound2 digitalmars.com> Jan 23 2011
- Trass3r <un known.com> Jan 26 2011
- Walter Bright <newshound2 digitalmars.com> Jan 26 2011
- Dmitry Olshansky <dmitry.olsh gmail.com> Jan 27 2011
- Trass3r <un known.com> Jan 28 2011
- Dmitry Olshansky <dmitry.olsh gmail.com> Jan 28 2011
- Trass3r <un known.com> Jan 27 2011
- Trass3r <un known.com> Jan 23 2011
- Trass3r <un known.com> Feb 06 2011
- Jacob Carlborg <doob me.com> Feb 07 2011
- Trass3r <un known.com> Feb 07 2011
- Andrej Mitrovic <andrej.mitrovich gmail.com> Feb 07 2011
- Dmitry Olshansky <dmitry.olsh gmail.com> Feb 11 2011
- Daniel Gibson <metalcaedes gmail.com> Feb 11 2011
- Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> Feb 11 2011
- Dmitry Olshansky <dmitry.olsh gmail.com> Feb 11 2011
- Trass3r <un known.com> Feb 12 2011
- Rainer Schuetze <r.sagitario gmx.de> Feb 12 2011
- David Nadlinger <see klickverbot.at> Feb 12 2011
- Dmitry Olshansky <dmitry.olsh gmail.com> Feb 12 2011
- Walter Bright <newshound2 digitalmars.com> Feb 12 2011
- "Akakima" <akakima33 gmail.com> Feb 12 2011
- Walter Bright <newshound2 digitalmars.com> Feb 12 2011
- bearophile <bearophileHUGS lycos.com> Feb 12 2011
- "Akakima" <akakima33 gmail.com> Feb 13 2011
- Walter Bright <newshound2 digitalmars.com> Feb 13 2011
- Lutger Blijdestijn <lutger.blijdestijn gmail.com> Feb 13 2011
- Walter Bright <newshound2 digitalmars.com> Feb 13 2011
- Jonathan M Davis <jmdavisProg gmx.com> Feb 12 2011
- %u <u pathlink.org> Feb 12 2011
- Andrej Mitrovic <andrej.mitrovich gmail.com> Feb 12 2011
- Andrej Mitrovic <andrej.mitrovich gmail.com> Feb 12 2011
- Andrej Mitrovic <andrej.mitrovich gmail.com> Feb 12 2011
Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit After hitting some troubles with optlink in the past (though my problems got solved others may be not) I was in constant search for alternatives. The good news: there *do* happen to be very versatile linker for windows able to produce 32/64bit PE, that supports OMF format and so on. Another cool thing that has is so called mix-mode linking i.e. linking objects produced by different compilers with different RTLs (tricky but might worth the effort). Link to ftp with docs and binary: ftp://ftp.styx.cabel.net/pub/UniLink/ According to author it does fully support DMC++ along with a huge pack of other C++ compilers. Now the bad news: it doesn't quite work with DMD for the moment. It's able to link test.d, but fails to link test2.d. (see attach) Stating that the resulting OMF is invalid. I asked the developer about it, his reply: /reply DM is in list of supported compilers (with DM C extensions). But D extension in sample is suspicious. There it attempt to define "select any" comdat that referenced to TLS-type segment. This is similar to common backend error. Another possibility is that it is an D "extension". But we don't possess documentation of D object format extensions and only can gess what it is. If the D becomes common used language we can "examine" its new features and take in into consideration. But... Maybe is's easier to contact with D developer and ask for new features in objects. /end reply QUESTION --->>> So here I am asking what kind of OMF extensions DMD uses, and what's difference between test.d and test2.d in terms of object file output? <<<--- Any hints on where to look for them are also welcome. If they are minor then we can convince the developer and have 64bit linker to experiment with on windows. NOTES I used dmd 2.051, though with 2.050 it's the same issue; command line: dmd -c<xxx>.d For those interested in experimenting with unilink themselves, use the following config (ulink.cfg) adjusting the path to dmd as needed: -zkernel32;advapi32;user32;wsock32;shell32;snn.lib -LC:\dmd2\windows\lib -Go Then linking is just: ulink <xxx>.obj -- Dmitry Olshansky
Jan 23 2011
Yeah I can reproduce the same thing as you have. I did manage to convert an OMF to COFF file format, and link it with GCC (MinGW). See my thread here: http://www.digitalmars.com/pnews/read.php?server=news.digitalmars.com&group=digitalmars.D&artnum=127079 I have yet to try it on more complex examples though. Give it a try and let me know if it works? :)
Jan 23 2011
On 23.01.2011 20:43, Andrej Mitrovic wrote:Yeah I can reproduce the same thing as you have. I did manage to convert an OMF to COFF file format, and link it with GCC (MinGW). See my thread here: http://www.digitalmars.com/pnews/read.php?server=news.digitalmars.com&group=digitalmars.D&artnum=127079 I have yet to try it on more complex examples though. Give it a try and let me know if it works? :)
Fatal: Entry point must be specified So, yes it strips something, yet it's no longer an invalid OMF ;) Anyhow after jumping though some hoops with objconv I got ld to linktime, but it gets a pack of unresolved externals from D's runtime/phobos and I don't think converting them all is wise idea. -- Dmitry Olshansky
Jan 23 2011
Perhaps this page would be helpful for the linker guys: http://www.digitalmars.com/d/2.0/abi.html
Jan 23 2011
Yeah, we're kind of stuck with Optlink for now.
Jan 23 2011
Dmitry Olshansky wrote:If the D becomes common used language we can "examine" its new features and take in into consideration. But... Maybe is's easier to contact with D developer and ask for new features in objects.
If the author wants to email me with any specific questions, I'd be happy to help him.
Jan 23 2011
On 23.01.2011 23:55, Walter Bright wrote:Dmitry Olshansky wrote:If the D becomes common used language we can "examine" its new features and take in into consideration. But... Maybe is's easier to contact with D developer and ask for new features in objects.
If the author wants to email me with any specific questions, I'd be happy to help him.
himself unless someone proves it's usefulness. It's more like we need a little help not the other way around. Well, anyway I can mail him and hope that he will do it just out of curiosity, what's yours relevant email then? May I just as well tell him that you are interested in it or anything? -- Dmitry Olshansky
Jan 23 2011
Dmitry Olshansky wrote:On 23.01.2011 23:55, Walter Bright wrote:Dmitry Olshansky wrote:If the D becomes common used language we can "examine" its new features and take in into consideration. But... Maybe is's easier to contact with D developer and ask for new features in objects.
If the author wants to email me with any specific questions, I'd be happy to help him.
himself unless someone proves it's usefulness. It's more like we need a little help not the other way around. Well, anyway I can mail him and hope that he will do it just out of curiosity, what's yours relevant email then? May I just as well tell him that you are interested in it or anything?
First off, there are no D extensions to the OMF. There was an extension added around 1993 or so to support long identifier names. There were also a couple extensions to the codeview debug format, which are documented in the ABI. My offer is if there is something specific about the OMF files generated that I can explain to him, I'd be happy to do so. My mail is walter followed by digitalmars.com.
Jan 23 2011
Walter Bright Wrote:Dmitry Olshansky wrote:Well, anyway I can mail him and hope that he will do it just out of curiosity, what's yours relevant email then? May I just as well tell him that you are interested in it or anything?
My offer is if there is something specific about the OMF files generated that I can explain to him, I'd be happy to do so. My mail is walter followed by digitalmars.com.
Any news here?
Jan 26 2011
Trass3r wrote:Walter Bright Wrote:Dmitry Olshansky wrote:Well, anyway I can mail him and hope that he will do it just out of curiosity, what's yours relevant email then? May I just as well tell him that you are interested in it or anything?
can explain to him, I'd be happy to do so. My mail is walter followed by digitalmars.com.
Any news here?
No, I have not heard anything.
Jan 26 2011
On 26.01.2011 21:52, Walter Bright wrote:Trass3r wrote:Walter Bright Wrote:Dmitry Olshansky wrote:Well, anyway I can mail him and hope that he will do it just out of curiosity, what's yours relevant email then? May I just as well tell him that you are interested in it or anything?
generated that I can explain to him, I'd be happy to do so. My mail is walter followed by digitalmars.com.
Any news here?
No, I have not heard anything.
I've got their detailed report on the sample object file, and relayed it to Walter. Something related to D's stance on TLS it seems. With me being middle man all this can take some extra time, but I don't mind. The end result may still well worth it. -- Dmitry Olshansky
Jan 27 2011
I wonder why this tool isn't promoted in any way, no website etc. I also wonder if he was willing to make it open-source. Then we could help support D and if it works someday, we can even include D symbol demangling :)
Jan 28 2011
On 28.01.2011 13:21, Trass3r wrote:I wonder why this tool isn't promoted in any way, no website etc.
I got to this ftp only because I was 100% sure that tool existed, as a friend of mine suggested to use it instead of Borland's crappy linker few years ago.I also wonder if he was willing to make it open-source. Then we could help support D and if it works someday, we can even include D symbol demangling :)
-- Dmitry Olshansky
Jan 28 2011
I've got their detailed report on the sample object file, and relayed it to Walter. Something related to D's stance on TLS it seems. With me being middle man all this can take some extra time, but I don't mind. The end result may still well worth it.
Hope so too.
Jan 27 2011
Hopefully we can get the author to support dmd. I can't wait years until optlink might support x64.
Jan 23 2011
Just another thought: dmd uses ld on linux, couldn't it use MinGW's ld on Windows?
Feb 06 2011
On 2011-02-06 23:26, Trass3r wrote:Just another thought: dmd uses ld on linux, couldn't it use MinGW's ld on Windows?
MinGW's ld doesn't use the same object format as DMD/DMC -- /Jacob Carlborg
Feb 07 2011
Just another thought: dmd uses ld on linux, couldn't it use MinGW's ld on Windows?
MinGW's ld doesn't use the same object format as DMD/DMC
I know, of course this would include discarding OMF.
Feb 07 2011
I found this gem in digitalmars' bookshelf: http://www.digitalmars.com/bibliography.html#fileformats
Feb 07 2011
Ok, bumping this up with the latest news from UniLink developers: <quote> Ok, we release it's as D extension in next release. Best regards, UniLink </quote> That's just plain awesome ;) -- Dmitry Olshansky
Feb 11 2011
Am 11.02.2011 19:56, schrieb Dmitry Olshansky:Ok, bumping this up with the latest news from UniLink developers: <quote> Ok, we release it's as D extension in next release. Best regards, UniLink </quote> That's just plain awesome ;)
Great :)
Feb 11 2011
On 2/11/11 12:56 PM, Dmitry Olshansky wrote:Ok, bumping this up with the latest news from UniLink developers: <quote> Ok, we release it's as D extension in next release. Best regards, UniLink </quote> That's just plain awesome ;)
Is UniLink working with 64 bits? Andrei
Feb 11 2011
On 12.02.2011 3:13, Andrei Alexandrescu wrote:On 2/11/11 12:56 PM, Dmitry Olshansky wrote:Ok, bumping this up with the latest news from UniLink developers: <quote> Ok, we release it's as D extension in next release. Best regards, UniLink </quote> That's just plain awesome ;)
Is UniLink working with 64 bits? Andrei
Yes, that's actually one of it's benefits, as I told somewhere along the thread. To be 101% sure I just linked a simple 'hello x64' program. -- Dmitry Olshansky
Feb 11 2011
Dmitry Olshansky wrote:Ok, bumping this up with the latest news from UniLink developers: <quote> Ok, we release it's as D extension in next release. Best regards, UniLink </quote> That's just plain awesome ;)
yeah, it would be great to be able to link against anything else in the windows world. I just tried unilink on a small program compiled with dmd-2.051 and it worked alright. Surprisingly, the executable is less than half the size of the optlink generated file. Comparing the map files, it does not seem to reference less symbols (the corrupted map files written by optlink do not allow perfect comparison), so maybe there is a rather large alignment used by optlink? Unfortunately, unilink does not support the debug information emitted by dmd: UniLink v1.07 [beta] (build 1.19) Warning: CV(1)-style debug-info not supported Warning: Linked modules don't contain source-level debug information BTW: The authors seem to plan to make unilink a commercial product. Here's a quote from the help file: "If current version of UniLink is beta (all other versions are commercial and can't be used w/o license), keep in mind, any beta will "yaff" after ~1.5 month of expired."
Feb 12 2011
On 2/12/11 3:30 PM, Andrej Mitrovic wrote:What linker does GDC use on Windows? And has anyone had any success with using it on Windows?
IIRC, it emits COFF and uses the ld from MinGW to link. David
Feb 12 2011
On 12.02.2011 16:32, Rainer Schuetze wrote:Dmitry Olshansky wrote:Ok, bumping this up with the latest news from UniLink developers: <quote> Ok, we release it's as D extension in next release. Best regards, UniLink </quote> That's just plain awesome ;)
yeah, it would be great to be able to link against anything else in the windows world. I just tried unilink on a small program compiled with dmd-2.051 and it worked alright. Surprisingly, the executable is less than half the size of the optlink generated file. Comparing the map files, it does not seem to reference less symbols (the corrupted map files written by optlink do not allow perfect comparison), so maybe there is a rather large alignment used by optlink?
Same things here, as to the executable size it might be some smart technology they plan to sell ;)Unfortunately, unilink does not support the debug information emitted by dmd: UniLink v1.07 [beta] (build 1.19) Warning: CV(1)-style debug-info not supported Warning: Linked modules don't contain source-level debug information
Yeah, that spoils the party...BTW: The authors seem to plan to make unilink a commercial product. Here's a quote from the help file: "If current version of UniLink is beta (all other versions are commercial and can't be used w/o license), keep in mind, any beta will "yaff" after ~1.5 month of expired."
I've seen that "yaff" :) Merely a message in console that states that your beta linker is out of date, functionality not crippled. More interestingly I haven't seen any commercial versions yet.. -- Dmitry Olshansky
Feb 12 2011
Andrej Mitrovic wrote:Great, from one closed-source linker to another.
Making optlink open source won't make any difference. Few are skilled at asm anymore, and likely none of them would want to work on optlink for free.
Feb 12 2011
Making optlink open source won't make any difference. Few are skilled at asm anymore, and likely none of them would want to work on optlink for free.
That's true. But the real problem is not optlink. Optlink is a very good linker. The problem is OMF. 11 years ago OMF was a good choice. But not anymore. I know you are a competent (probably very competent) compiler writer. You modified D on linux, so it produce ELF. How much time would that take to modify DMD so it produce COFF ? Given all the bad publicity OMF gives to D, it should be viewed as a good choice. There are many (not much), but there are open source linkers. Of course ld is not as fast as optlink, but it's good. And there is a faster version made by the project Ultimate++ IDE. Going to COFF would have a lot of advantages for everybody and for D. Do you agree ?
Feb 12 2011
Akakima wrote:Making optlink open source won't make any difference. Few are skilled at asm anymore, and likely none of them would want to work on optlink for free.
That's true. But the real problem is not optlink. Optlink is a very good linker. The problem is OMF. 11 years ago OMF was a good choice. But not anymore. I know you are a competent (probably very competent) compiler writer. You modified D on linux, so it produce ELF. How much time would that take to modify DMD so it produce COFF ? Given all the bad publicity OMF gives to D, it should be viewed as a good choice. There are many (not much), but there are open source linkers. Of course ld is not as fast as optlink, but it's good. And there is a faster version made by the project Ultimate++ IDE. Going to COFF would have a lot of advantages for everybody and for D. Do you agree ?
Changing the object module format is not sufficient. The symbolic debug info would have to be changed (and Microsoft's is undocumented) and then there's the dependency on Microsoft's C runtime library if linking with VC generated object files.
Feb 12 2011
Walter:Changing the object module format is not sufficient. The symbolic debug info would have to be changed (and Microsoft's is undocumented) and then there's the dependency on Microsoft's C runtime library if linking with VC generated object files.
What about linking with MinGW (GCC 4.5+) object files? Bye, bearophile
Feb 12 2011
Changing the object module format is not sufficient. The symbolic debug info would have to be changed (and Microsoft's is undocumented) and then there's the dependency on Microsoft's C runtime library if linking with VC generated object files.
I found some doc there: http://pierrelib.pagesperso-orange.fr/exec_formats/index.html Microsoft Symbol and Type Information By TIS / Microsoft. Entry added 12/28/2004. Keywords: ms, symbol, type, info File: MS_Symbol_Type_v1.0.pdf « This document describes Microsoft Symbol and Type Information, a debugging information format fromMicrosoft Corporation for the 32-bit Windows environment. » There is also some doc on the MSDN CD that comes with Visual C++ 6.0.
Feb 13 2011
Akakima wrote:Changing the object module format is not sufficient. The symbolic debug info would have to be changed (and Microsoft's is undocumented) and then there's the dependency on Microsoft's C runtime library if linking with VC generated object files.
I found some doc there: http://pierrelib.pagesperso-orange.fr/exec_formats/index.html Microsoft Symbol and Type Information By TIS / Microsoft. Entry added 12/28/2004. Keywords: ms, symbol, type, info File: MS_Symbol_Type_v1.0.pdf « This document describes Microsoft Symbol and Type Information, a debugging information format fromMicrosoft Corporation for the 32-bit Windows environment. »
That document describes the Codeview symbol debug format, which Microsoft abandoned 15 years ago in favor of a proprietary format. Dmd generates that older format :-)
Feb 13 2011
Walter Bright wrote:Akakima wrote:Changing the object module format is not sufficient. The symbolic debug info would have to be changed (and Microsoft's is undocumented) and then there's the dependency on Microsoft's C runtime library if linking with VC generated object files.
I found some doc there: http://pierrelib.pagesperso-orange.fr/exec_formats/index.html Microsoft Symbol and Type Information By TIS / Microsoft. Entry added 12/28/2004. Keywords: ms, symbol, type, info File: MS_Symbol_Type_v1.0.pdf « This document describes Microsoft Symbol and Type Information, a debugging information format fromMicrosoft Corporation for the 32-bit Windows environment. »
That document describes the Codeview symbol debug format, which Microsoft abandoned 15 years ago in favor of a proprietary format. Dmd generates that older format :-)
Are you going to do Elf? With optlink in D? (Does that even make sense?)
Feb 13 2011
Lutger Blijdestijn wrote:Walter Bright wrote:Akakima wrote:Changing the object module format is not sufficient. The symbolic debug info would have to be changed (and Microsoft's is undocumented) and then there's the dependency on Microsoft's C runtime library if linking with VC generated object files.
http://pierrelib.pagesperso-orange.fr/exec_formats/index.html Microsoft Symbol and Type Information By TIS / Microsoft. Entry added 12/28/2004. Keywords: ms, symbol, type, info File: MS_Symbol_Type_v1.0.pdf « This document describes Microsoft Symbol and Type Information, a debugging information format fromMicrosoft Corporation for the 32-bit Windows environment. »
That document describes the Codeview symbol debug format, which Microsoft abandoned 15 years ago in favor of a proprietary format. Dmd generates that older format :-)
Are you going to do Elf? With optlink in D? (Does that even make sense?)
Elf is not a symbolic debug format. It is possible to use Codeview with Elf. In any case, such decisions will have to revolve around the availability of things like debuggers.
Feb 13 2011
On Saturday 12 February 2011 17:09:36 Walter Bright wrote:Akakima wrote:Making optlink open source won't make any difference. Few are skilled at asm anymore, and likely none of them would want to work on optlink for free.
That's true. But the real problem is not optlink. Optlink is a very good linker. The problem is OMF. 11 years ago OMF was a good choice. But not anymore. I know you are a competent (probably very competent) compiler writer. You modified D on linux, so it produce ELF. How much time would that take to modify DMD so it produce COFF ? Given all the bad publicity OMF gives to D, it should be viewed as a good choice. There are many (not much), but there are open source linkers. Of course ld is not as fast as optlink, but it's good. And there is a faster version made by the project Ultimate++ IDE. Going to COFF would have a lot of advantages for everybody and for D. Do you agree ?
Changing the object module format is not sufficient. The symbolic debug info would have to be changed (and Microsoft's is undocumented) and then there's the dependency on Microsoft's C runtime library if linking with VC generated object files.
That's a problem for some code and pretty much an absolute necessity for other code. Any project that I'd do at work would _have_ to use Microsoft's runtime library. In the long run, we really do need a solution which is compatible with Microsoft's compiler. Without that, there are a lot of programmers who just won't be able to use D in any project which would be using C or C++ code, and not being able to use any C or C++ code in D can be a major problem - especially if you're trying to use D with an existing code base. So, I really don't know what the best way to handle all of this is, and I really don't know all that much about linkers, but the fact that you can't link D code with C code generated by Microsoft's compiler is crippling for Windows development. - Jonathan M Davis
Feb 12 2011
Andrej Mitrovic Wrote:On 2/12/11, Walter Bright <newshound2 digitalmars.com> wrote:Andrej Mitrovic wrote:Great, from one closed-source linker to another.
Making optlink open source won't make any difference. Few are skilled at asm anymore, and likely none of them would want to work on optlink for free.
Ok, but what about your upcoming C version? Don't get me wrong, I appreciate that we even have a linker! But it's kind of sad when one of the most active D developers quits because of linker issues.
We should rename Optlink to Vuvulink, it's the vuvuzela of D's world. We can't seriously use D with any commercial technologies since the code isn't binary compatible. This is getting ridiculous. 12 years of fail. We could even pay to get a proper linker. But how much would it cost? Around $200000 - $500000. That's too much. We could outsource our whole software development at that price and focus on profits.
Feb 12 2011
On 2/12/11, Rainer Schuetze <r.sagitario gmx.de> wrote:BTW: The authors seem to plan to make unilink a commercial product. Here's a quote from the help file: "If current version of UniLink is beta (all other versions are commercial and can't be used w/o license), keep in mind, any beta will "yaff" after ~1.5 month of expired."
Great, from one closed-source linker to another. What linker does GDC use on Windows? And has anyone had any success with using it on Windows?
Feb 12 2011
On 2/12/11, David Nadlinger <see klickverbot.at> wrote:On 2/12/11 3:30 PM, Andrej Mitrovic wrote:What linker does GDC use on Windows? And has anyone had any success with using it on Windows?
IIRC, it emits COFF and uses the ld from MinGW to link. David
That's nice to know. I'll give GDC a spin and see how it's working out.
Feb 12 2011
On 2/12/11, Walter Bright <newshound2 digitalmars.com> wrote:Andrej Mitrovic wrote:Great, from one closed-source linker to another.
Making optlink open source won't make any difference. Few are skilled at asm anymore, and likely none of them would want to work on optlink for free.
Ok, but what about your upcoming C version? Don't get me wrong, I appreciate that we even have a linker! But it's kind of sad when one of the most active D developers quits because of linker issues.
Feb 12 2011









Dmitry Olshansky <dmitry.olsh gmail.com> 