www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Anticipate the usage of Visual Studio 2017 Developer Command Prompt?

reply Andre Pany <andre s-e-a-p.de> writes:
Hi,

Visual Studio has a batch file which sets all needed environment 
variables for the Microsoft linker (LIB environment variables 
points to all necessary folders). Even the folder containing the 
link.exe for the specified architecture (x86, amd64) is added to 
the path variable as first path.

 vsdevcmd -arch=amd64
As the sc.ini targets the windows development (visuals studio) after calling the developer command prompt, dmd should be able to compile -m32mscoff, -m64 out of the box without the need to change anything within sc.ini. There is even a comment in sc.ini which points in this direction: ; Add the library subdirectories of all VC and Windows SDK versions so things ; just work for users using dmd from the VS Command Prompt Unfortunately it does not work for various reasons: - optlink has the filename link.exe which conflicts with the filename of the microsoft linker - from my investigation I have the assumption that even if an environment variable LIB exists, the value is discarded by the sc.ini logic. I tried hard, but it seems existing environment variable LIB cannot be used in sc.ini If you can use the windows installer, it will set the right values in sc.ini. But there are scenarios in which you cannot use the windows installer but have to use the dmd zip archive (build infrastructure scripts). In my scenario it seems the only possibility is to adapt my build infrastructure scripts to: - extract dmd archive - delete link.exe (optlink) - modify the sc.ini That is a solution for my case, but my gut feeling is, it should be solved in general to ease the usage of the microsoft linker. I thought it was planned to rename the optlink link.exe to optlink.exe? Kind regards André
Nov 03
parent reply Andre Pany <andre s-e-a-p.de> writes:
On Friday, 3 November 2017 at 11:52:10 UTC, Andre Pany wrote:
 Hi,

 Visual Studio has a batch file which sets all needed 
 environment variables for the Microsoft linker (LIB environment 
 variables points to all necessary folders). Even the folder 
 containing the link.exe for the specified architecture (x86, 
 amd64) is added to the path variable as first path.

 vsdevcmd -arch=amd64
As the sc.ini targets the windows development (visuals studio) after calling the developer command prompt, dmd should be able to compile -m32mscoff, -m64 out of the box without the need to change anything within sc.ini. There is even a comment in sc.ini which points in this direction: ; Add the library subdirectories of all VC and Windows SDK versions so things ; just work for users using dmd from the VS Command Prompt Unfortunately it does not work for various reasons: - optlink has the filename link.exe which conflicts with the filename of the microsoft linker - from my investigation I have the assumption that even if an environment variable LIB exists, the value is discarded by the sc.ini logic. I tried hard, but it seems existing environment variable LIB cannot be used in sc.ini If you can use the windows installer, it will set the right values in sc.ini. But there are scenarios in which you cannot use the windows installer but have to use the dmd zip archive (build infrastructure scripts). In my scenario it seems the only possibility is to adapt my build infrastructure scripts to: - extract dmd archive - delete link.exe (optlink) - modify the sc.ini That is a solution for my case, but my gut feeling is, it should be solved in general to ease the usage of the microsoft linker. I thought it was planned to rename the optlink link.exe to optlink.exe? Kind regards André
I have an idea which solves several problems. Current state: The dmd windows archive has a folder "bin" with a 32 bit dmd compiler which is able to create x86 and x86_64 applications. By default the dmd compiler creates 32 bit applications using the optlink link.exe contained also in this folder. Idea: Within the folder "bin64" the 64 bit dmd compiler could be shipped within the dmd windows archive. I assume the 64 bit dmd compiler is able to create x86 and x86_64 applications. In folder "bin64" no optlink link.exe should be available. The usage of either the microsoft link.exe or another linker (maybe the one from LDC) is anticipated. The 64 bit dmd compiler could have the option -m64 set as default. The user can decide whether he wants to use the "stable" dmd compiler with usage of optlink or he want to use the 64 bit dmd compiler which is open for other linkers than optlink by simply setting his path variable to bin or to bin64. Using this approach: - DMD compiler is working out of the box for the VS Developer Command Prompt - no need to rename the optlink link.exe - A 64 bit dmd compiler is shipped which solves issues on large D projects What do you think? Kind regards André
Nov 03
parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 03.11.2017 21:51, Andre Pany wrote:
 On Friday, 3 November 2017 at 11:52:10 UTC, Andre Pany wrote:
 Hi,

 Visual Studio has a batch file which sets all needed environment 
 variables for the Microsoft linker (LIB environment variables points 
 to all necessary folders). Even the folder containing the link.exe for 
 the specified architecture (x86, amd64) is added to the path variable 
 as first path.

 vsdevcmd -arch=amd64
As the sc.ini targets the windows development (visuals studio) after calling the developer command prompt, dmd should be able to compile -m32mscoff, -m64 out of the box without the need to change anything within sc.ini. There is even a comment in sc.ini which points in this direction: ; Add the library subdirectories of all VC and Windows SDK versions so things ; just work for users using dmd from the VS Command Prompt Unfortunately it does not work for various reasons: - optlink has the filename link.exe which conflicts with the filename of the microsoft linker - from my investigation I have the assumption that even if an environment variable LIB exists, the value is discarded by the sc.ini logic. I tried hard, but it seems existing environment variable LIB cannot be used in sc.ini If you can use the windows installer, it will set the right values in sc.ini. But there are scenarios in which you cannot use the windows installer but have to use the dmd zip archive (build infrastructure scripts). In my scenario it seems the only possibility is to adapt my build infrastructure scripts to: - extract dmd archive - delete link.exe (optlink) - modify the sc.ini That is a solution for my case, but my gut feeling is, it should be solved in general to ease the usage of the microsoft linker. I thought it was planned to rename the optlink link.exe to optlink.exe? Kind regards André
I have an idea which solves several problems. Current state: The dmd windows archive has a folder "bin" with a 32 bit dmd compiler which is able to create x86 and x86_64 applications. By default the dmd compiler creates 32 bit applications using the optlink link.exe contained also in this folder. Idea: Within the folder "bin64" the 64 bit dmd compiler could be shipped within the dmd windows archive. I assume the 64 bit dmd compiler is able to create x86 and x86_64 applications. In folder "bin64" no optlink link.exe should be available. The usage of either the microsoft link.exe or another linker (maybe the one from LDC) is anticipated. The 64 bit dmd compiler could have the option -m64 set as default. The user can decide whether he wants to use the "stable" dmd compiler with usage of optlink or he want to use the 64 bit dmd compiler which is open for other linkers than optlink by simply setting his path variable to bin or to bin64. Using this approach: - DMD compiler is working out of the box for the VS Developer Command Prompt - no need to rename the optlink link.exe - A 64 bit dmd compiler is shipped which solves issues on large D projects What do you think? Kind regards André
Finding the wrong linker seems to happen quite often, but I wonder why: sc.ini usually has the absolute path to the linker set via the LINKCMD variable, so finding the correct executable should not be a problem if the installer has set things up correctly. If extracted from the zip there are also defaults that might not work but should not find the wrong linker. Is it the result of trying to fix the sc.ini manually? The problem with overriding the LIB environment variable is that optlink and the MS linker both use it, but stumble over the libraries for the other linker due to different file formats. I think the best option forward is to not rely on the installer to find the correct setup at installation time, but detect the available linker and its environment before invoking it: - if the environment is already set up via vcvarsall, i.e. VCINSTALLDIR set, use that to call the approriate linker - if VCINSTALLDIR is not set, use the registry to find the best fit, i.e. highest available VS version - some way to override the automatism should be available, too. We might also be able to provide some fallback if no VS installation is found, based on the LLVM linker, but with limited C runtime/platform library support.
Nov 04
parent reply Andre Pany <andre s-e-a-p.de> writes:
On Saturday, 4 November 2017 at 09:39:02 UTC, Rainer Schuetze 
wrote:
 On 03.11.2017 21:51, Andre Pany wrote:
 On Friday, 3 November 2017 at 11:52:10 UTC, Andre Pany wrote:
 [...]
I have an idea which solves several problems. Current state: The dmd windows archive has a folder "bin" with a 32 bit dmd compiler which is able to create x86 and x86_64 applications. By default the dmd compiler creates 32 bit applications using the optlink link.exe contained also in this folder. Idea: Within the folder "bin64" the 64 bit dmd compiler could be shipped within the dmd windows archive. I assume the 64 bit dmd compiler is able to create x86 and x86_64 applications. In folder "bin64" no optlink link.exe should be available. The usage of either the microsoft link.exe or another linker (maybe the one from LDC) is anticipated. The 64 bit dmd compiler could have the option -m64 set as default. The user can decide whether he wants to use the "stable" dmd compiler with usage of optlink or he want to use the 64 bit dmd compiler which is open for other linkers than optlink by simply setting his path variable to bin or to bin64. Using this approach: - DMD compiler is working out of the box for the VS Developer Command Prompt - no need to rename the optlink link.exe - A 64 bit dmd compiler is shipped which solves issues on large D projects What do you think? Kind regards André
Finding the wrong linker seems to happen quite often, but I wonder why: sc.ini usually has the absolute path to the linker set via the LINKCMD variable, so finding the correct executable should not be a problem if the installer has set things up correctly. If extracted from the zip there are also defaults that might not work but should not find the wrong linker. Is it the result of trying to fix the sc.ini manually? The problem with overriding the LIB environment variable is that optlink and the MS linker both use it, but stumble over the libraries for the other linker due to different file formats. I think the best option forward is to not rely on the installer to find the correct setup at installation time, but detect the available linker and its environment before invoking it: - if the environment is already set up via vcvarsall, i.e. VCINSTALLDIR set, use that to call the approriate linker - if VCINSTALLDIR is not set, use the registry to find the best fit, i.e. highest available VS version - some way to override the automatism should be available, too. We might also be able to provide some fallback if no VS installation is found, based on the LLVM linker, but with limited C runtime/platform library support.
I would say, by executing vcvarsall the user showed his intention to use the coff libraries and the Microsoft linker for the actual console session instead of the omf libraries. I see a disadvantage by detecting the vc installation directory / lib directories. All paths are specific for the vs version. If there is x new vs version we are always behind. But Microsoft has a solution for this problem. Vcvarsall sets the linker as first element of the path variable and the lib env variable points to the right folder. This solution will work for old vs version and any upcoming version. I set linkcmd to "link.exe". Unfortunately the link.exe in the dmd bin folder has precedence over the path env variable. Kind regards Andre
Nov 04
parent Rainer Schuetze <r.sagitario gmx.de> writes:
On 04.11.2017 12:16, Andre Pany wrote:
 On Saturday, 4 November 2017 at 09:39:02 UTC, Rainer Schuetze wrote:
 On 03.11.2017 21:51, Andre Pany wrote:
 On Friday, 3 November 2017 at 11:52:10 UTC, Andre Pany wrote:
 [...]
I have an idea which solves several problems. Current state: The dmd windows archive has a folder "bin" with a 32 bit dmd compiler which is able to create x86 and x86_64 applications. By default the dmd compiler creates 32 bit applications using the optlink link.exe contained also in this folder. Idea: Within the folder "bin64" the 64 bit dmd compiler could be shipped within the dmd windows archive. I assume the 64 bit dmd compiler is able to create x86 and x86_64 applications. In folder "bin64" no optlink link.exe should be available. The usage of either the microsoft link.exe or another linker (maybe the one from LDC) is anticipated. The 64 bit dmd compiler could have the option -m64 set as default. The user can decide whether he wants to use the "stable" dmd compiler with usage of optlink or he want to use the 64 bit dmd compiler which is open for other linkers than optlink by simply setting his path variable to bin or to bin64. Using this approach: - DMD compiler is working out of the box for the VS Developer Command Prompt - no need to rename the optlink link.exe - A 64 bit dmd compiler is shipped which solves issues on large D projects What do you think? Kind regards André
Finding the wrong linker seems to happen quite often, but I wonder why: sc.ini usually has the absolute path to the linker set via the LINKCMD variable, so finding the correct executable should not be a problem if the installer has set things up correctly. If extracted from the zip there are also defaults that might not work but should not find the wrong linker. Is it the result of trying to fix the sc.ini manually? The problem with overriding the LIB environment variable is that optlink and the MS linker both use it, but stumble over the libraries for the other linker due to different file formats. I think the best option forward is to not rely on the installer to find the correct setup at installation time, but detect the available linker and its environment before invoking it: - if the environment is already set up via vcvarsall, i.e. VCINSTALLDIR set, use that to call the approriate linker - if VCINSTALLDIR is not set, use the registry to find the best fit, i.e. highest available VS version - some way to override the automatism should be available, too. We might also be able to provide some fallback if no VS installation is found, based on the LLVM linker, but with limited C runtime/platform library support.
I would say, by executing vcvarsall the user showed his intention to use the coff libraries and the Microsoft linker for the actual console session instead of the omf libraries. I see a disadvantage by detecting the vc installation directory / lib directories. All paths are specific for the vs version. If there is x new vs version we are always behind. But Microsoft has a solution for this problem. Vcvarsall sets the linker as first element of the path variable and the lib env variable points to the right folder. This solution will work for old vs version and any upcoming version.
That's what I wrote above, too: the settings from vcvarsall should take precedence. The library path for the phobos library must be added, though.
 
 I set linkcmd to "link.exe". Unfortunately the link.exe in the dmd bin 
 folder has precedence over the path env variable.
This is caused by CreateProcess looking in the directory of the calling process first. This could be avoided in dmd by searching the PATH folders explicitly before invoking the linker.
Nov 04