www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - Required to link windows header modules (?)

reply Sean Kelly <sean f4.ca> writes:
The standard C headers I've defined don't need to be linked against my 
app in any circumstances I've encountered so far--they serve as pure 
import modules.  However, I've recently discovered that this does not 
hold true for the Windows headers I use, which seem to require me to 
link against them if I reference any struct they declare.  Is it 
possible that extern (C) declarations tell the compiler that the struct 
will be available elsewhere and extern (Windows) doesn't?  It's worth 
noting that the link errors I get display a D-style mangled name for the 
struct definitions rather than a C-style name as I'd expect.  Should 
these structs all be declared as extern (C)?


Sean
Jun 08 2006
next sibling parent reply Walter Bright <newshound digitalmars.com> writes:
Sean Kelly wrote:
 The standard C headers I've defined don't need to be linked against my 
 app in any circumstances I've encountered so far--they serve as pure 
 import modules.  However, I've recently discovered that this does not 
 hold true for the Windows headers I use, which seem to require me to 
 link against them if I reference any struct they declare.  Is it 
 possible that extern (C) declarations tell the compiler that the struct 
 will be available elsewhere and extern (Windows) doesn't?  It's worth 
 noting that the link errors I get display a D-style mangled name for the 
 struct definitions rather than a C-style name as I'd expect.  Should 
 these structs all be declared as extern (C)?

What link errors are you getting?
Jun 08 2006
parent reply Sean Kelly <sean f4.ca> writes:
Walter Bright wrote:
 Sean Kelly wrote:
 The standard C headers I've defined don't need to be linked against my 
 app in any circumstances I've encountered so far--they serve as pure 
 import modules.  However, I've recently discovered that this does not 
 hold true for the Windows headers I use, which seem to require me to 
 link against them if I reference any struct they declare.  Is it 
 possible that extern (C) declarations tell the compiler that the 
 struct will be available elsewhere and extern (Windows) doesn't?  It's 
 worth noting that the link errors I get display a D-style mangled name 
 for the struct definitions rather than a C-style name as I'd expect.  
 Should these structs all be declared as extern (C)?

What link errors are you getting?

For example, if I modify the Phobos makefile by removing any reference to std.c.windows.windows, then rebuilding Phobos gives me this at the end. It's essentially the same as the issues I've been having: Digital Mars Librarian Version 8.00n Copyright (C) Digital Mars 2000-2002 All Rights Reserved www.digitalmars.com Digital Mars Librarian complete. \bin\dmd\bin\dmd unittest -g C:\bin\dmd\bin\..\..\dm\bin\link.exe unittest,,,user32+kernel32/co/noi; OPTLINK (R) for Win32 Release 7.50B1 Copyright (C) Digital Mars 1989 - 2001 All Rights Reserved phobos.lib(date) Error 42: Symbol Undefined __init_3std1c7windows7windows21TIME_ZONE_INFORMATION phobos.lib(file) Error 42: Symbol Undefined __init_3std1c7windows7windows16WIN32_FIND_DATAW phobos.lib(file) Error 42: Symbol Undefined __init_3std1c7windows7windows15WIN32_FIND_DATA phobos.lib(gcx) Error 42: Symbol Undefined __init_3std1c7windows7windows7CONTEXT phobos.lib(syserror) Error 42: Symbol Undefined _MAKELANGID 8 --- errorlevel 5 --- errorlevel 5 C:\bin\dmd\src\phobos>
Jun 08 2006
parent reply Walter Bright <newshound digitalmars.com> writes:
Sean Kelly wrote:
 phobos.lib(date)
  Error 42: Symbol Undefined 
 __init_3std1c7windows7windows21TIME_ZONE_INFORMATION
 
 phobos.lib(file)
  Error 42: Symbol Undefined __init_3std1c7windows7windows16WIN32_FIND_DATAW
 phobos.lib(file)
  Error 42: Symbol Undefined __init_3std1c7windows7windows15WIN32_FIND_DATA
 phobos.lib(gcx)
  Error 42: Symbol Undefined __init_3std1c7windows7windows7CONTEXT
 phobos.lib(syserror)
  Error 42: Symbol Undefined _MAKELANGID 8

Ok, these are the static initializers for the struct, and a function for MAKELANGID. My next question is what's the issue with linking with windows.obj?
Jun 08 2006
parent reply Sean Kelly <sean f4.ca> writes:
Walter Bright wrote:
 Sean Kelly wrote:
 phobos.lib(date)
  Error 42: Symbol Undefined 
 __init_3std1c7windows7windows21TIME_ZONE_INFORMATION

 phobos.lib(file)
  Error 42: Symbol Undefined 
 __init_3std1c7windows7windows16WIN32_FIND_DATAW
 phobos.lib(file)
  Error 42: Symbol Undefined 
 __init_3std1c7windows7windows15WIN32_FIND_DATA
 phobos.lib(gcx)
  Error 42: Symbol Undefined __init_3std1c7windows7windows7CONTEXT
 phobos.lib(syserror)
  Error 42: Symbol Undefined _MAKELANGID 8

Ok, these are the static initializers for the struct, and a function for MAKELANGID. My next question is what's the issue with linking with windows.obj?

Linking with the windows module present in Phobos isn't a big deal. But the Windows header project Don has been working on is quite expansive, and simply importing windows.d ends up pulling in a substantial chunk of the windows modules. This slows compilation noticeably and increases binary size significantly--simply importing Don's windows.d in my test app made the executable size increase from ~90K to over 150K. This appears to be related at least in part to what appears to be a fixed minimum for importing a module using DMD of approximately 500 bytes, which I assume is the ModuleInfo data and similar things. What I find confusing, however, is that I don't appear to need to link in the standard C modules I've defined in order to use the structs defined there. I performed a quick test using struct tm and no dependency problems came up simply by compiling the test app itself and linking against a library without std.c.time in it. Assuming I didn't mess up the test somehow, can you explain the difference between this situation and the Windows one? If it's simply a matter of having the static initializers available, I'm fine with building everything, but the inconsistency doesn't seem to make sense. Sean
Jun 08 2006
parent reply Walter Bright <newshound digitalmars.com> writes:
Sean Kelly wrote:
 Linking with the windows module present in Phobos isn't a big deal.  But 
 the Windows header project Don has been working on is quite expansive, 
 and simply importing windows.d ends up pulling in a substantial chunk of 
 the windows modules.  This slows compilation noticeably and increases 
 binary size significantly--simply importing Don's windows.d in my test 
 app made the executable size increase from ~90K to over 150K.  This 
 appears to be related at least in part to what appears to be a fixed 
 minimum for importing a module using DMD of approximately 500 bytes, 
 which I assume is the ModuleInfo data and similar things.

It's pretty easy to see what's in a .obj file by running obj2asm on it.
 What I find confusing, however, is that I don't appear to need to link 
 in the standard C modules I've defined in order to use the structs 
 defined there.  I performed a quick test using struct tm and no 
 dependency problems came up simply by compiling the test app itself and 
 linking against a library without std.c.time in it.  Assuming I didn't 
 mess up the test somehow, can you explain the difference between this 
 situation and the Windows one?  If it's simply a matter of having the 
 static initializers available, I'm fine with building everything, but 
 the inconsistency doesn't seem to make sense.

You shouldn't need to link in the static initializer for a struct if the static initializer is all 0's.
Jun 09 2006
parent reply Sean Kelly <sean f4.ca> writes:
Walter Bright wrote:
 Sean Kelly wrote:
 Linking with the windows module present in Phobos isn't a big deal.  
 But the Windows header project Don has been working on is quite 
 expansive, and simply importing windows.d ends up pulling in a 
 substantial chunk of the windows modules.  This slows compilation 
 noticeably and increases binary size significantly--simply importing 
 Don's windows.d in my test app made the executable size increase from 
 ~90K to over 150K.  This appears to be related at least in part to 
 what appears to be a fixed minimum for importing a module using DMD of 
 approximately 500 bytes, which I assume is the ModuleInfo data and 
 similar things.

It's pretty easy to see what's in a .obj file by running obj2asm on it.

Done that. And for the smaller modules, the bulk of this does indeed appear to be ModuleInfo data.
 What I find confusing, however, is that I don't appear to need to link 
 in the standard C modules I've defined in order to use the structs 
 defined there.  I performed a quick test using struct tm and no 
 dependency problems came up simply by compiling the test app itself 
 and linking against a library without std.c.time in it.  Assuming I 
 didn't mess up the test somehow, can you explain the difference 
 between this situation and the Windows one?  If it's simply a matter 
 of having the static initializers available, I'm fine with building 
 everything, but the inconsistency doesn't seem to make sense.

You shouldn't need to link in the static initializer for a struct if the static initializer is all 0's.

Ah, that explains the dependencies then, as I believe all of the structs I listed have char arrays of some sort in them. I do still think it's a tad odd that the program size would increase by 50K simply for linking in the Windows dependencies for a "hello world" program though. For reference, this is the "hello.d" sample from Mango I've been referencing: http://svn.dsource.org/projects/mango/trunk/example/hello.d But perhaps it is merely that the Windows headers are a vast interdependent mess and this won't be an issue elsewhere. As a side note, this is also my first experience with using Derek's Build program and I have seen reports of it producing unusually large executables in the past. Is it possible that optlink can be tricked to not always discard all unnecessary data? I do find it strange that Build could compile an application using DMD and produce a different result than if the traditional library method were used. To this end, it does seem possible that a program with more information (ie. the compiler) could do a better job at this. I don't suppose you've considered a full compilation approach for DMD? If dependent modules must be opened and parsed anyway, it seems a fairly trivial extension to simply compile them as well. This would actually address a problem I noticed with Build in that it isn't able to detect if a module is a "header" module and will compile and link the code in anyway, which produces "previous definition different" errors at link time. Sean
Jun 09 2006
parent reply Bruno Medeiros <brunodomedeirosATgmail SPAM.com> writes:
Sean Kelly wrote:
  This would actually address a problem I 
 noticed with Build in that it isn't able to detect if a module is a 
 "header" module and will compile and link the code in anyway, which 
 produces "previous definition different" errors at link time.
 
 
 Sean

Hum, Have you used build exclusions (option -X) in order not to compile and link a module? -- Bruno Medeiros - CS/E student http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D
Jun 11 2006
parent reply "Derek Parnell" <derek psych.ward> writes:
On Mon, 12 Jun 2006 00:31:42 +1000, Bruno Medeiros  
<brunodomedeirosATgmail SPAM.com> wrote:

 Sean Kelly wrote:
  This would actually address a problem I noticed with Build in that it  
 isn't able to detect if a module is a "header" module and will compile  
 and link the code in anyway, which produces "previous definition  
 different" errors at link time.
   Sean

Hum, Have you used build exclusions (option -X) in order not to compile and link a module?

I think that the best option for now is to use version(build) pragma(nolink); This will ensure that the file containing this pragma is not compiled or linked. -- Derek Parnell Melbourne, Australia
Jun 11 2006
parent reply Sean Kelly <sean f4.ca> writes:
Derek Parnell wrote:
 On Mon, 12 Jun 2006 00:31:42 +1000, Bruno Medeiros 
 <brunodomedeirosATgmail SPAM.com> wrote:
 
 Sean Kelly wrote:
  This would actually address a problem I noticed with Build in that 
 it isn't able to detect if a module is a "header" module and will 
 compile and link the code in anyway, which produces "previous 
 definition different" errors at link time.
   Sean

Hum, Have you used build exclusions (option -X) in order not to compile and link a module?

I think that the best option for now is to use version(build) pragma(nolink); This will ensure that the file containing this pragma is not compiled or linked.

Yup. This is what I've been doing. Sean
Jun 11 2006
parent reply Bruno Medeiros <brunodomedeirosATgmail SPAM.com> writes:
Sean Kelly wrote:
 Derek Parnell wrote:
 On Mon, 12 Jun 2006 00:31:42 +1000, Bruno Medeiros 
 <brunodomedeirosATgmail SPAM.com> wrote:

 Sean Kelly wrote:
  This would actually address a problem I noticed with Build in that 
 it isn't able to detect if a module is a "header" module and will 
 compile and link the code in anyway, which produces "previous 
 definition different" errors at link time.
   Sean

Hum, Have you used build exclusions (option -X) in order not to compile and link a module?

I think that the best option for now is to use version(build) pragma(nolink); This will ensure that the file containing this pragma is not compiled or linked.

Yup. This is what I've been doing. Sean

Then what you meant about Build "isn't able to detect if a module is a "header" module" ? -- Bruno Medeiros - CS/E student http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D
Jun 11 2006
parent reply Sean Kelly <sean f4.ca> writes:
Bruno Medeiros wrote:
 Sean Kelly wrote:
 Derek Parnell wrote:
 On Mon, 12 Jun 2006 00:31:42 +1000, Bruno Medeiros 
 <brunodomedeirosATgmail SPAM.com> wrote:

 Sean Kelly wrote:
  This would actually address a problem I noticed with Build in that 
 it isn't able to detect if a module is a "header" module and will 
 compile and link the code in anyway, which produces "previous 
 definition different" errors at link time.
   Sean

Hum, Have you used build exclusions (option -X) in order not to compile and link a module?

I think that the best option for now is to use version(build) pragma(nolink); This will ensure that the file containing this pragma is not compiled or linked.

Yup. This is what I've been doing.

Then what you meant about Build "isn't able to detect if a module is a "header" module" ?

It isn't able to by itself--you have to tell it. And while I'm not certain DMD could do better, I'd like to believe so. That was my only point. That in the long term it may be more useful to have such features built directly into the compiler. In my brief experience with Build it's a darn sight better than generating C-style libraries. And I suspect it will get around the template linking issues I've run into to boot. Sean
Jun 11 2006
next sibling parent Bruno Medeiros <brunodomedeirosATgmail SPAM.com> writes:
Sean Kelly wrote:
 Bruno Medeiros wrote:
 Then what you meant about Build "isn't able to detect if a module is a 
 "header" module" ?

It isn't able to by itself--you have to tell it. And while I'm not certain DMD could do better, I'd like to believe so. That was my only point. That in the long term it may be more useful to have such features built directly into the compiler. In my brief experience with Build it's a darn sight better than generating C-style libraries. And I suspect it will get around the template linking issues I've run into to boot. Sean

How could Build, or *any* tool at all, know by itself if a D file is to be compiled or not? I don't think it's possible, unless the file extension is ".di". -- Bruno Medeiros - CS/E student http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D
Jun 11 2006
prev sibling parent reply "Derek Parnell" <derek psych.ward> writes:
On Mon, 12 Jun 2006 05:55:18 +1000, Sean Kelly <sean f4.ca> wrote:


  Then what you meant about Build "isn't able to detect if a module is a  
 "header" module" ?

It isn't able to by itself--you have to tell it. And while I'm not certain DMD could do better, I'd like to believe so.

The problem as I see it is how any anyone or anything know that a module is a 'hesder' module. The only two clear cut signals are a module that has *zero* implementation details or the module is presented as a .di file. The first signal could be detectable by Build and the second one currently is. But that doesn't handle other 'header' modules that don't fit into these two categories. That is why I adopted the simple method of having the coder put in the pragma to identify a module as a 'header' type. Can you give me the algorithm to use to detect that a module is a 'header' or not? Maybe if I had such an algorithm I might be able to implement it. -- Derek Parnell Melbourne, Australia
Jun 11 2006
parent Sean Kelly <sean f4.ca> writes:
Derek Parnell wrote:
 
 Can you give me the algorithm to use to detect that a module is a 
 'header' or not? Maybe if I had such an algorithm I might be able to 
 implement it.

It probably isn't possible to come up with a general solution. But I imagined that if the compiler could determine that no object level code generation were required (ie. the module contained only type definitions and inlinable code) then it could be assumed to be a header. But of course what's inlined varies from compiler to compiler, and then there's the issue of static ctors for structs and such. Being aware of .di files as build is now is probably the best and easiest solution. Sean
Jun 11 2006
prev sibling parent Sean Kelly <sean f4.ca> writes:
Sean Kelly wrote:
 The standard C headers I've defined don't need to be linked against my 
 app in any circumstances I've encountered so far--they serve as pure 
 import modules.  However, I've recently discovered that this does not 
 hold true for the Windows headers I use, which seem to require me to 
 link against them if I reference any struct they declare.  Is it 
 possible that extern (C) declarations tell the compiler that the struct 
 will be available elsewhere and extern (Windows) doesn't?  It's worth 
 noting that the link errors I get display a D-style mangled name for the 
 struct definitions rather than a C-style name as I'd expect.  Should 
 these structs all be declared as extern (C)?

By the way, I ask this because the executable size increases by a minimum of 500 bytes for every module included, even if they contain just declarations. The full Windows header set being what it is (thanks to Don and crew's efforts), this means more than a 50K increase in program size just for a "hello world" application if Build actually links all the Windows headers referenced. Sean
Jun 08 2006