www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Suggestion to use common Win32 headers

reply bobef <bobef abv-nospam.bg> writes:
Hi guys,

I run into this problem often. Each [win32] library uses its own win32 headers,
typically only what it needs. For example - phobos has some common headers,
tango has some minimal win32 headers, dfl has its own, dwt has its own, etc...
So if you use more than one of these libraries you run into conflicts. On top
of that their win32 headers support is limited and you need yet another headers
(which cause even more conflicts) or do what they do - manually copy paste from
msdn, which sucks :) 

So how could this be solved? By the adoption of full set of the win32 headers
by the both standard libraries - phobos and tango. Such comprehensive set of
headers exists in the dsource/bindings/win32 project. So if these headers are
present in the std lib, the the other major libraries like dfl, dwt, etc would
not need to redefine the win32 api and one would be able to use the same
headers which these libraries use, so you won't run into conflicts like
void(HANDLE,UINT) does not match void(HANDLE,UINT).

Maybe there is other way around - I don't know, but this seems a pretty good
solution to me.

Best regards,
bobef
Mar 05 2008
next sibling parent reply Trevor Parscal <trevorparscal hotmail.com> writes:
bobef Wrote:

 Hi guys,
 
 I run into this problem often. Each [win32] library uses its own win32
headers, typically only what it needs. For example - phobos has some common
headers, tango has some minimal win32 headers, dfl has its own, dwt has its
own, etc... So if you use more than one of these libraries you run into
conflicts. On top of that their win32 headers support is limited and you need
yet another headers (which cause even more conflicts) or do what they do -
manually copy paste from msdn, which sucks :) 
 
 So how could this be solved? By the adoption of full set of the win32 headers
by the both standard libraries - phobos and tango. Such comprehensive set of
headers exists in the dsource/bindings/win32 project. So if these headers are
present in the std lib, the the other major libraries like dfl, dwt, etc would
not need to redefine the win32 api and one would be able to use the same
headers which these libraries use, so you won't run into conflicts like
void(HANDLE,UINT) does not match void(HANDLE,UINT).
 
 Maybe there is other way around - I don't know, but this seems a pretty good
solution to me.
 
 Best regards,
 bobef

As the person who contributed Tango's minimal windows header, and also has authored a new even more restrictive header for my own project UniD, I feel I may be a good person to answer this question. "Windows headers" can mean just windows.h or poitentially hundreds of files which for the majority of users are not needed. Even with automatic conversion software (h2d) it is a tedious process to get bug-free D modules from C header files. So it's often been a question of scope; how many headers should we consider "common"? My approach with UniD is the least complete [1] - only defining the things I need - however I personally suggest going this route to supplement what Tango already has, and if it's working for you, offering it as a patch to Tango. The other issue is a licensing issue. When I hand write my own module from scratch based on documentation, I own it - or at least they don't own it and I can use it in anything without legal problems [2]. So even if you wrote a program that converted all the latest header files from a release of VisualStudio, there are licensing problems there. There are other renditions of the headers are are of an unrestricted nature [3] however, so if you were to take those, and convert them all, you could have what you want. So there you have it - the answer is "nobody has taken the time to convert everything yet, but you certainly can!" - Trevor [1] http://www.dsource.org/projects/unid/browser/trunk/source/uni/lib/platforms/Windows.d [2] AFAIK, but is this really true? [4] http://www.mingw.org/mingwfaq.shtml#faq-w32api
Mar 05 2008
parent reply bobef <bobef abv-nospam.bg> writes:
Trevor Parscal Wrote:

 As the person who contributed Tango's minimal windows header, and also has
authored a new even more restrictive header for my own project UniD, I feel I
may be a good person to answer this question.
 

I didn't intend to insult you by saying 'minimal'. In this context 'minimal' means only the types and constants basically, which are very useful, of course. The problem is 1) These are not common with Phobos. I.e. Phobos may provide less (or more) win32 types and constants. 2) Other libraries are not using these (probably because of 1) 3) This causes conflicts 4) There are no functions, only types and constants
 "Windows headers" can mean just windows.h or poitentially hundreds of files
which for the majority of users are not needed. Even with automatic conversion
software (h2d) it is a tedious process to get bug-free D modules from C header
files.
 
 So it's often been a question of scope; how many headers should we consider
"common"?
 

I am not saying 'common'. I am saying all (or most of) win32 headers. There is no need for conversations or anything. The headers are already there - http://dsource.org/projects/bindings/browser/trunk/win32 So what needs to be done, IMHO, is for Phobos and Tango to adopt these. So one could just write import win32.windows; The only problem I see with this is that these headers are massive and not needed by linux users. I don't know this could be solved. But I see a bigger problem when I have to manually define each win32 function that I am using even though there are the appropriate headers available. And yet they remain virtually useless because they conflict with at least one more library. To outline the problem once more- imagine this scenario (which I believe is not uncommon for win32 apps): You need to write a Win32 GUI app that does more than just windows. So, naturally, you choose to use DFL or DWT (I don't know if wxD or gtkD suffers from these problems) to do your windows. The first problem arises when these (DFL or DWT) conflict with Tango/Phobos win32 declarations. This is somehow solvable but still very annoying. Now if you need to do some not-so-common task and need some win32 functionality what do you do? 1) copy function declarations from MSDN or existing D headers, which is annoying and potentially the cause of conflicts. For example - the function you have just specified needs a HANDLE from, say, DFL. You write something like extern(Windows) VOID SomeWin32Function(HANDLE); And here is the conflict. I include tango.sys.win32.Types in my program, not the internal DFL stuff, but DFL returns HANDLE that is defined in it's internal files, not in tango....Types. You could probably include dfl.internal.winapi or something like that, instead of tango.Types, but this is ugly and not applicable to libraries that needs to be distributed. 2) try to use some existing headers like the ones in the bindings projects. But they become absolutely unusable because they conflict with both Tango and DFL. Why with both? Because if you are writing a library you can't make it depend (in most cases) on DFL's internal winapi definition, so you use tango.sys.win32.Types where you need win32 types and DFL uses its internal stuff and you get 3 conflicts. So you are stuck with point 1 3) correct me if I am wrong
Mar 05 2008
parent reply Sean Kelly <sean invisibleduck.org> writes:
== Quote from bobef (bobef abv-nospam.bg)'s article
 Trevor Parscal Wrote:
 As the person who contributed Tango's minimal windows header, and also has
authored a new even




 1) These are not common with Phobos. I.e. Phobos may provide less (or more)
win32 types and

 2) Other libraries are not using these (probably because of 1)
 3) This causes conflicts
 4) There are no functions, only types and constants
 "Windows headers" can mean just windows.h or poitentially hundreds of files
which for the majority


bug-free D modules from C header files.
 So it's often been a question of scope; how many headers should we consider
"common"?


http://dsource.org/projects/bindings/browser/trunk/win32
 So what needs to be done, IMHO, is for Phobos and Tango to adopt these.

Tango actually originally used these headers, but dropped them early on because there were some problems with them. They may since have been fixed, but the issues I remember are: * The headers are immense and interlinked. Using basically any of them in a Windows app increased binary size tremendously. * Some of the headers were broken, and because of the size and number of headers, fixing them proved to be somewhat complicated. The extension of this being that maintaining the headers promised to be a considerable task that no one wanted to take on. * Windows has some versioned headers that conflict with one another, such as the winsock and winsock2 headers. The bindings project didn't provide any good way to handle this at the time, and it caused problems with some applications (for me anyway). Also, very few people really need the bulk of what's in the Windows headers. Tango has been using Trevor's headers for ages now, and the number of requests for Windows header additions is practically nil. I'm inclined to say that the bindings project is really better suited to be maintained and distributed separately. It decreases maintenance load on Tango/Phobos, allows the default impl. to remain slim, and still allows those who have need of the full set to download/use them. It's probably also worth noting that neither Tango or Phobos contain a full set of headers for any other OS either. Sean
Mar 05 2008
next sibling parent reply Sascha Katzner <sorry.no spam.invalid> writes:
Sean Kelly wrote:
 Using basically any of them in a Windows app increased binary size
 tremendously.

That is simply wrong. Perhaps it was true in the far past, but as long as I know/commit to the Win32API project (~1 year) that was never the case.
 * Some of the headers were broken, and because of the size and number
 of headers, fixing them proved to be somewhat complicated.  The
 extension of this being that maintaining the headers promised to be a
 considerable task that no one wanted to take on.

The only file at the moment which is problematic is "winsock.d", because nobody seems to care enough to fix it... I think that is because there is no reason for it, if you have "winsock2.d". But nonetheless this is no problem, because the makefile accounts for this. The whole build process not even gives a warning. All you have to do, to use the Win32API headers: 1) type "make" in the project directory 2) link "win32.lib" and add the project directory to your include path
 * Windows has some versioned headers that conflict with one another,
 such as the winsock and winsock2 headers.  The bindings project
 didn't provide any good way to handle this at the time, and it caused
 problems with some applications (for me anyway).

In the build process you can specify which version do you want, the default in the makefile is for example: DFLAGS += -version=Unicode -version=WindowsVista but you can easily specify other versions like "Windows2003", "WindowsXP" etc. ... this is all defined in "w32api.d".
 Also, very few people really need the bulk of what's in the Windows
 headers.  Tango has been using Trevor's headers for ages now, and the
 number of requests for Windows header additions is practically nil.

I use them for every project... but then again I'm kind of a purist and don't like the existing window libraries in D. ;) LLAP, Sascha Project: http://www.dsource.org/projects/bindings/wiki/WindowsApi Forum: http://www.dsource.org/forums/viewforum.php?f=30
Mar 06 2008
next sibling parent reply Lawrence <lawrence.hemsley gmail.com> writes:
Sascha Katzner Wrote:

 Sean Kelly wrote:
 Using basically any of them in a Windows app increased binary size
 tremendously.

That is simply wrong. Perhaps it was true in the far past, but as long as I know/commit to the Win32API project (~1 year) that was never the case.
 * Some of the headers were broken, and because of the size and number
 of headers, fixing them proved to be somewhat complicated.  The
 extension of this being that maintaining the headers promised to be a
 considerable task that no one wanted to take on.

The only file at the moment which is problematic is "winsock.d", because nobody seems to care enough to fix it... I think that is because there is no reason for it, if you have "winsock2.d". But nonetheless this is no problem, because the makefile accounts for this. The whole build process not even gives a warning. All you have to do, to use the Win32API headers: 1) type "make" in the project directory 2) link "win32.lib" and add the project directory to your include path
 * Windows has some versioned headers that conflict with one another,
 such as the winsock and winsock2 headers.  The bindings project
 didn't provide any good way to handle this at the time, and it caused
 problems with some applications (for me anyway).

In the build process you can specify which version do you want, the default in the makefile is for example: DFLAGS += -version=Unicode -version=WindowsVista but you can easily specify other versions like "Windows2003", "WindowsXP" etc. ... this is all defined in "w32api.d".
 Also, very few people really need the bulk of what's in the Windows
 headers.  Tango has been using Trevor's headers for ages now, and the
 number of requests for Windows header additions is practically nil.

I use them for every project... but then again I'm kind of a purist and don't like the existing window libraries in D. ;) LLAP, Sascha Project: http://www.dsource.org/projects/bindings/wiki/WindowsApi Forum: http://www.dsource.org/forums/viewforum.php?f=30

I am very new to D and I am not familiar with dsource.org. How to I get this library. lh
Mar 06 2008
parent reply Sascha Katzner <sorry.no spam.invalid> writes:
Lawrence wrote:
 Sascha Katzner Wrote:
 Project: http://www.dsource.org/projects/bindings/wiki/WindowsApi
 Forum:   http://www.dsource.org/forums/viewforum.php?f=30

I am very new to D and I am not familiar with dsource.org. How to I get this library.

Either you use a Subversion client (like tortoisesvn.tigris.org) to download the trunk or you use the following URL to download a Zip file with the latest revision: http://www.dsource.org/projects/bindings/changeset/251/trunk/win32?old_path=%2F&format=zip "251" in this URL is the revision and should be changed to the latest version. LLAP, Sascha
Mar 06 2008
parent reply Lawrence <lawrence.hemsley gmail.com> writes:
Sascha Katzner Wrote:

 Lawrence wrote:
 Sascha Katzner Wrote:
 Project: http://www.dsource.org/projects/bindings/wiki/WindowsApi
 Forum:   http://www.dsource.org/forums/viewforum.php?f=30

I am very new to D and I am not familiar with dsource.org. How to I get this library.

Either you use a Subversion client (like tortoisesvn.tigris.org) to download the trunk or you use the following URL to download a Zip file with the latest revision: http://www.dsource.org/projects/bindings/changeset/251/trunk/win32?old_path=%2F&format=zip "251" in this URL is the revision and should be changed to the latest version. LLAP, Sascha

Thanks for the information and the link. I have actually tried to use tortoise svn but I have not been able to get it to work for me. I do not know why for sure but I get 3 bytes with two "PROPEND" what ever that means. lh
Mar 07 2008
parent reply Lawrence <lawrence.hemsley gmail.com> writes:
Lawrence Wrote:

 Sascha Katzner Wrote:
 
 Lawrence wrote:
 Sascha Katzner Wrote:
 Project: http://www.dsource.org/projects/bindings/wiki/WindowsApi
 Forum:   http://www.dsource.org/forums/viewforum.php?f=30

I am very new to D and I am not familiar with dsource.org. How to I get this library.

Either you use a Subversion client (like tortoisesvn.tigris.org) to download the trunk or you use the following URL to download a Zip file with the latest revision: http://www.dsource.org/projects/bindings/changeset/251/trunk/win32?old_path=%2F&format=zip "251" in this URL is the revision and should be changed to the latest version. LLAP, Sascha

Thanks for the information and the link. I have actually tried to use tortoise svn but I have not been able to get it to work for me. I do not know why for sure but I get 3 bytes with two "PROPEND" what ever that means. lh

Sorry, it is late for me. What I get returned is 3 bytes and two errors that start with the word "PROPEND". lh
Mar 07 2008
parent reply Sascha Katzner <sorry.no spam.invalid> writes:
Lawrence wrote:
 Sorry, it is late for me. What I get returned is 3 bytes and two
 errors that start with the word "PROPEND".

Sorry I don't know where this error comes from. All you need to do with Tortoise SVN is click right mouse button on a directory -> "SVN Checkout". Then input the URL of the repository: http://svn.dsource.org/projects/bindings/trunk/win32 "HEAD revision" should automatically be marked (means the newest version). Et Voilą, you have the newest version of the Win32 API headers in your directory ready to use. LLAP, Sascha
Mar 07 2008
parent Lawrence <lawrence.hemsley gmail.com> writes:
Sascha Katzner Wrote:

 Lawrence wrote:
 Sorry, it is late for me. What I get returned is 3 bytes and two
 errors that start with the word "PROPEND".

Sorry I don't know where this error comes from. All you need to do with Tortoise SVN is click right mouse button on a directory -> "SVN Checkout". Then input the URL of the repository: http://svn.dsource.org/projects/bindings/trunk/win32 "HEAD revision" should automatically be marked (means the newest version). Et Voilą, you have the newest version of the Win32 API headers in your directory ready to use. LLAP, Sascha

Thanks again! I was leaving out the svn. of the URL. Yes, I am new to SVN also. I used the correct URL and got an authentication request which I did not have. My guess is I need to sign up at dsource.org? lh
Mar 07 2008
prev sibling parent Sean Kelly <sean invisibleduck.org> writes:
== Quote from Sascha Katzner (sorry.no spam.invalid)'s article
 Sean Kelly wrote:
 Using basically any of them in a Windows app increased binary size
 tremendously.

as I know/commit to the Win32API project (~1 year) that was never the case.

It was when the package was in Tango, but that was over 1.5 years ago I believe. The problem was that all the constants were labeled 'const' rather than 'enum'.
 * Some of the headers were broken, and because of the size and number
 of headers, fixing them proved to be somewhat complicated.  The
 extension of this being that maintaining the headers promised to be a
 considerable task that no one wanted to take on.

nobody seems to care enough to fix it... I think that is because there is no reason for it, if you have "winsock2.d". But nonetheless this is no problem, because the makefile accounts for this. The whole build process not even gives a warning.

Then things have improved tremendously since we looked at them, which was admittedly quite a while ago. I'll admit also that one reason Tango didn't stick with those headers was that, given the state they were in at the time they appeared to require a dedicated maintainer, and neither Kris nor myself (the two principal Windows folks at the time) had the time or inclination to do so.
 All you have to do, to use the Win32API headers:
 1) type "make" in the project directory
 2) link "win32.lib" and add the project directory to your include path
 * Windows has some versioned headers that conflict with one another,
 such as the winsock and winsock2 headers.  The bindings project
 didn't provide any good way to handle this at the time, and it caused
 problems with some applications (for me anyway).

default in the makefile is for example: DFLAGS += -version=Unicode -version=WindowsVista but you can easily specify other versions like "Windows2003", "WindowsXP" etc. ... this is all defined in "w32api.d".
 Also, very few people really need the bulk of what's in the Windows
 headers.  Tango has been using Trevor's headers for ages now, and the
 number of requests for Windows header additions is practically nil.

don't like the existing window libraries in D. ;)

Me either :) But they do the trick for what's in Tango. From what you've said though, perhaps it's time to revisit this decision. If the full headers could be used in Tango without adversely affecting binary size then I'd certainly consider the idea. Sean
Mar 06 2008
prev sibling parent reply Georg Wrede <georg nospam.org> writes:
Sean, it seems to me you're writing in a window that's wider than 80 
chars, AND that for some reason the outgoing text seems not to flow 
(that is, if I, or you yourself look at it in an 80 character window, 
then every line is split into two, with the remaining couple of words on 
the latter line).

Ok, it's merely an annoyance, but I thougth to mention it, since you 
yourself wouldn't see the problem reading your own posts on your machine.


Sean Kelly wrote:
 == Quote from bobef (bobef abv-nospam.bg)'s article
 
Trevor Parscal Wrote:

As the person who contributed Tango's minimal windows header, and also has
authored a new even


more restrictive header for my own project UniD, I feel I may be a good person to answer this question.
I didn't intend to insult you by saying 'minimal'. In this context 'minimal'
means only the types and

constants basically, which are very useful, of course. The problem is
1) These are not common with Phobos. I.e. Phobos may provide less (or more)
win32 types and

constants.
2) Other libraries are not using these (probably because of 1)
3) This causes conflicts
4) There are no functions, only types and constants

"Windows headers" can mean just windows.h or poitentially hundreds of files
which for the majority


of users are not needed. Even with automatic conversion software (h2d) it is a tedious process to get bug-free D modules from C header files.
So it's often been a question of scope; how many headers should we consider
"common"?

I am not saying 'common'. I am saying all (or most of) win32 headers. There is no need for

conversations or anything. The headers are already there - http://dsource.org/projects/bindings/browser/trunk/win32
So what needs to be done, IMHO, is for Phobos and Tango to adopt these.

Tango actually originally used these headers, but dropped them early on because there were some problems with them. They may since have been fixed, but the issues I remember are: * The headers are immense and interlinked. Using basically any of them in a Windows app increased binary size tremendously. * Some of the headers were broken, and because of the size and number of headers, fixing them proved to be somewhat complicated. The extension of this being that maintaining the headers promised to be a considerable task that no one wanted to take on. * Windows has some versioned headers that conflict with one another, such as the winsock and winsock2 headers. The bindings project didn't provide any good way to handle this at the time, and it caused problems with some applications (for me anyway). Also, very few people really need the bulk of what's in the Windows headers. Tango has been using Trevor's headers for ages now, and the number of requests for Windows header additions is practically nil. I'm inclined to say that the bindings project is really better suited to be maintained and distributed separately. It decreases maintenance load on Tango/Phobos, allows the default impl. to remain slim, and still allows those who have need of the full set to download/use them. It's probably also worth noting that neither Tango or Phobos contain a full set of headers for any other OS either.

Mar 09 2008
next sibling parent darrylb <a b.c> writes:
Maybe, just maybe, someday we'll have screens with decent resolutions and not
be tied to arbitrary limits that will seem just plain silly in retrospect.

I know, it sounds fantastic. But a man can dream, can't he?


(Consider, if you can't see his posts correctly, is that his problem.. or
yours?)


Georg Wrede Wrote:

 
 Sean, it seems to me you're writing in a window that's wider than 80 
 chars, AND that for some reason the outgoing text seems not to flow 
 (that is, if I, or you yourself look at it in an 80 character window, 
 then every line is split into two, with the remaining couple of words on 
 the latter line).
 
 Ok, it's merely an annoyance, but I thougth to mention it, since you 
 yourself wouldn't see the problem reading your own posts on your machine.

Mar 09 2008
prev sibling parent Sean Kelly <sean invisibleduck.org> writes:
== Quote from Georg Wrede (georg nospam.org)'s article
 Sean, it seems to me you're writing in a window that's wider than 80
 chars, AND that for some reason the outgoing text seems not to flow
 (that is, if I, or you yourself look at it in an 80 character window,
 then every line is split into two, with the remaining couple of words on
 the latter line).

Yup. I switched from posting via Thunderbird to the web client recently, so I can't really control the output. I'll have to install Thunderbird or some other newsreader on my new laptop (don't need it for email any more so it hasn't been a priority). Sean
Mar 09 2008
prev sibling next sibling parent "Vladimir Panteleev" <thecybershadow gmail.com> writes:
On Wed, 05 Mar 2008 19:53:31 +0200, bobef <bobef abv-nospam.bg> wrote:

 Hi guys,

 I run into this problem often. Each [win32] library uses its own win32
headers, typically only what it needs. For example - phobos has some common
headers, tango has some minimal win32 headers, dfl has its own, dwt has its
own, etc... So if you use more than one of these libraries you run into
conflicts. On top of that their win32 headers support is limited and you need
yet another headers (which cause even more conflicts) or do what they do -
manually copy paste from msdn, which sucks :)

 So how could this be solved? By the adoption of full set of the win32 headers
by the both standard libraries - phobos and tango. Such comprehensive set of
headers exists in the dsource/bindings/win32 project. So if these headers are
present in the std lib, the the other major libraries like dfl, dwt, etc would
not need to redefine the win32 api and one would be able to use the same
headers which these libraries use, so you won't run into conflicts like
void(HANDLE,UINT) does not match void(HANDLE,UINT).

 Maybe there is other way around - I don't know, but this seems a pretty good
solution to me.

I agree with this. Trying to use Tangobos+Tango+DFL in one project makes using Win32 API quite a mess. However, for the reasons Sean mentioned, I suggest that instead of adopting the headers, the libraries introduce a version() block to use the much more complete Win32 bindings project. -- Best regards, Vladimir mailto:thecybershadow gmail.com
Mar 06 2008
prev sibling parent Don Clugston <dac nospam.com.au> writes:
bobef wrote:
 Hi guys,
 
 I run into this problem often. Each [win32] library uses its own win32
headers, typically only what it needs. For example - phobos has some common
headers, tango has some minimal win32 headers, dfl has its own, dwt has its
own, etc... So if you use more than one of these libraries you run into
conflicts. On top of that their win32 headers support is limited and you need
yet another headers (which cause even more conflicts) or do what they do -
manually copy paste from msdn, which sucks :) 

That is exactly why the win32 project was begun -- the current situation is ridiculous.
 So how could this be solved? By the adoption of full set of the win32 headers
by the both standard libraries - phobos and tango. Such comprehensive set of
headers exists in the dsource/bindings/win32 project. So if these headers are
present in the std lib, the the other major libraries like dfl, dwt, etc would
not need to redefine the win32 api and one would be able to use the same
headers which these libraries use, so you won't run into conflicts like
void(HANDLE,UINT) does not match void(HANDLE,UINT).

A few important points: 1. IIRC, Walter has permission from Microsoft to redistribute the Windows headers. 2. The headers in the Win32 project weren't used because all the values declared as 'const' were stored in the obj files. Because of this, D2.0 introduced syntax for manifest constants (using 'enum' -- an unpopular choice, but it does provide the functionality). 3. Another annoyance is that the .lib files that come with the compiler are incomplete and date from an old version of Windows. It seems to be easy to fix (3) at least -- Walter just needs to download the latest WinSDK, run coffimplib on each .lib file, then add them to the distribution.
Mar 06 2008