www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Windows API and druntime/Phobos

reply Jonathan M Davis <jmdavisProg gmx.com> writes:
The level of support for the Windows API in druntime and Phobos is pretty low. 
As I understand it, Windows users are pretty much forced to use 
http://www.dsource.org/projects/bindings/browser/trunk/win32 if they need 
comprehensive Win32 API bindings. druntime seems to be trying to define all of 
the OS-specific stuff like that, but on top of it missing much of it, in the 
case of the Win32 API, that's a _lot_ of functions, and I don't know if we 
want to put that much in druntime. So, the question is, how do we want to 
support the Win32 API in druntime and Phobos?

Do we want to put all of the Win32 API bindings in druntime? If not, then do 
we want to put them in Phobos? Or do we just want to send Windows developers 
to a 3rd party library like the Win32 bindings project on dsource? Given that 
they're OS bindings, I would _think_ that we'd want them in druntime, but I 
don't know.

Regardless, this is one of those issues which frequently plagues D Windows 
developers, and we really should at least get a plan together as to how we 
want to handle it.

- Jonathan M Davis


P.S. A related pull request: https://github.com/D-Programming-
Language/druntime/pull/139
Jan 24 2012
next sibling parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Wednesday, 25 January 2012 at 19:16:34 UTC, Jonathan M Davis 
wrote:

 The level of support for the Windows API in druntime and Phobos 
 is pretty low. As I understand it, Windows users are pretty 
 much forced to use 
 http://www.dsource.org/projects/bindings/browser/trunk/win32 if 
 they need comprehensive Win32 API bindings. druntime seems to 
 be trying to define all of the OS-specific stuff like that, but 
 on top of it missing much of it, in the case of the Win32 API, 
 that's a _lot_ of functions, and I don't know if we want to put 
 that much in druntime. So, the question is, how do we want to 
 support the Win32 API in druntime and Phobos?

 Do we want to put all of the Win32 API bindings in druntime? If 
 not, then do we want to put them in Phobos? Or do we just want 
 to send Windows developers to a 3rd party library like the 
 Win32 bindings project on dsource? Given that they're OS 
 bindings, I would _think_ that we'd want them in druntime, but 
 I don't know.


I support putting all public domain modules from the Win32 
bindings project into Druntime.

The entirety of the Windows API modules (headers in C) and 
accompanying import libraries are an integral part of The Windows 
SDK. There is no obvious finer-grained separation of modules, so 
including only a part of them would need to be done on mostly 
subjective decisions.

The only non-public-domain modules are the DirectX headers (which 
don't belong in Druntime either way), and WinSock modules, which 
are "Portions Copyright (c) 1980, 1983, 1988, 1993 The Regents of 
the University of California.  All rights reserved." and 
"Portions Copyright (c) 1993 by Digital Equipment Corporation."
Jan 25 2012
next sibling parent Trass3r <un known.com> writes:
 There is no obvious finer-grained separation of modules, so including  
 only a part of them would need to be done on mostly subjective decisions.


Yep, Walter just added what he used.
Jan 25 2012
prev sibling next sibling parent reply "Kagamin" <spam here.lot> writes:
On Wednesday, 25 January 2012 at 19:27:47 UTC, Vladimir Panteleev 
wrote:

 I support putting all public domain modules from the Win32 
 bindings project into Druntime.

 The entirety of the Windows API modules (headers in C) and 
 accompanying import libraries are an integral part of The 
 Windows SDK.


Public domain? PSDK is not public domain. Isn't "Microsoft & 
public domain" an oxymoron?
Jan 27 2012
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Friday, 27 January 2012 at 18:16:38 UTC, Kagamin wrote:

 On Wednesday, 25 January 2012 at 19:27:47 UTC, Vladimir 
 Panteleev wrote:

 I support putting all public domain modules from the Win32 
 bindings project into Druntime.

 The entirety of the Windows API modules (headers in C) and 
 accompanying import libraries are an integral part of The 
 Windows SDK.


 Public domain? PSDK is not public domain. Isn't "Microsoft & 
 public domain" an oxymoron?


The files in the Windows API bindings project are translations 
from the MinGW headers, which were created based on the 
documentation (which explains the occasional errors). The MinGW 
headers are public domain. Considering that MinGW is not an 
obscure project, I think we're on safe legal ground here.
Jan 27 2012
parent reply "Kagamin" <spam here.lot> writes:
On Friday, 27 January 2012 at 20:37:04 UTC, Vladimir Panteleev 
wrote:

 The files in the Windows API bindings project are translations 
 from the MinGW headers, which were created based on the 
 documentation (which explains the occasional errors). The MinGW 
 headers are public domain. Considering that MinGW is not an 
 obscure project, I think we're on safe legal ground here.


Things like CRITICAL_SECTION are documented as opaque structures.
http://social.msdn.microsoft.com/Forums/en-US/winembplatdev/thread/3856c437-9a7e-4315-a954-fc50e25a12f2
Jan 28 2012
parent reply "Kagamin" <spam here.lot> writes:
I also remember MinGW folks were asked about this, but they don't 
know, where the headers came from.
Jan 28 2012
parent "Marco Leise" <Marco.Leise gmx.de> writes:
Am 28.01.2012, 11:22 Uhr, schrieb Kagamin <spam here.lot>:


 I also remember MinGW folks were asked about this, but they don't know,  
 where the headers came from.


I understand that Microsoft prefers Visual Studio to be used with one of  
their compilers, but you can't honestly sue anyone nowadays for copying  
headers to interface with the OS. It is Windows, the most used OS on  
consumer PCs. Who's gain was it if a major part of open source software  
was locked out because they couldn't port the application due to missing  
OS headers. If you always looked only on the legal side of things, you  
would be allowed and disallowed absurd things:
http://www.guardian.co.uk/politics/2007/nov/07/uk.queensspeech20072
The other ingredient is common sense. - So you don't buy Visual Studio,  
which may be a loss for MS, if those OSS developers would have switched to  
Windows, bought VS etc... in the other case, Windows loses popularity as a  
platform, because a lot of software wont be ported. With the background of  
the legal debate about making IE4 a part of the operating system I'd  
assume that some degree of freedom and choice of software is today  
considered almost as valuable as a formal copyright in a header file.
It would be different if WINE somehow got the Windows source code and was  
only through this event becoming an alternative that made Windows  
obsolete. But I think we had the header/interface discussion here already  
:)
Jan 29 2012
prev sibling parent reply Sean Kelly <sean invisibleduck.org> writes:
On Jan 25, 2012, at 11:27 AM, Vladimir Panteleev wrote:


 On Wednesday, 25 January 2012 at 19:16:34 UTC, Jonathan M Davis wrote:

 The level of support for the Windows API in druntime and Phobos is =




pretty low. As I understand it, Windows users are pretty much forced to =
use http://www.dsource.org/projects/bindings/browser/trunk/win32 if they =
need comprehensive Win32 API bindings. druntime seems to be trying to =
define all of the OS-specific stuff like that, but on top of it missing =
much of it, in the case of the Win32 API, that's a _lot_ of functions, =
and I don't know if we want to put that much in druntime. So, the =
question is, how do we want to support the Win32 API in druntime and =
Phobos?

=20
 Do we want to put all of the Win32 API bindings in druntime? If not, =




then do we want to put them in Phobos? Or do we just want to send =
Windows developers to a 3rd party library like the Win32 bindings =
project on dsource? Given that they're OS bindings, I would _think_ that =
we'd want them in druntime, but I don't know.

=20
 I support putting all public domain modules from the Win32 bindings =


project into Druntime.

=20
 The entirety of the Windows API modules (headers in C) and =


accompanying import libraries are an integral part of The Windows SDK. =
There is no obvious finer-grained separation of modules, so including =
only a part of them would need to be done on mostly subjective =
decisions.

=20
 The only non-public-domain modules are the DirectX headers (which =


don't belong in Druntime either way), and WinSock modules, which are =
"Portions Copyright (c) 1980, 1983, 1988, 1993 The Regents of the =
University of California.  All rights reserved." and "Portions Copyright =
(c) 1993 by Digital Equipment Corporation."

For what it's worth, I tried integrating these headers ages ago (into =
Tango I believe) and stopped mostly because they were broken at the =
time.  I imagine they're sorted by now, and so long as there's little =
maintenance effort on my part and there are no license concerns I'm all =
for it.  The only other thing to be aware of is that each module linked =
into an executable incurs some overhead (500 bytes minimum, if I =
remember correctly), so moving to these new Win32 headers will increase =
the footprint of D apps on Windows.  I imagine it's a worthwhile =
tradeoff, but I did want to bring this up so no one is surprised later.=
Jan 27 2012
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Friday, 27 January 2012 at 18:36:28 UTC, Sean Kelly wrote:

 The only other thing to be aware of is that each module linked 
 into an executable incurs some overhead (500 bytes minimum, if 
 I remember correctly), so moving to these new Win32 headers 
 will increase the footprint of D apps on Windows.  I imagine 
 it's a worthwhile tradeoff, but I did want to bring this up so 
 no one is surprised later.


Would it be possible to fix this overhead in the compiler 
(avoiding generation of such overhead for modules without 
classes, static ctors/dtors, etc.)?

Even though Windows API programs classically begin with #include 
<windows.h>, which includes most of the other headers, this isn't 
a requirement, and it's possible to import modules on a 
finer-grained basis.
Jan 27 2012
next sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
I really hope druntime doesn't get one gigantic windows.d file. I
don't want to wait seconds for syntax highlighting to kick in because
a file is huge.
Jan 28 2012
parent "Marco Leise" <Marco.Leise gmx.de> writes:
Am 28.01.2012, 09:41 Uhr, schrieb Andrej Mitrovic  
<andrej.mitrovich gmail.com>:


 I really hope druntime doesn't get one gigantic windows.d file. I
 don't want to wait seconds for syntax highlighting to kick in because
 a file is huge.


No, it should be done like everywhere else where the IDEs have no problem  
dealing with the Windows headers. C++, Delphi, ... . I think COM and  
WinSock are usually separate, whereas user32 and kernel32 are commonly  
imported through one file. I may be wrong. If a particular IDE/editor is  
slow it must be fixed :)
Jan 29 2012
prev sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday, January 28, 2012 09:41:21 Andrej Mitrovic wrote:

 I really hope druntime doesn't get one gigantic windows.d file. I
 don't want to wait seconds for syntax highlighting to kick in because
 a file is huge.


It will definitely be more organized than that. I would think that it would be 
organized along the lines of how the win32 API itself is organized, but I 
don't know. It probably also depends on what the current bindings project 
does. A giant windows.d would definitely be rejected though.

- Jonathan M Davis
Jan 28 2012
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Saturday, 28 January 2012 at 09:22:33 UTC, Jonathan M Davis 
wrote:

 On Saturday, January 28, 2012 09:41:21 Andrej Mitrovic wrote:

 I really hope druntime doesn't get one gigantic windows.d 
 file. I
 don't want to wait seconds for syntax highlighting to kick in 
 because
 a file is huge.


 It will definitely be more organized than that. I would think 
 that it would be organized along the lines of how the win32 API 
 itself is organized, but I don't know. It probably also depends 
 on what the current bindings project does. A giant windows.d 
 would definitely be rejected though.


http://www.dsource.org/projects/bindings/browser/trunk/win32/windows.d
Jan 28 2012
parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 1/28/12, Vladimir Panteleev <vladimir thecybershadow.net> wrote:

 http://www.dsource.org/projects/bindings/browser/trunk/win32/windows.d


Well sheeeit, somehow I never noticed that file being there.
Jan 28 2012
prev sibling next sibling parent reply Don <nospam nospam.com> writes:
On 25.01.2012 05:48, Jonathan M Davis wrote:

 The level of support for the Windows API in druntime and Phobos is pretty low.
 As I understand it, Windows users are pretty much forced to use
 http://www.dsource.org/projects/bindings/browser/trunk/win32 if they need
 comprehensive Win32 API bindings. druntime seems to be trying to define all of
 the OS-specific stuff like that, but on top of it missing much of it, in the
 case of the Win32 API, that's a _lot_ of functions, and I don't know if we
 want to put that much in druntime. So, the question is, how do we want to
 support the Win32 API in druntime and Phobos?

 Do we want to put all of the Win32 API bindings in druntime? If not, then do
 we want to put them in Phobos? Or do we just want to send Windows developers
 to a 3rd party library like the Win32 bindings project on dsource? Given that
 they're OS bindings, I would _think_ that we'd want them in druntime, but I
 don't know.

 Regardless, this is one of those issues which frequently plagues D Windows
 developers, and we really should at least get a plan together as to how we
 want to handle it.

 - Jonathan M Davis


 P.S. A related pull request: https://github.com/D-Programming-
 Language/druntime/pull/139


Please note that the Win32 API project on dsource was created with the 
explicit intention of including it in Phobos. See bug 317.
Jan 25 2012
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday, January 25, 2012 20:38:01 Don wrote:

 Please note that the Win32 API project on dsource was created with the
 explicit intention of including it in Phobos. See bug 317.


Good to know. I would point out though that for the most part, std.c has been 
being phased out with all of its parts being moved to Phobos, so I'm not sure 
that putting the Win32 API bindings in std.c.windows makes sense.

Regardless, my point is that we need to figure out what we want to do at this 
point. As it stands, we end up with pull requests such as

https://github.com/D-Programming-Language/druntime/pull/139

which just add more bindings which a particular programmer happens to want. 
And while that might be okay in the short term, that doesn't work in the long 
term. We need to decide where we want the Win32 API bindings, how we want them 
organized more or less, and start putting them there. The Win32 API project is 
a great start, but it's been around for years without being moved into 
druntime or Phobos, and we should start looking at moving those bindings into 
either druntime or Phobos. At this point, given that they're OS bindings, and 
the OS bindings have been going in druntime, I would expect them to go into 
druntime. But a decision must be made, or the situation won't improve.

- Jonathan M Davis
Jan 25 2012
parent Jacob Carlborg <doob me.com> writes:
On 2012-01-26 00:50, Jonathan M Davis wrote:

 On Wednesday, January 25, 2012 20:38:01 Don wrote:

 Please note that the Win32 API project on dsource was created with the
 explicit intention of including it in Phobos. See bug 317.


 Good to know. I would point out though that for the most part, std.c has been
 being phased out with all of its parts being moved to Phobos, so I'm not sure
 that putting the Win32 API bindings in std.c.windows makes sense.

 Regardless, my point is that we need to figure out what we want to do at this
 point. As it stands, we end up with pull requests such as

 https://github.com/D-Programming-Language/druntime/pull/139

 which just add more bindings which a particular programmer happens to want.
 And while that might be okay in the short term, that doesn't work in the long
 term. We need to decide where we want the Win32 API bindings, how we want them
 organized more or less, and start putting them there. The Win32 API project is
 a great start, but it's been around for years without being moved into
 druntime or Phobos, and we should start looking at moving those bindings into
 either druntime or Phobos. At this point, given that they're OS bindings, and
 the OS bindings have been going in druntime, I would expect them to go into
 druntime. But a decision must be made, or the situation won't improve.

 - Jonathan M Davis


It depends. If we want to keep druntime small and only contain what's 
necessary for the runtime it would be better to add the bindings to 
Phobos. But that would mean to duplicate parts of the bindings.

-- 
/Jacob Carlborg
Jan 25 2012
prev sibling parent reply "Jesse Phillips" <jessekphillips+D gmail.com> writes:
On Wednesday, 25 January 2012 at 19:16:34 UTC, Jonathan M Davis 
wrote:

 The level of support for the Windows API in druntime and Phobos 
 is pretty low. As I understand it, Windows users are pretty 
 much forced to use 
 http://www.dsource.org/projects/bindings/browser/trunk/win32 if 
 they need comprehensive Win32 API bindings. druntime seems to 
 be trying to define all of the OS-specific stuff like that, but 
 on top of it missing much of it, in the case of the Win32 API, 
 that's a _lot_ of functions, and I don't know if we want to put 
 that much in druntime. So, the question is, how do we want to 
 support the Win32 API in druntime and Phobos?


It seems to me that calling OS functions are expected to work out 
of the box. This would preclude requiring a third party library.

I don't think it makes sense in Druntime. It isn't required for 
the compiler to work that I know of. Maybe their are some Windows 
functions used within druntime? Also what is there currently 
isn't generating documentation.

Phobos already has std.windows, it just needs expanded.
Jan 25 2012
next sibling parent reply "Yao Gómez" <yaoltzin gmail.com> writes:
On Wednesday, 25 January 2012 at 19:58:40 UTC, Jesse Phillips 
wrote:

[snip]

 Phobos already has std.windows, it just needs expanded.


The issue is that this would be a huge undertaking, as the 
std.windows (or core.sys.windows, to be more precise) is awfully 
incomplete. Maybe we could add the Win32 bindings into the Deimos 
project, and make it more prominent somehow.

--
Yao Gómez
Jan 25 2012
parent "Jesse Phillips" <jessekphillips+D gmail.com> writes:
On Wednesday, 25 January 2012 at 21:36:36 UTC, Yao Gómez wrote:

 On Wednesday, 25 January 2012 at 19:58:40 UTC, Jesse Phillips 
 wrote:

 [snip]

 Phobos already has std.windows, it just needs expanded.


 The issue is that this would be a huge undertaking, as the 
 std.windows (or core.sys.windows, to be more precise) is 
 awfully incomplete. Maybe we could add the Win32 bindings into 
 the Deimos project, and make it more prominent somehow.

 --
 Yao Gómez


I don't see how that is the problem, this post is about what to 
do with the Win32 bindings project which is quite a bit more 
complete and created for the intent of inclusion of Phobos.
Jan 25 2012
prev sibling next sibling parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Wednesday, 25 January 2012 at 19:58:40 UTC, Jesse Phillips 
wrote:

 I don't think it makes sense in Druntime. It isn't required for 
 the compiler to work that I know of.


Underlying implementation of language features trickles down to 
using either the DMC runtime or Windows APIs. Druntime defines 
the Windows APIs locally where needed, which is a hack.

Example:

https://github.com/D-Programming-Language/druntime/blob/master/src/rt/dmain2.d#L31


 Also what is there currently isn't generating documentation.


This is not true: http://dlang.org/phobos/core_thread.html
Jan 25 2012
next sibling parent reply "Jesse Phillips" <jessekphillips+D gmail.com> writes:
On Wednesday, 25 January 2012 at 21:44:21 UTC, Vladimir Panteleev 
wrote:

 Also what is there currently isn't generating documentation.


 This is not true: http://dlang.org/phobos/core_thread.html


Yes it is, there aren't doc comments on anything:

https://github.com/D-Programming-Language/druntime/blob/master/src/core/sys/windows/windows.d
Jan 25 2012
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, January 26, 2012 00:20:52 Jesse Phillips wrote:

 On Wednesday, 25 January 2012 at 21:44:21 UTC, Vladimir Panteleev
 
 wrote:

 Also what is there currently isn't generating documentation.


 
 This is not true: http://dlang.org/phobos/core_thread.html


 
 Yes it is, there aren't doc comments on anything:
 
 https://github.com/D-Programming-Language/druntime/blob/master/src/core/sys/
 windows/windows.d


Pretty much none of the C bindings in druntime generate documentation - I 
believe primarily because Sean doesn't want to redo C documentation. However, 
that doesn't preclude giving them empty documentation so that you can see 
what's there, and that hasn't been done.

- Jonathan M Davis
Jan 25 2012
parent Jacob Carlborg <doob me.com> writes:
On 2012-01-26 00:44, Jonathan M Davis wrote:

 On Thursday, January 26, 2012 00:20:52 Jesse Phillips wrote:

 On Wednesday, 25 January 2012 at 21:44:21 UTC, Vladimir Panteleev

 wrote:

 Also what is there currently isn't generating documentation.


 This is not true: http://dlang.org/phobos/core_thread.html


 Yes it is, there aren't doc comments on anything:

 https://github.com/D-Programming-Language/druntime/blob/master/src/core/sys/
 windows/windows.d


 Pretty much none of the C bindings in druntime generate documentation - I
 believe primarily because Sean doesn't want to redo C documentation. However,
 that doesn't preclude giving them empty documentation so that you can see
 what's there, and that hasn't been done.

 - Jonathan M Davis


That would be good to have. A link to the C documentation would hurt either.

-- 
/Jacob Carlborg
Jan 25 2012
prev sibling parent Sean Kelly <sean invisibleduck.org> writes:
On Jan 25, 2012, at 1:44 PM, Vladimir Panteleev wrote:


 On Wednesday, 25 January 2012 at 19:58:40 UTC, Jesse Phillips wrote:

 I don't think it makes sense in Druntime. It isn't required for the =




compiler to work that I know of.

=20
 Underlying implementation of language features trickles down to using =


either the DMC runtime or Windows APIs. Druntime defines the Windows =
APIs locally where needed, which is a hack.

=20
 Example:
=20
 =


https://github.com/D-Programming-Language/druntime/blob/master/src/rt/dmai=
n2.d#L31

This was done to reduce coupling between modules and to reduce the =
runtime footprint in executables.  It's not terribly =
maintenance-friendly however, and so Druntime is slowly transition to =
simply importing the appropriate module.  A number of these remain =
mostly because it ain't broke as-is.
Jan 27 2012
prev sibling next sibling parent Sean Kelly <sean invisibleduck.org> writes:
On Jan 25, 2012, at 11:58 AM, Jesse Phillips wrote:


 On Wednesday, 25 January 2012 at 19:16:34 UTC, Jonathan M Davis wrote:

 The level of support for the Windows API in druntime and Phobos is =




pretty low. As I understand it, Windows users are pretty much forced to =
use http://www.dsource.org/projects/bindings/browser/trunk/win32 if they =
need comprehensive Win32 API bindings. druntime seems to be trying to =
define all of the OS-specific stuff like that, but on top of it missing =
much of it, in the case of the Win32 API, that's a _lot_ of functions, =
and I don't know if we want to put that much in druntime. So, the =
question is, how do we want to support the Win32 API in druntime and =
Phobos?

=20
 It seems to me that calling OS functions are expected to work out of =


the box. This would preclude requiring a third party library.

=20
 I don't think it makes sense in Druntime. It isn't required for the =


compiler to work that I know of. Maybe their are some Windows functions =
used within druntime? Also what is there currently isn't generating =
documentation.

System headers are absolutely required for Druntime to build.  This is =
one of the reasons core.stdc and core.sys exist.  Documentation isn't =
generated because these are interfaces to external APIs, so the =
documentation for those APIs should be referenced when coding to them.  =
We could add an empty doc comment to every declaration so something was =
generated in the Druntime docs, but I don't know why anyone would ever =
actually refer to them.=
Jan 27 2012
prev sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, January 27, 2012 10:39:41 Sean Kelly wrote:

 System headers are absolutely required for Druntime to build. This is one
 of the reasons core.stdc and core.sys exist. Documentation isn't generated
 because these are interfaces to external APIs, so the documentation for
 those APIs should be referenced when coding to them. We could add an empty
 doc comment to every declaration so something was generated in the Druntime
 docs, but I don't know why anyone would ever actually refer to them.


The value in having empty ddoc comments is so that if you need to use any of 
the functions in them for some reason (which _does_ happen upon occasion), 
then you know what module has what without having to go find the source code 
and look through it. So, ideally, all of the declarations in druntime for the 
C APIs should at least have empty ddoc so that they're documented. I 
completely agree that it's best not to try and duplicate the full 
documentation for them - that's best left to the C docs - but documenting 
what's there is still valuable, and we should probably do that at some point.

- Jonathan M Davis
Jan 27 2012