www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - The new core.sys.windows

reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
Hi all,

Barring last minute objections, the new core.sys.windows package 
is ready for merging:

https://github.com/D-Programming-Language/druntime/pull/1402

Essentially this is the win32 package from the DSource bindings 
project with the necessary adjustments for Druntime inclusion. 
See the PR description for details.

There are two remaining open questions, which although don't 
block the merge, warrant wider discussion:

1. License. Most headers in the win32 project state that they are 
in the "public domain". As such, we could relicense them under 
the Boost software license (the license used for most D 
components). This could also be done later, but cannot be 
reversed. I understand that releasing something as public domain 
is also not legally recognized in many jurisdictions.

2. Successive edits in layout, style, strictness and D-ification. 
Once the PR is merged, we will have a window until the next 
master-borne release during which we can make breaking changes. 
After that, no breaking changes should be made.

For example: usage of named enums. In C, all enums (or #defines) 
all live within the name namespace, with a prefix serving as the 
disambiguating "namespace", and all are implicitly convertible to 
each other. In many places, the bindings use named enums instead, 
which provides the advantage of strong typing - if a function 
accepts a parameter of a certain enum, you cannot accidentally 
pass a member of another enum.

So, if you have any ideas along these lines, speak up now.
Oct 14 2015
next sibling parent reply Mike Parker <aldacron gmail.com> writes:
On Wednesday, 14 October 2015 at 21:52:05 UTC, Vladimir Panteleev 
wrote:


 For example: usage of named enums. In C, all enums (or 
 #defines) all live within the name namespace, with a prefix 
 serving as the disambiguating "namespace", and all are 
 implicitly convertible to each other. In many places, the 
 bindings use named enums instead, which provides the advantage 
 of strong typing - if a function accepts a parameter of a 
 certain enum, you cannot accidentally pass a member of another 
 enum.
I think named enums are a bad idea. For one thing, it's inconsistent with the other system modules. For another, it's a pain point for porting existing C code to D. If they are kept, then at the very least aliases ought to be provided.
Oct 14 2015
next sibling parent reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Thursday, 15 October 2015 at 01:51:50 UTC, Mike Parker wrote:
 On Wednesday, 14 October 2015 at 21:52:05 UTC, Vladimir 
 Panteleev wrote:


 For example: usage of named enums. In C, all enums (or 
 #defines) all live within the name namespace, with a prefix 
 serving as the disambiguating "namespace", and all are 
 implicitly convertible to each other. In many places, the 
 bindings use named enums instead, which provides the advantage 
 of strong typing - if a function accepts a parameter of a 
 certain enum, you cannot accidentally pass a member of another 
 enum.
I think named enums are a bad idea. For one thing, it's inconsistent with the other system modules. For another, it's a pain point for porting existing C code to D. If they are kept, then at the very least aliases ought to be provided.
Well, aliases would allow both C source compatibility and strong typing. Plus, it won't be a breaking change, so they can be added whenever. Now that we don't need to support D1, this can be done with a mixin OSLT.
Oct 14 2015
parent reply Suliman <evermind live.ru> writes:
Vladimir, what benefits its give?
Oct 14 2015
next sibling parent reply Suliman <evermind live.ru> writes:
On Thursday, 15 October 2015 at 06:23:41 UTC, Suliman wrote:
 Vladimir, what benefits its give?
I mean adding this lib to Phobos. I mean will it make creation of D native toolkit more easier?
Oct 14 2015
parent Rikki Cattermole <alphaglosined gmail.com> writes:
On 15/10/15 7:24 PM, Suliman wrote:
 On Thursday, 15 October 2015 at 06:23:41 UTC, Suliman wrote:
 Vladimir, what benefits its give?
I mean adding this lib to Phobos. I mean will it make creation of D native toolkit more easier?
Definately: https://github.com/rikkimax/alphaPhobos/blob/master/source/std/experimental/ui/window/internal.d#L54
Oct 14 2015
prev sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, 15 October 2015 at 06:23:41 UTC, Suliman wrote:
 Vladimir, what benefits its give?
Ideally, druntime is supposed to have C bindings for all of the system libraries for each of the platforms that we support. What it has for the POSIX systems is pretty good, but the bindings for Windows have been seriously lacking. This is supposed to fix that. That way, when you need to use system calls, you don't need to figure out where you can find bindings for them or have to do them yourself. That will continue to be the case for 3rd party C libraries, but for the system stuff, we want their bindings in druntime. - Jonathan M Davis
Oct 15 2015
parent reply Jacob Carlborg <doob me.com> writes:
On 2015-10-15 09:32, Jonathan M Davis wrote:

 What it has for the  POSIX systems is pretty good
Not if you count OS X and the non-POSIX API's. -- /Jacob Carlborg
Oct 15 2015
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, 15 October 2015 at 17:32:34 UTC, Jacob Carlborg 
wrote:
 On 2015-10-15 09:32, Jonathan M Davis wrote:

 What it has for the  POSIX systems is pretty good
Not if you count OS X and the non-POSIX API's.
It's not even complete for Linux, but it's _way_ more complete than the Windows headers have been. LOL. However, it looks like Windows will probably have the best bindings now. - Jonathan M Davis
Oct 15 2015
parent reply Jacob Carlborg <doob me.com> writes:
On 2015-10-15 20:39, Jonathan M Davis wrote:

 It's not even complete for Linux, but it's _way_ more complete than the
 Windows headers have been. LOL. However, it looks like Windows will
 probably have the best bindings now.
The trouble with Linux is that there are so many different distributions, especially if you include the GUI related libraries. It's a lot easier to include bindings for more of the system libraries on OS X because there's only one OS X. -- /Jacob Carlborg
Oct 16 2015
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, 16 October 2015 at 18:42:26 UTC, Jacob Carlborg wrote:
 On 2015-10-15 20:39, Jonathan M Davis wrote:

 It's not even complete for Linux, but it's _way_ more complete 
 than the
 Windows headers have been. LOL. However, it looks like Windows 
 will
 probably have the best bindings now.
The trouble with Linux is that there are so many different distributions, especially if you include the GUI related libraries. It's a lot easier to include bindings for more of the system libraries on OS X because there's only one OS X.
True, OS X will be more uniform than Linux, because it doesn't have multiple distros - just multiple releases. However, the system APIs in Linux are not specific to any distro. They're specific to the libc that's being used - which in almost all cases is glibc, meaning that the system APIs are the same across almost all distros. So, in general, I really wouldn't expect it to be any more difficult to include bindings for Linux than for OS X. However, regardless of how easy it is to add all of the missing bindings, it still requires that someone be motivated enough to do it and have the time and necessary knowledge and skill to do so. And for the most part, no one seems to want to spend their time finishing the C bindings in druntime. Rather, they tend to get added piecemeal as someone needs another one. It's actually kind of amazing that Vladimir put in the time to do this, since even just porting oven the Win32 bindings from the project that they've been in is no small undertaking. - Jonathan M Davis
Oct 16 2015
prev sibling parent ponce <contact gam3sfrommars.fr> writes:
On Thursday, 15 October 2015 at 01:51:50 UTC, Mike Parker wrote:
 I think named enums are a bad idea. For one thing, it's 
 inconsistent with the other system modules. For another, it's a 
 pain point for porting existing C code to D. If they are kept, 
 then at the very least aliases ought to be provided.
+1 there is a lot of example code and documentation out there using the original names.
Oct 15 2015
prev sibling next sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 15-Oct-2015 00:52, Vladimir Panteleev wrote:
 Hi all,

 Barring last minute objections, the new core.sys.windows package is
 ready for merging:

 https://github.com/D-Programming-Language/druntime/pull/1402

 Essentially this is the win32 package from the DSource bindings project
 with the necessary adjustments for Druntime inclusion. See the PR
 description for details.

 There are two remaining open questions, which although don't block the
 merge, warrant wider discussion:

 1. License. Most headers in the win32 project state that they are in the
 "public domain". As such, we could relicense them under the Boost
 software license (the license used for most D components). This could
 also be done later, but cannot be reversed. I understand that releasing
 something as public domain is also not legally recognized in many
 jurisdictions.
I'd opt to Boost all the way down. I'm not really a lawyer nor do I have a problem with public domain.
 2. Successive edits in layout, style, strictness and D-ification. Once
 the PR is merged, we will have a window until the next master-borne
 release during which we can make breaking changes. After that, no
 breaking changes should be made.
Makes sense.
 For example: usage of named enums. In C, all enums (or #defines) all
 live within the name namespace, with a prefix serving as the
 disambiguating "namespace", and all are implicitly convertible to each
 other. In many places, the bindings use named enums instead, which
 provides the advantage of strong typing - if a function accepts a
 parameter of a certain enum, you cannot accidentally pass a member of
 another enum.
Being a windows developer while I was young (and foolish) I would use wrong constants in the wrong parameter all the time. Even with some time passing I had to double check every argument manually. So yes, please - strong enums are hugely beneficial. Interestingly ORing together enums will yield the same enum which is a good thing in this case.
 So, if you have any ideas along these lines, speak up now.
-- Dmitry Olshansky
Oct 15 2015
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/15/2015 1:18 AM, Dmitry Olshansky wrote:
 On 15-Oct-2015 00:52, Vladimir Panteleev wrote:
 1. License. Most headers in the win32 project state that they are in the
 "public domain". As such, we could relicense them under the Boost
 software license (the license used for most D components). This could
 also be done later, but cannot be reversed. I understand that releasing
 something as public domain is also not legally recognized in many
 jurisdictions.
I'd opt to Boost all the way down. I'm not really a lawyer nor do I have a problem with public domain.
Since Windows headers are just lists of declarations, I don't even think they are copyrightable. To the extent that they are, we should use Boost all the way. And yes, the reason we have been using Boost at all is because public domain is not recognized in some jurisdictions. I would have preferred to make it all public domain. Boost is the best license we could find that was both widely recognized and used, and was closest to public domain in effect.
Oct 15 2015
next sibling parent reply Kagamin <spam here.lot> writes:
On Thursday, 15 October 2015 at 08:25:21 UTC, Walter Bright wrote:
 Since Windows headers are just lists of declarations, I don't 
 even think they are copyrightable.
Wasn't Oracle v Google case about API copyrightability? As I understand, Supreme Court and Solicitor General ruled agains Google.
Oct 15 2015
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 10/15/15 7:37 AM, Kagamin wrote:
 On Thursday, 15 October 2015 at 08:25:21 UTC, Walter Bright wrote:
 Since Windows headers are just lists of declarations, I don't even
 think they are copyrightable.
Wasn't Oracle v Google case about API copyrightability? As I understand, Supreme Court and Solicitor General ruled agains Google.
I think Walter misstated what he meant. Of course any document is copyrightable. The problem is, however, that one can write such a document knowing what the names of the functions are, and this is not an infringement of the original copyright. The Oracle decision was based on the fact that the API file was identical or derived in the "structure, sequence and organization". That is, I'm assuming the API file was listed similarly to the Oracle version, and possibly had the same documentation. According to wikipedia, the supreme court declined to review the case, but it isn't over yet (I think the "reversal" was to remand the case back to the lower court to reconsider). You can't copyright a *single* declaration. So if I tell you there's a function called CreateFile, and it takes these parameters, you can write a declaration in a d file, and it's not an infringement of the original header file. The word "CreateFile" can't be copyrighted by itself. So I think what Walter meant was that it's not *worth* claiming copyright on an API that is just declarations, because one can just recreate the file without having to copy the original. But if the file is clearly marked public domain, all of this is moot. -Steve
Oct 15 2015
parent reply Kagamin <spam here.lot> writes:
On Thursday, 15 October 2015 at 15:00:26 UTC, Steven 
Schveighoffer wrote:
 You can't copyright a *single* declaration. So if I tell you 
 there's a function called CreateFile, and it takes these 
 parameters, you can write a declaration in a d file, and it's 
 not an infringement of the original header file. The word 
 "CreateFile" can't be copyrighted by itself.
I'm afraid, retelling is still a copy :) But as I read more into the Brief, looks like Google tried to foreclose copyright for declarations for being method of operation, which fails (and I agree with that). Then Solicitor General recommends fair use defense to address feared effects on software development (and almost surprised notes that Google didn't go that way for some reason).
Oct 15 2015
next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, 15 October 2015 at 15:55:09 UTC, Kagamin wrote:
 On Thursday, 15 October 2015 at 15:00:26 UTC, Steven 
 Schveighoffer wrote:
 You can't copyright a *single* declaration. So if I tell you 
 there's a function called CreateFile, and it takes these 
 parameters, you can write a declaration in a d file, and it's 
 not an infringement of the original header file. The word 
 "CreateFile" can't be copyrighted by itself.
I'm afraid, retelling is still a copy :) But as I read more into the Brief, looks like Google tried to foreclose copyright for declarations for being method of operation, which fails (and I agree with that). Then Solicitor General recommends fair use defense to address feared effects on software development (and almost surprised notes that Google didn't go that way for some reason).
IIRC, if they had won the way that they were trying to, it would have been farther reaching - legally speaking. With fair use, you have to argue each instance of it, whereas with a broader ruling that's not tied to fair use, they might have gotten a ruling that would apply to more than just this case - especially if the Supreme Court ruled on it. But IANAL. As I understand it though, there is definitely fear in the software community at large that the ruling on this case could have a devastating effect on APIs. Pretty much all of the software folks agree that copyrighting them makes no sense. It would completely destroy interoperability. But what the actual legal situation will end up being is another question entirely. In this particular case though, since the headers are in the public domain, it really shouldn't matter what happens with the Oracle case. And while MS has done plenty of stupid and/or evil stuff over the years, I don't think that they're anywhere near stupid enough to go after anyone for creating bindings to their APIs and actually trying to use them. They _want_ you to use them, because they want you to use their products and to write software for their products. So, I really don't think that we have anything to fear here. - Jonathan M Davis
Oct 15 2015
parent reply Brad Roberts via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 10/15/2015 9:07 AM, Jonathan M Davis via Digitalmars-d wrote:

 In this particular case though, since the headers are in the public
 domain, it really shouldn't matter what happens with the Oracle case.
 And while MS has done plenty of stupid and/or evil stuff over the years,
 I don't think that they're anywhere near stupid enough to go after
 anyone for creating bindings to their APIs and actually trying to use
 them. They _want_ you to use them, because they want you to use their
 products and to write software for their products. So, I really don't
 think that we have anything to fear here.

 - Jonathan M Davis
I'd really like to see a trace of the history of those headers to make sure they're _really_ in the public domain. I have doubts that microsoft ever declared that in headers they released and that someone else just copied (or transcribed) the apis and declared them to be in the public domain.
Oct 15 2015
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, 16 October 2015 at 01:11:01 UTC, Brad Roberts wrote:
 I'd really like to see a trace of the history of those headers 
 to make sure they're _really_ in the public domain.  I have 
 doubts that microsoft ever declared that in headers they 
 released and that someone else just copied (or transcribed) the 
 apis and declared them to be in the public domain.
Yeah, I've always wondered how mingw had a version of the Windows headers in the public domain (which is where our bindings are derived from IIRC0. It wouldn't surprise me in the least if they had been copied from MS' official headers at some point, in which case they wouldn't be legit, but at the same time, I'm not sure that I want to figure out that they're not legit, because we're kind of screwed if they're not. That being said, I really doubt that Microsoft cares all that much if all you're doing is trying to bind to their public API to use it, since they want you to use it. AFAIK, they've never even done anything against the WINE guys, and they're actually reimplementing the API in order to get away from MS (though I don't know where they got whatever headers they use). - Jonathan M Davis
Oct 15 2015
parent Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Friday, 16 October 2015 at 02:14:10 UTC, Jonathan M Davis 
wrote:
 It wouldn't surprise me in the least if they had been copied 
 from MS' official headers at some point, in which case they 
 wouldn't be legit, but at the same time, I'm not sure that I 
 want to figure out that they're not legit, because we're kind 
 of screwed if they're not.
More importantly, if somehow this does result in a problem, MinGW will be screwed even harder, and MinGW is a much bigger project than D - the great majority of open-source cross-platform applications are built using MinGW for Windows. The impact on D would essentially be collateral damage.
Oct 15 2015
prev sibling parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 10/15/15 11:55 AM, Kagamin wrote:
 On Thursday, 15 October 2015 at 15:00:26 UTC, Steven Schveighoffer wrote:
 You can't copyright a *single* declaration. So if I tell you there's a
 function called CreateFile, and it takes these parameters, you can
 write a declaration in a d file, and it's not an infringement of the
 original header file. The word "CreateFile" can't be copyrighted by
 itself.
I'm afraid, retelling is still a copy :)
Retelling of what? I cannot believe that anyone would ever successfully sue someone because they put an extern declaration in their implementation file.
 But as I read more into the Brief, looks like Google tried to foreclose
 copyright for declarations for being method of operation, which fails
 (and I agree with that). Then Solicitor General recommends fair use
 defense to address feared effects on software development (and almost
 surprised notes that Google didn't go that way for some reason).
I didn't read into it that far ;) -Steve
Oct 15 2015
prev sibling parent Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Thursday, 15 October 2015 at 08:25:21 UTC, Walter Bright wrote:
 On 10/15/2015 1:18 AM, Dmitry Olshansky wrote:
 I'd opt to Boost all the way down. I'm not really a lawyer nor 
 do I have
 a problem with public domain.
Since Windows headers are just lists of declarations, I don't even think they are copyrightable. To the extent that they are, we should use Boost all the way. And yes, the reason we have been using Boost at all is because public domain is not recognized in some jurisdictions. I would have preferred to make it all public domain. Boost is the best license we could find that was both widely recognized and used, and was closest to public domain in effect.
I updated the PR and changed the license to Boost. I think the PR is ready for merging.
Oct 15 2015
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/15/2015 1:18 AM, Dmitry Olshansky wrote:
 On 15-Oct-2015 00:52, Vladimir Panteleev wrote:
 For example: usage of named enums. In C, all enums (or #defines) all
 live within the name namespace, with a prefix serving as the
 disambiguating "namespace", and all are implicitly convertible to each
 other. In many places, the bindings use named enums instead, which
 provides the advantage of strong typing - if a function accepts a
 parameter of a certain enum, you cannot accidentally pass a member of
 another enum.
Being a windows developer while I was young (and foolish) I would use wrong constants in the wrong parameter all the time. Even with some time passing I had to double check every argument manually. So yes, please - strong enums are hugely beneficial.
I know how tempting it is to "fix" windows headers (or other operating system headers). We made a decision long ago that this is not on the agenda of the D project. We simply make the closest approximation of them possible, and go with that. Taking on the task of "fixing" headers like that makes us then responsible for documenting them, being responsible for them, etc., that is way, way beyond our resources. By sticking with them the way they are, we simply refer people to the OS vendor documentation on how they are to be correctly used, for better or worse. Also, it means that C code that deals with those APIs is directly transcribable to D without going through another translation process. I agree that they would be better if fixed, but the D project is not going to do that. If anyone wants to make a set of fixed windows imports, that's great, but it wouldn't be a part of the official D release.
Oct 15 2015
next sibling parent Meta <jared771 gmail.com> writes:
On Thursday, 15 October 2015 at 08:34:10 UTC, Walter Bright wrote:
 I agree that they would be better if fixed, but the D project 
 is not going to do that. If anyone wants to make a set of fixed 
 windows imports, that's great, but it wouldn't be a part of the 
 official D release.
Agreed; we don't want to start down this slippery slope.
Oct 15 2015
prev sibling parent reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Thursday, 15 October 2015 at 08:34:10 UTC, Walter Bright wrote:
 I know how tempting it is to "fix" windows headers (or other 
 operating system headers). We made a decision long ago that 
 this is not on the agenda of the D project. We simply make the 
 closest approximation of them possible, and go with that.

 Taking on the task of "fixing" headers like that makes us then 
 responsible for documenting them, being responsible for them, 
 etc., that is way, way beyond our resources. By sticking with 
 them the way they are, we simply refer people to the OS vendor 
 documentation on how they are to be correctly used, for better 
 or worse.

 Also, it means that C code that deals with those APIs is 
 directly transcribable to D without going through another 
 translation process.

 I agree that they would be better if fixed, but the D project 
 is not going to do that. If anyone wants to make a set of fixed 
 windows imports, that's great, but it wouldn't be a part of the 
 official D release.
All good points. I'd like to add some: 1. Some new versions of C/C++ Windows headers do define a "strict mode", which enables additional compile-time checks by using static typing, e.g. for all the various handle-like types. 2. The changes I had in mind would not change the way you would write code (and I agree C source compatibility is important and must be preserved). The "strict mode" mentioned above works in the same way. Compatibility with the old core.sys.windows.windows remains the highest priority. 3. This effort is likely going to prevent any third-party D Windows headers project become popular. Why bother if the headers in the language's runtime library are "good enough"? Anyway, since there are supporters of both sides, it seems like the best course of action here and now is to leave the declarations as they are. If there is desire for it, we can improve C-source compatibility or introduce opt-in (enabled by a version) strict mode checks down the line.
Oct 15 2015
parent Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Thursday, 15 October 2015 at 16:05:11 UTC, Vladimir Panteleev 
wrote:
 I'd like to add some:
One more point I forgot to mention: 4. We are currently in a great position for such an initiative, because we can start with a strict-but-potentially-broken state (as far as all new declarations that weren't in core.sys.windows.windows go), and then gradually loosen things up as necessary. We cannot go in the other direction, and we will almost surely never have this opportunity again.
Oct 15 2015
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2015-10-15 10:18, Dmitry Olshansky wrote:

 I'd opt to Boost all the way down. I'm not really a lawyer nor do I have
 a problem with public domain.
I think we have, or at least had quite a lot of files declared as public domain in druntime. -- /Jacob Carlborg
Oct 16 2015
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2015-10-14 23:52, Vladimir Panteleev wrote:

 For example: usage of named enums. In C, all enums (or #defines) all
 live within the name namespace, with a prefix serving as the
 disambiguating "namespace", and all are implicitly convertible to each
 other. In many places, the bindings use named enums instead, which
 provides the advantage of strong typing - if a function accepts a
 parameter of a certain enum, you cannot accidentally pass a member of
 another enum.
Apple exposes their C/Objective-C enums in Swift without the usual prefix, example: Objective-C: enum { NSOrderedAscending = -1, NSOrderedSame, NSOrderedDescending }; typedef NSInteger NSComparisonResult; Swift: enum NSComparisonResult : Int { case OrderedAscending case OrderedSame case OrderedDescending } -- /Jacob Carlborg
Oct 16 2015