www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - Getting back into translating the Win32 headers - anyone?

reply Stewart Gordon <smjg_1998 yahoo.com> writes:
Not long ago (read: three months ago), there was a thread here to the
effect that this effort to translate the Windows API headers had gone quiet.

http://www.digitalmars.com/d/archives/digitalmars/D/announce/4605.html
http://www.prowiki.org/wiki4d/wiki.cgi?WindowsAPI

However, activity since then has continued to be minimal.  Can anyone
hazard a guess at why?  I can't....

The good news is that I've recently got an Internet connection set up on
the PC I do my D programming on.  Despite being broadband through a
router, it's intermittent, but at least I've got it at all.  And I've
even got Subversion working!  So no longer will I have to zip up my
updates and upload them to my website.  I guess I ought to remove the
link to my zip edition from the wiki page....

I'm working on implementing the status scheme proposed by Don Clugston:

- Initial conversion, but with sections commented out -- fixme
- Initial conversion with basic functionality -- compiles
- Fully functional (including versioning, pragma(lib)) -- alpha
- Polished -- beta


There are a few open issues that I feel still need to be addressed:

1. Should having the types of all constants (where available) be a
prerequisite for counting as "fully functional", and therefore having
alpha status?  Or is this something for the beta stage?  I suppose it 
depends on whether you consider the difference between alpha and beta to 
be purely cosmetic issues.

2. Which should be the default - winsock or winsock2?  Don once wrote:

"I really think that Winsock2 should be the default, since it was
supported in Win95. It was definitely included in Win95 B and C. It's
possible that Winsock2 might not have been included in the original
Win95 release, but any such computers with networks connections would be
immediately wiped out by viruses if they've never been upgraded to
Winsock2. We can be confident that there are no Win95 PCs that want to
use Winsock1. Winsock1 is a 16-bit technology, I doubt that any D
programs will ever use it. -- Don"

What does everybody else think?  If nobody disagrees or gets there
first, I'll change Winsock2 to be the default.

3. When should imports be public, and when should they be private?

(a) If the whole point of a module is to import other modules, then
obviously the imports have to be public.
(b) If qwert.h doesn't include yuiop.h, but yuiop.d is required for
qwert.d to compile, then the import should be private.
(c) This leaves the general case where a C header includes another.

Should we make imports of kind (c) public or private?  Moreover, should
we declare (b) and (c) imports using "private import" or simply "import"?

4. There seem to be a few ways of doing the variable-length structs in use.

     struct MIB_IPADDRTABLE {
         DWORD            dwNumEntries;
         MIB_IPADDRROW[1] _table;

         MIB_IPADDRROW* table() { return _table.ptr; }
     }

or

     struct MIB_IPADDRTABLE {
         DWORD         dwNumEntries;
         MIB_IPADDRROW _table;

         MIB_IPADDRROW* table() { return &_table; }
     }

or (merely proposed, I don't think used yet)

     struct MIB_IPADDRTABLE {
         DWORD         dwNumEntries;
         MIB_IPADDRROW _table;

         MIB_IPADDRROW[] table() { return (&_table)[0..dwNumEntries]; }
     }

Which should we settle with?  I personally think the proposed idea is a
good one.  Just as the C headers make use of C's lack of ABC, so the D
translation would make use of D's ABC.  Look also at the idea for custom
allocators for these things on the discussion page of the wiki.

5. I've noticed that bit field setters have variously, depending on who
coded them up, been declared to return void or to return the value
passed in.  Is there any reason to have them returning void?  We ought
to make these consistent.

6. In many modules, class IDs defined externally in a library are being
declared without the const attribute.  Is there some reason for this?
They're constants, so surely they should be declared as such.


We've less than 30 days to get it done and dusted if we're going to
submit a full translation for DMD 1.0.  Even better would be if it can
be got done by the time I go back to my immediate family for Christmas
around the 20th.  So come on guys!

Stewart.
Dec 02 2006
next sibling parent reply Sean Kelly <sean f4.ca> writes:
Here's an implementation of process.h.  I did it manually from the MSDN 
docs a long time ago.
Dec 02 2006
parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
Sean Kelly wrote:

 Here's an implementation of process.h.  I did it manually from the MSDN 
 docs a long time ago.


process.h is part of the C/C++ RTL, not the Win32 API.  So it's beyond
the scope of this project.

Please look at the Wiki4D page if you haven't already.  It contains a
list of the Win32 header files, and is the means for you to assign
modules to yourself before translating them in order to avoid
duplication of effort.  And get yourself a copy of the MinGW headers, as
this is what the rest of us are using.  The MinGW site linked to from
the wiki seems to be down at the mo, but there's still

http://www.mingw.org/download.shtml#hdr2
http://prdownloads.sourceforge.net/mingw/w32api-3.6-src.tar.gz?download

Stewart.
Dec 02 2006
parent Sean Kelly <sean f4.ca> writes:
Stewart Gordon wrote:

 Sean Kelly wrote:

 Here's an implementation of process.h.  I did it manually from the 
 MSDN docs a long time ago.


 
 process.h is part of the C/C++ RTL, not the Win32 API.  So it's beyond
 the scope of this project.


Oops :-)  I guess it shows that when programming for Windows, the C/C++ 
RTL is practically the only thing I care about.


Sean
Dec 03 2006
prev sibling next sibling parent Stewart Gordon <smjg_1998 yahoo.com> writes:
Stewart Gordon wrote:
<snip>

 There are a few open issues that I feel still need to be addressed:


<snip>

I'd thought of a few more open issues, but I can't seem to remember what 
half of them are!

7. Should we keep the pragma(msg) instances, or rewrite them to use 
static assert with a custom message?  The only problem with static 
assert messages is that DMD doesn't present them in the best way.

8. In some instances, templates have been used to reimplement 
preprocessor macros from the C headers that are used to define 
constants.  In others, the constants defined by these macros have been 
defined directly by literals.  Which should we stick to?

Stewart.
Dec 02 2006
prev sibling next sibling parent reply "Chris Miller" <chris dprogramming.com> writes:
I was told that bcd worked great at converting MinGW's windows.h to D. I'm  
sure it's not as nice as what you're doing, but it could at least help  
with some of the tedious stuff. bcd is at  
http://www.dsource.org/projects/bcd/
Dec 02 2006
parent Stewart Gordon <smjg_1998 yahoo.com> writes:
Chris Miller wrote:

 I was told that bcd worked great at converting MinGW's windows.h to D. 
 I'm sure it's not as nice as what you're doing, but it could at least 
 help with some of the tedious stuff. bcd is at 
 http://www.dsource.org/projects/bcd/


windows.h has been done.  It's numerous other Windows API headers in 
MinGW that still need to be translated.  See the wiki page.

I'll have a look at bcd and see how well it works.  Using a tool is 
certainly a possible starting point for translating headers.  But the 
output from these invariably needs some cleaning up.

Stewart.
Dec 03 2006
prev sibling next sibling parent reply "Chris Miller" <chris dprogramming.com> writes:
On Sat, 02 Dec 2006 09:54:14 -0500, Stewart Gordon <smjg_1998 yahoo.com>  
wrote:

 2. Which should be the default - winsock or winsock2?  Don once wrote:

 "I really think that Winsock2 should be the default, since it was
 supported in Win95. It was definitely included in Win95 B and C. It's
 possible that Winsock2 might not have been included in the original
 Win95 release, but any such computers with networks connections would be
 immediately wiped out by viruses if they've never been upgraded to
 Winsock2. We can be confident that there are no Win95 PCs that want to
 use Winsock1. Winsock1 is a 16-bit technology, I doubt that any D
 programs will ever use it. -- Don"

 What does everybody else think?  If nobody disagrees or gets there
 first, I'll change Winsock2 to be the default.


I'd go with winsock2.


 3. When should imports be public, and when should they be private?

 (a) If the whole point of a module is to import other modules, then
 obviously the imports have to be public.
 (b) If qwert.h doesn't include yuiop.h, but yuiop.d is required for
 qwert.d to compile, then the import should be private.
 (c) This leaves the general case where a C header includes another.

 Should we make imports of kind (c) public or private?  Moreover, should
 we declare (b) and (c) imports using "private import" or simply "import"?


I guess the main windows.d would public import everything else, and most  
everything else might be private import, unless such files were intended  
to pull other things. e.g. people include windows.h expecting the bulk of  
windows stuff to be included as well, but they generally don't include  
winbase.h and expect winerror.h to be tagged along, even if it is.


 4. There seem to be a few ways of doing the variable-length structs in  
 use.

      struct MIB_IPADDRTABLE {
          DWORD            dwNumEntries;
          MIB_IPADDRROW[1] _table;

          MIB_IPADDRROW* table() { return _table.ptr; }
      }

 or

      struct MIB_IPADDRTABLE {
          DWORD         dwNumEntries;
          MIB_IPADDRROW _table;

          MIB_IPADDRROW* table() { return &_table; }
      }

 or (merely proposed, I don't think used yet)

      struct MIB_IPADDRTABLE {
          DWORD         dwNumEntries;
          MIB_IPADDRROW _table;

          MIB_IPADDRROW[] table() { return (&_table)[0..dwNumEntries]; }
      }

 Which should we settle with?  I personally think the proposed idea is a
 good one.  Just as the C headers make use of C's lack of ABC, so the D
 translation would make use of D's ABC.  Look also at the idea for custom
 allocators for these things on the discussion page of the wiki.


I think I prefer the first one, as it most resembles what MS did. The last  
one is probably too D-like for a C interface, especially considering the C  
version of table would decay into a pointer, and yours would be a D slice.


 5. I've noticed that bit field setters have variously, depending on who
 coded them up, been declared to return void or to return the value
 passed in.  Is there any reason to have them returning void?  We ought
 to make these consistent.


Unsure. Perhaps D's properties should automatically call the getter if the  
setter is used as an rvalue and it returns void. Also consider foo.prop++;  
it should probably call the getter, apply ++ and call the setter with the  
new value. This automatic calling of getter/setter as appropriate seems  
more simplistic and pure, and should probably be addressed before 1.0 so  
that properties aren't crippled in the spec.


 6. In many modules, class IDs defined externally in a library are being
 declared without the const attribute.  Is there some reason for this?
 They're constants, so surely they should be declared as such.


Is const stuff even added to object files? This might be why..
Dec 02 2006
parent Stewart Gordon <smjg_1998 yahoo.com> writes:
Chris Miller wrote:

 On Sat, 02 Dec 2006 09:54:14 -0500, Stewart Gordon <smjg_1998 yahoo.com> 
 wrote:


<snip>

 4. There seem to be a few ways of doing the variable-length structs 
 in use.




<snip>

          MIB_IPADDRROW[1] _table;

          MIB_IPADDRROW* table() { return _table.ptr; }




<snip>

          MIB_IPADDRROW _table;

          MIB_IPADDRROW* table() { return &_table; }




<snip>

          MIB_IPADDRROW _table;

          MIB_IPADDRROW[] table() { return (&_table)[0..dwNumEntries]; }
      }

 Which should we settle with? I personally think the proposed idea
 is a good one. Just as the C headers make use of C's lack of ABC,
 so the D translation would make use of D's ABC. Look also at the
 idea for custom allocators for these things on the discussion page
 of the wiki.


 
 I think I prefer the first one, as it most resembles what MS did. The 
 last one is probably too D-like for a C interface, especially 
 considering the C version of table would decay into a pointer, and 
 yours would be a D slice.


I'm not sure what you mean by this.  We already make a few little bits 
make more sense for D, such as translating COM interfaces.  In C, table 
is a one-element array, which is neither a pointer nor a D slice.  D 
array slices decay into pointers too; even if they didn't, the 
difference between writing table and writing table.ptr is trivial IMO.


 5. I've noticed that bit field setters have variously, depending on who
 coded them up, been declared to return void or to return the value
 passed in.  Is there any reason to have them returning void?  We ought
 to make these consistent.


 
 Unsure. Perhaps D's properties should automatically call the getter if 
 the setter is used as an rvalue and it returns void.


I've proposed this before.

http://www.digitalmars.com/pnews/read.php?server=news.digitalmars.com&group=digitalmars.D&artnum=10199

I can't believe it's taken so long for anybody to second this idea!


 Also consider foo.prop++; it should probably call the getter, apply 
 ++ and call the setter with the new value. This automatic calling of 
 getter/setter as appropriate seems more simplistic and pure, and 
 should probably be addressed before 1.0 so that properties aren't 
 crippled in the spec.


I agree.


 6. In many modules, class IDs defined externally in a library are being
 declared without the const attribute.  Is there some reason for this?
 They're constants, so surely they should be declared as such.


 
 Is const stuff even added to object files? This might be why..


We don't need to add them to any object files.  They're already in the 
.lib files, where they need to be.

Stewart.
Dec 03 2006
prev sibling next sibling parent reply Don Clugston <dac nospam.com.au> writes:
Stewart Gordon wrote:

 Not long ago (read: three months ago), there was a thread here to the
 effect that this effort to translate the Windows API headers had gone 
 quiet.
 
 http://www.digitalmars.com/d/archives/digitalmars/D/announce/4605.html
 http://www.prowiki.org/wiki4d/wiki.cgi?WindowsAPI
 
 However, activity since then has continued to be minimal.  Can anyone
 hazard a guess at why?  I can't....
 
 The good news is that I've recently got an Internet connection set up on
 the PC I do my D programming on.  Despite being broadband through a
 router, it's intermittent, but at least I've got it at all.  And I've
 even got Subversion working!  So no longer will I have to zip up my
 updates and upload them to my website.  I guess I ought to remove the
 link to my zip edition from the wiki page....
 
 I'm working on implementing the status scheme proposed by Don Clugston:
 
 - Initial conversion, but with sections commented out -- fixme
 - Initial conversion with basic functionality -- compiles
 - Fully functional (including versioning, pragma(lib)) -- alpha
 - Polished -- beta
 
 
 There are a few open issues that I feel still need to be addressed:
 
 1. Should having the types of all constants (where available) be a
 prerequisite for counting as "fully functional", and therefore having
 alpha status?  Or is this something for the beta stage?  I suppose it 
 depends on whether you consider the difference between alpha and beta to 
 be purely cosmetic issues.
 
 2. Which should be the default - winsock or winsock2?  Don once wrote:
 
 "I really think that Winsock2 should be the default, since it was
 supported in Win95. It was definitely included in Win95 B and C. It's
 possible that Winsock2 might not have been included in the original
 Win95 release, but any such computers with networks connections would be
 immediately wiped out by viruses if they've never been upgraded to
 Winsock2. We can be confident that there are no Win95 PCs that want to
 use Winsock1. Winsock1 is a 16-bit technology, I doubt that any D
 programs will ever use it. -- Don"
 
 What does everybody else think?  If nobody disagrees or gets there
 first, I'll change Winsock2 to be the default.
 
 3. When should imports be public, and when should they be private?
 
 (a) If the whole point of a module is to import other modules, then
 obviously the imports have to be public.
 (b) If qwert.h doesn't include yuiop.h, but yuiop.d is required for
 qwert.d to compile, then the import should be private.
 (c) This leaves the general case where a C header includes another.
 
 Should we make imports of kind (c) public or private?  Moreover, should
 we declare (b) and (c) imports using "private import" or simply "import"?


I think (c) should be private, except in a few rare cases where they are 
  'top-level' modules that do nothing except publically import other 
modules.


 
 4. There seem to be a few ways of doing the variable-length structs in use.
 
     struct MIB_IPADDRTABLE {
         DWORD            dwNumEntries;
         MIB_IPADDRROW[1] _table;
 
         MIB_IPADDRROW* table() { return _table.ptr; }
     }
 
 or
 
     struct MIB_IPADDRTABLE {
         DWORD         dwNumEntries;
         MIB_IPADDRROW _table;
 
         MIB_IPADDRROW* table() { return &_table; }
     }
 
 or (merely proposed, I don't think used yet)
 
     struct MIB_IPADDRTABLE {
         DWORD         dwNumEntries;
         MIB_IPADDRROW _table;
 
         MIB_IPADDRROW[] table() { return (&_table)[0..dwNumEntries]; }
     }
 
 Which should we settle with?  I personally think the proposed idea is a
 good one.  Just as the C headers make use of C's lack of ABC, so the D
 translation would make use of D's ABC.  Look also at the idea for custom
 allocators for these things on the discussion page of the wiki.
 
 5. I've noticed that bit field setters have variously, depending on who
 coded them up, been declared to return void or to return the value
 passed in.  Is there any reason to have them returning void?  We ought
 to make these consistent.


For the ones done by me (which is probably a fair fraction of the 
total), I didn't think about it at all. Returning the value passed in 
probably makes more sense.


 
 6. In many modules, class IDs defined externally in a library are being
 declared without the const attribute.  Is there some reason for this?
 They're constants, so surely they should be declared as such.
 


Agreed.

 
 We've less than 30 days to get it done and dusted if we're going to
 submit a full translation for DMD 1.0.  Even better would be if it can
 be got done by the time I go back to my immediate family for Christmas
 around the 20th.  So come on guys!
 
 Stewart.
Dec 03 2006
parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
Don Clugston wrote:
<snip>

 (a) If the whole point of a module is to import other modules, then
 obviously the imports have to be public.
 (b) If qwert.h doesn't include yuiop.h, but yuiop.d is required for
 qwert.d to compile, then the import should be private.
 (c) This leaves the general case where a C header includes another.

 Should we make imports of kind (c) public or private?  Moreover, should
 we declare (b) and (c) imports using "private import" or simply "import"?


 
 I think (c) should be private, except in a few rare cases where they are 
 'top-level' modules that do nothing except publically import other 
 modules.


By (c), I actually meant that which doesn't fit under (a) or (b).  I 
can't think of any instances in which a module does nothing but imports 
other modules but that isn't the whole point of the module.

<snip>

 We've less than 30 days to get it done and dusted if we're going to
 submit a full translation for DMD 1.0.  Even better would be if it can
 be got done by the time I go back to my immediate family for Christmas
 around the 20th.  So come on guys!




Good to know you're still here.  You used to be one of the most active 
contributors to this project.  Hope you really are back!

Stewart.
Dec 03 2006
parent reply Don Clugston <dac nospam.com.au> writes:
Stewart Gordon wrote:

 Don Clugston wrote:
 <snip>

 (a) If the whole point of a module is to import other modules, then
 obviously the imports have to be public.
 (b) If qwert.h doesn't include yuiop.h, but yuiop.d is required for
 qwert.d to compile, then the import should be private.
 (c) This leaves the general case where a C header includes another.

 Should we make imports of kind (c) public or private?  Moreover, should
 we declare (b) and (c) imports using "private import" or simply 
 "import"?


 I think (c) should be private, except in a few rare cases where they 
 are 'top-level' modules that do nothing except publically import other 
 modules.


 
 By (c), I actually meant that which doesn't fit under (a) or (b).  I 
 can't think of any instances in which a module does nothing but imports 
 other modules but that isn't the whole point of the module.


I think I created windows.core, which does that, but it's probably 
unique. I think we should try to eliminate all cases of (c), turning 
them into (a) or (b). What are the case (c)'s at the moment?


 <snip>

 We've less than 30 days to get it done and dusted if we're going to
 submit a full translation for DMD 1.0.  Even better would be if it can
 be got done by the time I go back to my immediate family for Christmas
 around the 20th.  So come on guys!




 
 Good to know you're still here.  You used to be one of the most active 
 contributors to this project.  Hope you really are back!


I doubt I'll contribute much more to the Windows API project anytime 
before D 1.0, I'm too tied up in another D project.

A couple of suggestions, though:
* vfw is an obselete, buggy technology, and the MinGW header is a 
disaster area. I think we should just drop it.
* Some of the other MinGW header, like zmouse, are so incomplete 
there're probably not worth including.
* We don't need windowsx, either, it's just for C programmers without OO.
Dec 04 2006
parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
Don Clugston wrote:

 Stewart Gordon wrote:

 Don Clugston wrote:
 <snip>

 (a) If the whole point of a module is to import other modules, then 
 obviously the imports have to be public.
 (b) If qwert.h doesn't include yuiop.h, but yuiop.d is required for 
 qwert.d to compile, then the import should be private.
 (c) This leaves the general case where a C header includes another.








<snip>

 I think I created windows.core, which does that, but it's probably 
 unique. I think we should try to eliminate all cases of (c), turning 
 them into (a) or (b).


The cases (a), (b) and (c) are from the POV of the C headers.  That was 
about what each case should become in the D code.  It would make no 
sense to change anything to (b), because in D you can't use a symbol 
from another module without importing it.


 What are the case (c)'s at the moment?


See for yourself.  Grep the MinGW headers for "#include", and then cross 
off those headers that have no content but #include directives.

<snip>

 I doubt I'll contribute much more to the Windows API project anytime 
 before D 1.0, I'm too tied up in another D project.


I know what you mean - I get tied up in different D projects from time 
to time too.  Thank you for telling us - though it would've been nice to 
have known sooner.


 A couple of suggestions, though:
 * vfw is an obselete, buggy technology, and the MinGW header is a 
 disaster area. I think we should just drop it.


I don't know enough about vfw to give any first-hand opinion, but on the 
basis of what you say, if nobody disagrees then maybe we should get rid 
of it indeed.


 * Some of the other MinGW header, like zmouse, are so incomplete 
 there're probably not worth including.


Indeed, zmouse does nothing but defines constants that are already 
defined elsewhere.


 * We don't need windowsx, either, it's just for C programmers without OO.


I'm not sure what I think to this.  It defines macros for manipulating 
the older Windows controls, just as commctrl does for the newer ones. 
But they aren't documented in my win32.hlp.

Stewart.
Dec 07 2006
parent Don Clugston <dac nospam.com.au> writes:
Stewart Gordon wrote:

 Don Clugston wrote:

 Stewart Gordon wrote:

 Don Clugston wrote:
 <snip>

 (a) If the whole point of a module is to import other modules, then 
 obviously the imports have to be public.
 (b) If qwert.h doesn't include yuiop.h, but yuiop.d is required for 
 qwert.d to compile, then the import should be private.
 (c) This leaves the general case where a C header includes another.








 <snip>

 I think I created windows.core, which does that, but it's probably 
 unique. I think we should try to eliminate all cases of (c), turning 
 them into (a) or (b).


 
 The cases (a), (b) and (c) are from the POV of the C headers.  That was 
 about what each case should become in the D code.  It would make no 
 sense to change anything to (b), because in D you can't use a symbol 
 from another module without importing it.


There are many cases where the SDK docs say "this is in xxx.h, but do 
not #include xxx.h directly. Instead, use yyy.h which includes xxx.h".
Sometimes it turns out that yyy is including zzz.h which is including xxx.h.
In this situation, I think that zzz should privately import xxx, and yyy 
should publicly import both xxx and zzz.
That is, the only instances of public imports should be with modules 
intended to be directly imported by the user.


 What are the case (c)'s at the moment?


 
 See for yourself.  Grep the MinGW headers for "#include", and then cross 
 off those headers that have no content but #include directives.
 
 <snip>

 I doubt I'll contribute much more to the Windows API project anytime 
 before D 1.0, I'm too tied up in another D project.


 
 I know what you mean - I get tied up in different D projects from time 
 to time too.  Thank you for telling us - though it would've been nice to 
 have known sooner.
 

 A couple of suggestions, though:
 * vfw is an obselete, buggy technology, and the MinGW header is a 
 disaster area. I think we should just drop it.


 
 I don't know enough about vfw to give any first-hand opinion, but on the 
 basis of what you say, if nobody disagrees then maybe we should get rid 
 of it indeed.
 

 * Some of the other MinGW header, like zmouse, are so incomplete 
 there're probably not worth including.


 
 Indeed, zmouse does nothing but defines constants that are already 
 defined elsewhere.
 

 * We don't need windowsx, either, it's just for C programmers without OO.


 
 I'm not sure what I think to this.  It defines macros for manipulating 
 the older Windows controls, just as commctrl does for the newer ones. 
 But they aren't documented in my win32.hlp.


It's ancient. It was around in the Windows 3.0 days, maybe even older. 
They just kept including it for backwards compatibility.
It's probably best to think of windowsx.h as 'framework code', like MFC, 
rather than part of the WinAPI itself.
Arguably we want a D equivalent of it, but that would be very different 
to a port.
Dec 07 2006
prev sibling parent Stewart Gordon <smjg_1998 yahoo.com> writes:
On the basis of consensus and the lack thereof, I've made a few small 
changes to the instructions.  Please see the wiki page.

Stewart.
Dec 18 2006