www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - htod - convert C .h files to D import files

reply Walter Bright <newshound digitalmars.com> writes:
I'm not sure how useful this will be.

Here it is: http://www.digitalmars.com/d/htod.html
May 21 2006
next sibling parent Dave <Dave_member pathlink.com> writes:
Walter Bright wrote:
 I'm not sure how useful this will be.
 
 Here it is: http://www.digitalmars.com/d/htod.html

Thanks!
May 21 2006
prev sibling next sibling parent Tom <ihate spam.com> writes:
Walter Bright escribió:
 I'm not sure how useful this will be.
 
 Here it is: http://www.digitalmars.com/d/htod.html

Wow, that was quick. Seems like a great work!
May 21 2006
prev sibling next sibling parent Derek Parnell <derek psych.ward> writes:
On Sun, 21 May 2006 16:17:38 -0700, Walter Bright wrote:

 I'm not sure how useful this will be.

Very useful. Thanks for this Walter.
 Here it is: http://www.digitalmars.com/d/htod.html

I just used it on windows.h. It found a 'bug' in winscard.h ;-) After I added #include <BaseTsd.h> #include <Guiddef.h> to winscard.h and ran it again, it generated a very useful windows.d file. The command I used was ... htod windows.h -Iy:\dm\include -hs -DUNICODE and I got a 177823 lines in windows.d (mostly comments of course.). With the -hc switch this went down to 85572 lines of useful code. -- Derek (skype: derek.j.parnell) Melbourne, Australia "Down with mediocracy!" 22/05/2006 12:05:43 PM
May 21 2006
prev sibling next sibling parent reply =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
Walter Bright wrote:

 I'm not sure how useful this will be.
 
 Here it is: http://www.digitalmars.com/d/htod.html

Q: Will there be source code later, for porting to GDC ? "Bugs: 2. No Linux version" (no Mac OS X version either) --anders
May 22 2006
parent reply Walter Bright <newshound digitalmars.com> writes:
Anders F Björklund wrote:
 Walter Bright wrote:
 
 I'm not sure how useful this will be.

 Here it is: http://www.digitalmars.com/d/htod.html

Q: Will there be source code later, for porting to GDC ?

Sorry, but it's totally based on the C compiler. But that also means it could be done to gcc.
 "Bugs: 2. No Linux version" (no Mac OS X version either)

True.
May 22 2006
next sibling parent reply Lionello Lunesu <lio lunesu.remove.com> writes:
Walter Bright wrote:
 Anders F Björklund wrote:
 Walter Bright wrote:

 I'm not sure how useful this will be.

 Here it is: http://www.digitalmars.com/d/htod.html

Q: Will there be source code later, for porting to GDC ?

Sorry, but it's totally based on the C compiler. But that also means it could be done to gcc.

I've been playing around with libcpp (the c preprocessor) from gcc and was wondering if that was indeed enough to make a tool like htod. I mean, do you need to parse the C code further, or is a preprocessor (passing you the tokens) enough to do what htod does? (You'd have to detect the 3 tokens "unsigned long long" yourself and replace it with "ulong", but that's find&replace, hardly parsing) L.
May 22 2006
parent Walter Bright <newshound digitalmars.com> writes:
Lionello Lunesu wrote:
 I've been playing around with libcpp (the c preprocessor) from gcc and 
 was wondering if that was indeed enough to make a tool like htod.
 
 I mean, do you need to parse the C code further, or is a preprocessor 
 (passing you the tokens) enough to do what htod does?
 
 (You'd have to detect the 3 tokens "unsigned long long" yourself and 
 replace it with "ulong", but that's find&replace, hardly parsing)

htod does a real parse of the C code, and generates the D output from the internal symbol table. Parsing it all the way means that the corner cases work. Also, it means it'll work with C++ header files that have extern "C" declarations in them. "unsigned long long" can be: unsigned long long long unsigned long long long unsigned int unsigned long long unsigned long const int long etc.
May 22 2006
prev sibling parent reply =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
Walter Bright wrote:

 Q: Will there be source code later, for porting to GDC ?

Sorry, but it's totally based on the C compiler. But that also means it could be done to gcc.

It doesn't have to be complete... As long as it's something like "insert C/C++ front end here", it could then be adapted to GCC ? Anyway, we can probably do a re-implementation from the documentation but the more info you are able to provide the more similar they'll get. If we can do a larger implementation test, like Dstress for the compiler, then we can run that on the two "htod" to compare them... Will do a spike / hack in a scripting language. "ghtod", I suppose. ? Time will tell if that'll do, or if we need a "real" C or D program. --anders
May 22 2006
parent Walter Bright <newshound digitalmars.com> writes:
Anders F Björklund wrote:
 Walter Bright wrote:
 
 Q: Will there be source code later, for porting to GDC ?

Sorry, but it's totally based on the C compiler. But that also means it could be done to gcc.

It doesn't have to be complete... As long as it's something like "insert C/C++ front end here", it could then be adapted to GCC ? Anyway, we can probably do a re-implementation from the documentation but the more info you are able to provide the more similar they'll get. If we can do a larger implementation test, like Dstress for the compiler, then we can run that on the two "htod" to compare them... Will do a spike / hack in a scripting language. "ghtod", I suppose. ? Time will tell if that'll do, or if we need a "real" C or D program. --anders

There really isn't much to it, it just walks the symbol table of the C compiler.
May 22 2006
prev sibling next sibling parent Lionello Lunesu <lio lunesu.remove.com> writes:
Walter Bright wrote:
 I'm not sure how useful this will be.
 
 Here it is: http://www.digitalmars.com/d/htod.html

Cool tool! Seems to work fine with some small headers I've tested. But shouldn't the tool create a .di file, like dmd -H does? L.
May 22 2006
prev sibling next sibling parent reply Lionello Lunesu <lio lunesu.remove.com> writes:
//C typedef signed char GLbyte;
alias sbyte GLbyte;

dmd gl.d

?? something new or something old? L.
May 22 2006
parent reply Lionello Lunesu <lio lunesu.remove.com> writes:
I have found some other issues with htod, but am reluctant to post them. 
  Don't want Walter's focus to shift to other tools. (Who am I to worry 
about Walter's focus? :S)

Anyway, here are some things to keep in mind when using htod:

* signed char => sbyte
* wchar_t not known (it's built-in in vc?)
* void func( void (*callback)(void) ) => void func( void 
function(...)callback );
* comments are 1 line off

L.
May 23 2006
next sibling parent reply Walter Bright <newshound digitalmars.com> writes:
Lionello Lunesu wrote:
 I have found some other issues with htod, but am reluctant to post them. 
  Don't want Walter's focus to shift to other tools. (Who am I to worry 
 about Walter's focus? :S)
 
 Anyway, here are some things to keep in mind when using htod:
 
 * signed char => sbyte

What's the issue?
 * wchar_t not known (it's built-in in vc?)

Hmm. Example?
 * void func( void (*callback)(void) ) => void func( void 
 function(...)callback );

That's correct.
 * comments are 1 line off

This can happen with macros, but it shouldn't happen with declarations.
May 23 2006
parent reply "Lionello Lunesu" <lionello lunesu.remove.com> writes:
"Walter Bright" <newshound digitalmars.com> wrote in message 
news:e4vhd6$25ti$1 digitaldaemon.com...
 Lionello Lunesu wrote:
 I have found some other issues with htod, but am reluctant to post them. 
 Don't want Walter's focus to shift to other tools. (Who am I to worry 
 about Walter's focus? :S)

 Anyway, here are some things to keep in mind when using htod:

 * signed char => sbyte


dmd claims (correctly) it doesn't know about sbyte: gl.d(42): identifier 'sbyte' is not defined
 * wchar_t not known (it's built-in in vc?)


const wchar_t* APIENTRY gluErrorUnicodeStringEXT ( GLenum errCode); wchar_t wasn't defined anywhere (although I can find it in the headers, but it must have been ifdef'ed or something). And I think it's indeed handled as a built-in type by VC, which might be the reason why it's not typedef'en explicitely. This header is from the PlatformSDK, by the way, so I was expecting some ms-only stuff.
 * void func( void (*callback)(void) ) => void func( void 
 function(...)callback );


Thought so. Still, useful info for a knowledge base : )
 * comments are 1 line off


I'll have to check the exact occurence at the office tomorrow, but it happened during the conversion of gl.h (as did the others): #define SOME_CONSTANT 123 /* multi-line comment */ => /* multi-line const SOME_CONSTANT = 123; comment */ The code ended up being commented and I had to shift it up (or down; don't recall). Just another thing to be noted, that's all. All-in-all, it still saved me a lot of time! L.
May 23 2006
parent reply Walter Bright <newshound digitalmars.com> writes:
Lionello Lunesu wrote:
 "Walter Bright" <newshound digitalmars.com> wrote in message 
 news:e4vhd6$25ti$1 digitaldaemon.com...
 Lionello Lunesu wrote:
 I have found some other issues with htod, but am reluctant to post them. 
 Don't want Walter's focus to shift to other tools. (Who am I to worry 
 about Walter's focus? :S)

 Anyway, here are some things to keep in mind when using htod:

 * signed char => sbyte


dmd claims (correctly) it doesn't know about sbyte: gl.d(42): identifier 'sbyte' is not defined

<slaps forehead> Arrgh!
May 23 2006
parent reply "Lionello Lunesu" <lionello lunesu.remove.com> writes:
 * signed char => sbyte


dmd claims (correctly) it doesn't know about sbyte: gl.d(42): identifier 'sbyte' is not defined

<slaps forehead> Arrgh!

Hey! Don't get me wrong! I've posted before that "byte" should be unsigned! So, create an sbyte for the signed byte and let's have a laugh about it. L.
May 23 2006
parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
Lionello Lunesu wrote:
 * signed char => sbyte


gl.d(42): identifier 'sbyte' is not defined


Hey! Don't get me wrong! I've posted before that "byte" should be unsigned! So, create an sbyte for the signed byte and let's have a laugh about it.

The integer types are designed to be named consistently with each other. byte, short, int and long are signed. ubyte, ushort, uint and ulong are unsigned. Stewart.
May 24 2006
parent reply Lionello Lunesu <lio lunesu.remove.com> writes:
Stewart Gordon wrote:
 byte, short, int and long are signed.
 ubyte, ushort, uint and ulong are unsigned.
 
 Stewart.

Yes, obviously. Interesting though that Walter himself wrote "sbyte" and didn't notice what was wrong with it :) There's something to be said for intuitiveness, even when it's not consequent. L.
May 24 2006
parent Daniel Keep <daniel.keep.lists gmail.com> writes:
Lionello Lunesu wrote:
 Stewart Gordon wrote:
 byte, short, int and long are signed.
 ubyte, ushort, uint and ulong are unsigned.

 Stewart.

Yes, obviously. Interesting though that Walter himself wrote "sbyte" and didn't notice what was wrong with it :) There's something to be said for intuitiveness, even when it's not consequent. L.

Funny, I always thought having 'char' in C being signed unlike every other integer type was hugely unintuitive. Usually, the first thing I do with a C or C++ program is something like this: #define byte signed char #define ubyte unsigned char Otherwise I end up getting confused... -- Daniel -- v1sw5+8Yhw5ln4+5pr6OFma8u6+7Lw4Tm6+7l6+7D a2Xs3MSr2e4/6+7t4TNSMb6HTOp5en5g6RAHCP http://hackerkey.com/
May 24 2006
prev sibling parent Sean Kelly <sean f4.ca> writes:
Lionello Lunesu wrote:
 * wchar_t not known (it's built-in in vc?)

vc actually offers an option of whether to define wchar_t as a separate type or merely to make it a typedef of unsigned short. I prefer the former option, but it doesn't work with all third-party code. Sean
May 23 2006
prev sibling next sibling parent reply Don Clugston <dac nospam.com.au> writes:
"I'm not sure how useful this will be."

That's such a typical Walter quote <g>. About as useful as a Ferrari 
that only works on roads, I reckon. Time to throw away my regexp billy-cart.

A perfect H to D converter is probably impossible, because #defines are 
so ambiguous. It takes a human who knows how the header file is intended 
to be used. However, it ought to be possible to construct patch files to 
apply to C headers prior to running them into htod. It also seems 
feasible in the long term to persuade some open-source projects to make 
their headers D-friendly.

I'm left wondering what this means for the Windows API project. I 
certainly would have done things differently, if I'd known this was 
coming. Where to now?
May 23 2006
parent Walter Bright <newshound digitalmars.com> writes:
Don Clugston wrote:
 A perfect H to D converter is probably impossible, because #defines are 
 so ambiguous.

Not just that, there are things like using "char" as "byte" in C code. There's no way to reliably distinguish the usages.
 It takes a human who knows how the header file is intended 
 to be used.

That's right. That's why the original declarations are included as comments to make it easy to go through the results line by line and check them.
 However, it ought to be possible to construct patch files to 
 apply to C headers prior to running them into htod. It also seems 
 feasible in the long term to persuade some open-source projects to make 
 their headers D-friendly.

That's why the __HTOD__ macro is predefined when running htod.
 I'm left wondering what this means for the Windows API project. I 
 certainly would have done things differently, if I'd known this was 
 coming. Where to now?

Windows would still need a lot of hand work because of its heavy dependence on the preprocessor. There are still nearly 100,000 lines of code to check. What I hope is that the D outputting version of SWIG will no longer be necessary, that one can just run the C output of SWIG through htod. That way we can easily leverage all the work put into SWIG by others.
May 23 2006
prev sibling next sibling parent reply BCS <BCS pathlink.com> writes:
Walter Bright wrote:
 I'm not sure how useful this will be.
 
 Here it is: http://www.digitalmars.com/d/htod.html

Cool!! Just the ticket. Out of curiosity, how much new coding did this take? feature request: print to stdout (would be handy for piping to sed or the like)
May 23 2006
parent Walter Bright <newshound digitalmars.com> writes:
BCS wrote:
 Out of curiosity, how much new coding did this take?

Maybe 1500 lines.
May 23 2006
prev sibling parent reply BCS <BCS pathlink.com> writes:
Walter Bright wrote:
 I'm not sure how useful this will be.
 
 Here it is: http://www.digitalmars.com/d/htod.html

One case that it doesn't handle but would seem to be simple is: #define FOO "bar" | v const char[] FOO = "bar\0"; //explicitly null terminate
May 24 2006
next sibling parent reply Tom S <h3r3tic remove.mat.uni.torun.pl> writes:
BCS wrote:
 Walter Bright wrote:
 I'm not sure how useful this will be.

 Here it is: http://www.digitalmars.com/d/htod.html

One case that it doesn't handle but would seem to be simple is: #define FOO "bar" | v const char[] FOO = "bar\0"; //explicitly null terminate

Why would you need it to be explicitly null terminated ? D string literals have a zero anyway just after their normal data (it's not reflected by the 'length' property, but it's there). -- -----BEGIN GEEK CODE BLOCK----- Version: 3.1 GCS/M d-pu s+: a-->----- C+++$>++++ UL P+ L+ E--- W++ N++ o? K? w++ !O !M V? PS- PE- Y PGP t 5 X? R tv-- b DI- D+ G e>+++ h>++ !r !y ------END GEEK CODE BLOCK------ Tomasz Stachowiak /+ a.k.a. h3r3tic +/
May 24 2006
parent reply BCS <BCS_member pathlink.com> writes:
In article <e52ivj$3am$1 digitaldaemon.com>, Tom S says...
BCS wrote:
 Walter Bright wrote:
 I'm not sure how useful this will be.

 Here it is: http://www.digitalmars.com/d/htod.html

One case that it doesn't handle but would seem to be simple is: #define FOO "bar" | v const char[] FOO = "bar\0"; //explicitly null terminate

Why would you need it to be explicitly null terminated ? D string literals have a zero anyway just after their normal data (it's not reflected by the 'length' property, but it's there).

#1 because that behavior it not in standard (AFAIK) #2 copying <code file=a.h> #define FOO "start" #define FOO "stop" int func(char* string); <\code> <code file=b.d> import a; .. char[] command = FOO; .. char[] use = command.dup; // don't known why, but someone going to do it .. func(use.prt); // now what?? .. <\code>
May 24 2006
parent Lionello Lunesu <lio lunesu.remove.com> writes:
BCS wrote:
 In article <e52ivj$3am$1 digitaldaemon.com>, Tom S says...
 BCS wrote:
 Walter Bright wrote:
 I'm not sure how useful this will be.

 Here it is: http://www.digitalmars.com/d/htod.html

One case that it doesn't handle but would seem to be simple is: #define FOO "bar" | v const char[] FOO = "bar\0"; //explicitly null terminate

literals have a zero anyway just after their normal data (it's not reflected by the 'length' property, but it's there).

#1 because that behavior it not in standard (AFAIK) #2 copying <code file=a.h> #define FOO "start" #define FOO "stop" int func(char* string); <\code> <code file=b.d> import a; .. char[] command = FOO; .. char[] use = command.dup; // don't known why, but someone going to do it .. func(use.prt); // now what?? .. <\code>

But that's simply a bug in the code, not in the "header". What if "use" included the zero terminator, what would "use ~= "t";" result it? A "t" after the zero terminator? So, either the append code would have to strip zero terminators before appending (sure hope it won't), or the length property should not include the zero, which is exactly what's being done now. L.
May 24 2006
prev sibling parent reply Derek Parnell <derek psych.ward> writes:
On Wed, 24 May 2006 14:15:43 -0700, BCS wrote:

 Walter Bright wrote:
 I'm not sure how useful this will be.
 
 Here it is: http://www.digitalmars.com/d/htod.html

One case that it doesn't handle but would seem to be simple is: #define FOO "bar" | v const char[] FOO = "bar\0"; //explicitly null terminate

But if that happened, things lke concatenation would cause problems. #define FOO "foo" #define BAR "bar" | | V const char[] FOO = "foo\0"; //explicitly null terminate const char[] BAR = "bar\0"; //explicitly null terminate . . . char[] x = FOO ~ BAR; // Now it has an embedded \0 ! -- Derek (skype: derek.j.parnell) Melbourne, Australia "Down with mediocracy!" 25/05/2006 5:03:13 PM
May 25 2006
parent reply BCS <BCS_member pathlink.com> writes:
In article <1xu5l9je8ge3f.txbccq4m8i7m.dlg 40tude.net>, Derek Parnell says...
On Wed, 24 May 2006 14:15:43 -0700, BCS wrote:



But if that happened, things lke concatenation would cause problems. #define FOO "foo" #define BAR "bar" | | V const char[] FOO = "foo\0"; //explicitly null terminate const char[] BAR = "bar\0"; //explicitly null terminate . . . char[] x = FOO ~ BAR; // Now it has an embedded \0 ! --

I'll grant that. I think this will be another case of there being no 1-to-1 correspondence from C to D. I expect that with char constants, they are more likely to get copied than concatenated. The work around for either is easy as long as you remember it. Char[] foo = "bar", copy; copy = bar ~ \0; \\cat copies Char[] foo = "bar\0" bing = "baz\0", copy; copy = bar[0..$-1] ~ bing ; \\cat copies
May 25 2006
parent reply Rémy Mouëza <Rémy_member pathlink.com> writes:
But if that happened, things lke concatenation would cause problems.

#define FOO "foo"
#define BAR "bar"
   |
   |
   V
const char[] FOO = "foo\0"; //explicitly null terminate
const char[] BAR = "bar\0"; //explicitly null terminate

. . .

  char[] x = FOO ~ BAR; // Now it has an embedded \0 !


I thought that D managed the "\0" characters ( that was why we hadn't to put any in our char [] variables ). Therefore I made the following test : // constchar.d import std.stdio ; const char [] foo = "foo\0" ; typeof ( foo ) bar = "bar\0" ; void main () { char [] test = foo ~ bar ; writefln ( test ); } ray Moonraker:~/dee/tmp$ ./constchar foobar Seems that there is no problem. I may have not understand something.
May 27 2006
next sibling parent Lars Ivar Igesund <larsivar igesund.net> writes:
Rémy Mouëza wrote:

 import std.stdio ;
 
 const char [] foo  = "foo\0" ;
 typeof ( foo ) bar = "bar\0" ;
 
 void main ()
 {
 char [] test = foo ~ bar ;
 
 writefln ( test );
 }

No, there are no problems, but your string test has length 8 now (\0 is a non-printable character). -- Lars Ivar Igesund blog at http://larsivi.net DSource & #D: larsivi
May 27 2006
prev sibling parent reply Chris Nicholson-Sauls <ibisbasenji gmail.com> writes:
Rémy Mouëza wrote:
But if that happened, things lke concatenation would cause problems.

#define FOO "foo"
#define BAR "bar"
  |
  |
  V
const char[] FOO = "foo\0"; //explicitly null terminate
const char[] BAR = "bar\0"; //explicitly null terminate

. . .

 char[] x = FOO ~ BAR; // Now it has an embedded \0 !


I thought that D managed the "\0" characters ( that was why we hadn't to put any in our char [] variables ). Therefore I made the following test : // constchar.d import std.stdio ; const char [] foo = "foo\0" ; typeof ( foo ) bar = "bar\0" ; void main () { char [] test = foo ~ bar ; writefln ( test ); } ray Moonraker:~/dee/tmp$ ./constchar foobar Seems that there is no problem. I may have not understand something.

With DMD 0.158 on Windows, I got "foo bar" from this. Note the space between. Also note that writef is written in D, for D, and therefore doesn't use null characters as end-of-string (it relies on the .length property instead). The big problem had to do with sending these strings to C code, which /will/ see the null's as an end-of-array marker. Try replacing your writefln with a printf, and you will only get "foo" printed to the screen. -- Chris Nicholson-Sauls
May 27 2006
parent Rémy Mouëza <Rémy_member pathlink.com> writes:
In article <e5a6dp$ols$1 digitaldaemon.com>, Chris Nicholson-Sauls says...
With DMD 0.158 on Windows, I got "foo bar" from this.  Note the space between. 
Also note 
that writef is written in D, for D, and therefore doesn't use null characters
as 
end-of-string (it relies on the .length property instead).  The big problem had
to do with 
sending these strings to C code, which /will/ see the null's as an end-of-array
marker. 
Try replacing your writefln with a printf, and you will only get "foo" printed
to the screen.

Using printf with dmd 0.158 on Linux, I've got 5 spaces and then "foo". I guess that even a restricted use of the C preprocessor for our D interfaces won't solve the problem since it seems to be type specific, will it ?
May 28 2006