• Joakim (31/31) Jan 06 2014 I've been porting dmd to Android/x86 recently and I've been
• Jacob Carlborg (13/31) Jan 06 2014 For what case one could add a static assert.
• Joakim (11/28) Jan 07 2014 That's too bad, as the preprocessor is important in the headers.
• Jacob Carlborg (6/12) Jan 07 2014 Seems complicated. I need to be able to support the preprocessor anyway
• Joakim (5/19) Jan 07 2014 The "correct solution" isn't always the quick solution. ;)
• Jacob Carlborg (11/14) Jan 07 2014 Thanks.

"Joakim" <joakim airpost.net> writes:

I've been porting dmd to Android/x86 recently and I've been 
thinking about whether there's an easier way to handle such 
porting in the future.  The vast majority of my druntime patches 
simply find the equivalent function or macro in the Android C 
library, bionic.  This is just a lot of name and parameter type 
checking most of the time: couldn't this be automated for new 
platforms with libclang?

Write a D utility that finds all the platform blocks in druntime 
and calls libclang to parse the headers and translate the results 
into D, like htod does, then inserts the new version blocks into 
druntime, which you can then quickly manually check.  Perhaps 
this can be built on top of dstep, which I just found while 
googling about htod for this post: :)

https://github.com/jacob-carlborg/dstep

Obviously no utility will catch everything, for example, it's not 
going to know how to replace missing functions.  I'm not sure if 
libclang provides for a way to handle architecture-specific 
declarations either.  But it will save a bunch of time for 
druntime porters, and even catch bugs in the existing manually 
ported platforms for druntime.  Of course, it will only work on 
existing clang platforms, but I doubt we're going to beat clang 
to a platform. ;)

It could also provide a way to separate out glibc from linux, as 
we've been talking about on my Android stdc pull request.  After 
checking the relevant C header against druntime, this utility 
could query the local package manager, pacman in the case of Arch 
linux, to see what package owned that C header and then separate 
glibc from linux version blocks by using that info.

I don't have any experience with libclang, so I thought I'd run 
the idea by you all first.  Let me know if you see any problems 
with this idea.

Jacob Carlborg <doob me.com> writes:

On 2014-01-07 07:02, Joakim wrote:
 I've been porting dmd to Android/x86 recently and I've been thinking
 about whether there's an easier way to handle such porting in the
 future.  The vast majority of my druntime patches simply find the
 equivalent function or macro in the Android C library, bionic.  This is
 just a lot of name and parameter type checking most of the time:
 couldn't this be automated for new platforms with libclang?

 Write a D utility that finds all the platform blocks in druntime and
 calls libclang to parse the headers and translate the results into D,
 like htod does, then inserts the new version blocks into druntime, which
 you can then quickly manually check.  Perhaps this can be built on top
 of dstep, which I just found while googling about htod for this post: :)

 https://github.com/jacob-carlborg/dstep

That could probably be built on top of DStep or a modified version of DStep.

 Obviously no utility will catch everything, for example, it's not going
 to know how to replace missing functions.

For what case one could add a static assert.

 I'm not sure if libclang
 provides for a way to handle architecture-specific declarations either.

Unfortunately it doesn't. I need some way to deal with the preprocessor 
in DStep as well, also the Clang developers have showed interested in be 
able to deal with the preprocesssor from libclang.

Clang can of course deal with this. We probably need to expose an 
libclang API for PPCallbacks: 
http://clang.llvm.org/doxygen/classclang_1_1PPCallbacks.html

 I don't have any experience with libclang, so I thought I'd run the idea
 by you all first.  Let me know if you see any problems with this idea.

Any help on improving DStep is much appreciated. I can help you out if 
you have an questions about DStep and probably on libclang as well.

-- 
/Jacob Carlborg

"Joakim" <joakim airpost.net> writes:

On Tuesday, 7 January 2014 at 07:36:36 UTC, Jacob Carlborg wrote:
 I'm not sure if libclang
 provides for a way to handle architecture-specific 
 declarations either.

 Unfortunately it doesn't. I need some way to deal with the 
 preprocessor in DStep as well, also the Clang developers have 
 showed interested in be able to deal with the preprocesssor 
 from libclang.

 Clang can of course deal with this. We probably need to expose 
 an libclang API for PPCallbacks: 
 http://clang.llvm.org/doxygen/classclang_1_1PPCallbacks.html

That's too bad, as the preprocessor is important in the headers.  
Perhaps this can be finessed by passing various architectures or 
other preprocessor defines to the input of clang/libclang and 
then comparing the output of the various preprocessed files?  For 
the limited uses of the preprocessor that are translated for 
druntime, maybe that will be enough.

 I don't have any experience with libclang, so I thought I'd 
 run the idea
 by you all first.  Let me know if you see any problems with 
 this idea.

 Any help on improving DStep is much appreciated. I can help you 
 out if you have an questions about DStep and probably on 
 libclang as well.

At this point, I don't need such a tool, as I'm done doing most 
of it manually for Android/x86.  Maybe later, when we split linux 
from glibc and Android into linux and bionic or when I add 
Android/ARM support, we can look into building such a tool.

Jacob Carlborg <doob me.com> writes:

On 2014-01-07 12:17, Joakim wrote:

 That's too bad, as the preprocessor is important in the headers. Perhaps
 this can be finessed by passing various architectures or other
 preprocessor defines to the input of clang/libclang and then comparing
 the output of the various preprocessed files?  For the limited uses of
 the preprocessor that are translated for druntime, maybe that will be
 enough.

Seems complicated. I need to be able to support the preprocessor anyway 
in DStep. The correct solution is to extend the libclang API for this 
based on PPCallbacks.

-- 
/Jacob Carlborg

"Joakim" <joakim airpost.net> writes:

On Tuesday, 7 January 2014 at 12:24:25 UTC, Jacob Carlborg wrote:
 On 2014-01-07 12:17, Joakim wrote:

 That's too bad, as the preprocessor is important in the 
 headers. Perhaps
 this can be finessed by passing various architectures or other
 preprocessor defines to the input of clang/libclang and then 
 comparing
 the output of the various preprocessed files?  For the limited 
 uses of
 the preprocessor that are translated for druntime, maybe that 
 will be
 enough.

 Seems complicated. I need to be able to support the 
 preprocessor anyway in DStep. The correct solution is to extend 
 the libclang API for this based on PPCallbacks.

The "correct solution" isn't always the quick solution. ;) 
Anyway, thanks for building such a nice-sounding tool in DStep 
and for responding.  I'll let you know when I start playing with 
DStep.

Jacob Carlborg <doob me.com> writes:

On 2014-01-07 16:46, Joakim wrote:

 The "correct solution" isn't always the quick solution. ;) Anyway,
 thanks for building such a nice-sounding tool in DStep and for
 responding.  I'll let you know when I start playing with DStep.

Thanks.

There's another problem with DStep. Many types and functions, especially 
when we're talking about the system API's, are declared in internal 
headers and then a typedef, or similar, is used to include it in 
standard headers. DStep will not resolve the typedef and it will follow 
the same file structure as the headers do. The reason for not resolving 
typedefs is they're most likely hiding platform specifics in a 
cross-platform API.

-- 
/Jacob Carlborg