www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Hitchikers Guide to Porting Phobos / D Runtime to other architectures

reply "Iain Buclaw" <ibuclaw ubuntu.com> writes:
I got asked whether there are any porting hints for phobos on 
other architectures the other day from the debian GCC 
maintainers.  So I gathered this must be at least a dedicated 
wiki or article to be written up on the subject. :)

I know there are a few working on porting gdc and associated 
libraries over to ARM (with my assistance from the compiler 
side).  So please tell, what are your experiences? Successes?  
Failures?  What tips would you give to someone wanting to port to 
their own architecture?

Regards
Iain
Apr 08 2012
next sibling parent =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <xtzgzorex gmail.com> writes:
On 08-04-2012 21:08, Iain Buclaw wrote:
 I got asked whether there are any porting hints for phobos on other
 architectures the other day from the debian GCC maintainers. So I
 gathered this must be at least a dedicated wiki or article to be written
 up on the subject. :)

 I know there are a few working on porting gdc and associated libraries
 over to ARM (with my assistance from the compiler side). So please tell,
 what are your experiences? Successes? Failures? What tips would you give
 to someone wanting to port to their own architecture?

 Regards
 Iain

For the love of god, use D_LP64. I cannot count how many times X86 and X86_64 (and similar pairs) have been misused for this. Don't rely on extern (D) for inline asm in any capacity. It differs across compilers, architectures, bitnesses, and OSs (this is one seriously stupid aspect of the language). Not so much for when you're porting, but as a help for others who might have to port platform-specific code you're writing: *Always* include an else block for the unsupported case that static asserts. -- - Alex
Apr 08 2012
prev sibling next sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Sun, 08 Apr 2012 21:08:52 +0200
schrieb "Iain Buclaw" <ibuclaw ubuntu.com>:

 I got asked whether there are any porting hints for phobos on 
 other architectures the other day from the debian GCC 
 maintainers.  So I gathered this must be at least a dedicated 
 wiki or article to be written up on the subject. :)
 
 I know there are a few working on porting gdc and associated 
 libraries over to ARM (with my assistance from the compiler 
 side).  So please tell, what are your experiences? Successes?  
 Failures?  What tips would you give to someone wanting to port to 
 their own architecture?
 
 Regards
 Iain

(This is mostly about porting to a different C library. I don't remember many issues when porting to a different CPU architecture) Issues I hit with druntime: * Adapting the core.stdc bindings to something different than the currently supported C libraries sucks: The version blocks are sometimes completely wrong. For example Android's bionic is a C library based on BSD code, but running on Linux. As a result sometimes the version(FreeBSD) blocks apply for bionic, but sometimes the version(linux) blocks are right. I basically had to rewrite the complete core.stdc bindings. This is an issue because druntime and phobos do not distinguish between OS/Kernel and C library. * Wrong constants or macros in the C bindings are very hard to spot - you'll only notice those at runtime * When statically linking the phobos/druntime library you are no warned about missing symbols - For shared libraries -Wl,--no-undefined can be used, however, there are some issues with that as well: (http://stackoverflow.com/questions/2356168/force-gcc-to-notify-about-undefined-references-in-shared-libraries second answer) * Bionic just implements some functions as macros and never exports those as functions (htons, etc). Because of the last point it's easy to miss that Ideally all of the core.stdc bindings should be generated automatically. This is possible if we can run code (using offsetof, alignof, etc) but it's not that easy for cross compilation. I thought about hooking into the GCC C frontend to do that, but I had no time to look at it yet. * All those issues also apply to phobos, where phobos uses custom C bindings / extern(C) declarations. * I had to edit some stuff in std.stdio (because Android has no wide character/fwide support). Templates can be annoying in this case: some if(isOutputRange!T) chains hid an error in the IO code, it took me some time to find that problem. The reported error was completely misleading (cannot put dchar[] into LockingTextWriter or something) * When adding new, system specific code to a module and using selective imports, that may affect other modules (can't remember which compiler bug this was). This means that adding an import in one module might break another module on another architecture. * Porting the GC doesn't seem to be too difficult, but some care is needed to get stack scanning/TLS scanning right (If you have random crashes, it's either the GC not working(probably not scanning stack/tls) or fno-section-anchors missing) * Always use "-fno-section-anchors". It's not needed for simple code, but I was chasing a weird bug in derelict, till I realized I didn't compile derelict with "-fno-section-anchors". * Right now, issue 284 is a little annoying. At least unittest and phobos/druntime as shared libraries won't work at all till that's fixed. * AFAIK the unittests cannot be run when cross-compiling right now? * There might be more issues like this one where phobos is checking for a wrong status code: (https://github.com/D-Programming-Language/phobos/pull/487) * For systems where long double isn't available, fixing core.stdc.math is annoying. I have to implement a proper solution which works for all systems without long double. However, all that considered most issues are when interfacing C. The D code most of the time 'just works'.
Apr 09 2012
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-04-09 11:05, Johannes Pfau wrote:

 * Adapting the core.stdc bindings to something different than the
    currently supported C libraries sucks: The version blocks are
    sometimes completely wrong. For example Android's bionic is a C
    library based on BSD code, but running on Linux. As a result
    sometimes the version(FreeBSD) blocks apply for bionic, but sometimes
    the version(linux) blocks are right. I basically had to rewrite
    the complete core.stdc bindings. This is an issue because druntime
    and phobos do not distinguish between OS/Kernel and C library.

Is it possible to treat bionic as its own platform: version (bionic) {} else version (linux{} and so on. -- /Jacob Carlborg
Apr 09 2012
prev sibling parent Etienne <etcimon globecsys.com> writes:
On 2013-12-03 10:43 AM, Iain Buclaw wrote:
 On 3 December 2013 15:30, Joakim <joakim airpost.net> wrote:
 On Tuesday, 3 December 2013 at 15:13:36 UTC, Johannes Pfau wrote:
 Am Tue, 03 Dec 2013 12:26:57 +0100
 schrieb "Joakim" <joakim airpost.net>:

 Seems like you got pretty far with your Android port: are you planning on
 submitting any of these patches back upstream?

At some point, probably yes. However, I want to get the 'easy' stuff working first. This means a stable and well-tested ARM/Linux/glibc build should be available first. (It makes more sense this way as we still have/had some ARM codegen bugs in the compiler.)

OK, anything against putting your incomplete Android port online someplace, say on github? I'm working on an Android/x86 port now, that way we could work together and submit our patches together. This would especially allay Iain's concerns about future conflicts between Android/x86 and Android/ARM patches. My work is publicly available here: https://github.com/joakim-noah/druntime/tree/android

They are also the concerns of Walter, David and Kai. But yes I'm a bit more vocal about it than some. :~)

I'm very interested in this, more specifically when ARM support is such that porting the vibe.d library to Android/ARM or Linux/ARM in general would allow lightweight micro serial-port servers like the BeagleBone to be operated on with WebSockets. This would definitely make robotics more simple. I know it would be interesting for me, I'd use it to build a simple, mini RC airplane equipped with a GoPro! camera that works over LTE. Oh the fun I'd have :) Keep it up
Dec 03 2013
prev sibling next sibling parent Iain Buclaw <ibuclaw ubuntu.com> writes:
On 9 April 2012 10:35, Jacob Carlborg <doob me.com> wrote:
 On 2012-04-09 11:05, Johannes Pfau wrote:

 * Adapting the core.stdc bindings to something different than the
 =A0 currently supported C libraries sucks: The version blocks are
 =A0 sometimes completely wrong. For example Android's bionic is a C
 =A0 library based on BSD code, but running on Linux. As a result
 =A0 sometimes the version(FreeBSD) blocks apply for bionic, but sometime=


 =A0 the version(linux) blocks are right. I basically had to rewrite
 =A0 the complete core.stdc bindings. This is an issue because druntime
 =A0 and phobos do not distinguish between OS/Kernel and C library.

Is it possible to treat bionic as its own platform: version (bionic) {} else version (linux{} and so on.

Personally I feel that people porting to specific architectures should maintain their differences in separate files under a /ports directory structure - lets say core.stdc.stdio as a cod example. The version for bionic would be under /ports/bionic/core/stdc/stdio.d, and that is the module that gets compiled into the library when building for bionic. When installing, the build process generates a header file of the bionic version of core.stdc.stdio and puts the file in the correct /include/core/stdc/stdio.di location. Though it is fine to say using version {} else version {} else static assert(false); when dealing with a small set of architectures. I feel strongly this is not practical when considering there are 23+ architectures and 12+ platforms that could be in mixed combination. The result would either be lots of code duplications everywhere, or just a wiry long block of spaghetti code. Every port in one file would (eventually) make it difficult for maintainers IMO. --=20 Iain Buclaw *(p < e ? p++ : p) =3D (c & 0x0f) + '0';
Apr 09 2012
prev sibling next sibling parent "Mike" <none none.com> writes:
 Personally I feel that people porting to specific architectures 
 should
 maintain their differences in separate files under a /ports 
 directory
 structure - lets say core.stdc.stdio as a cod example. The 
 version for
 bionic would be under /ports/bionic/core/stdc/stdio.d, and that 
 is the
 module that gets compiled into the library when building for 
 bionic.
 When installing, the build process generates a header file of 
 the
 bionic version of core.stdc.stdio and puts the file in the 
 correct
 /include/core/stdc/stdio.di location.

 Though it is fine to say using version {} else version {} else 
 static
 assert(false); when dealing with a small set of architectures.  
 I feel
 strongly this is not practical when considering there are 23+
 architectures and 12+ platforms that could be in mixed 
 combination.
 The result would either be lots of code duplications 
 everywhere, or
 just a wiry long block of spaghetti code.  Every port in one 
 file
 would (eventually) make it difficult for maintainers IMO.

I agree. Submitted an enhancement here: https://d.puremagic.com/issues/show_bug.cgi?id=11666
Dec 02 2013
prev sibling next sibling parent Iain Buclaw <ibuclaw gdcproject.org> writes:
On 3 December 2013 02:43, Mike <none none.com> wrote:
 Personally I feel that people porting to specific architectures should
 maintain their differences in separate files under a /ports directory
 structure - lets say core.stdc.stdio as a cod example. The version for
 bionic would be under /ports/bionic/core/stdc/stdio.d, and that is the
 module that gets compiled into the library when building for bionic.
 When installing, the build process generates a header file of the
 bionic version of core.stdc.stdio and puts the file in the correct
 /include/core/stdc/stdio.di location.

 Though it is fine to say using version {} else version {} else static
 assert(false); when dealing with a small set of architectures.  I feel
 strongly this is not practical when considering there are 23+
 architectures and 12+ platforms that could be in mixed combination.
 The result would either be lots of code duplications everywhere, or
 just a wiry long block of spaghetti code.  Every port in one file
 would (eventually) make it difficult for maintainers IMO.

I agree. Submitted an enhancement here: https://d.puremagic.com/issues/show_bug.cgi?id=11666

Thanks. My name is Iain.
Dec 03 2013
prev sibling next sibling parent "Joakim" <joakim airpost.net> writes:
On Monday, 9 April 2012 at 09:05:25 UTC, Johannes Pfau wrote:
 Am Sun, 08 Apr 2012 21:08:52 +0200
 schrieb "Iain Buclaw" <ibuclaw ubuntu.com>:

 I got asked whether there are any porting hints for phobos on 
 other architectures the other day from the debian GCC 
 maintainers.  So I gathered this must be at least a dedicated 
 wiki or article to be written up on the subject. :)
 
 I know there are a few working on porting gdc and associated 
 libraries over to ARM (with my assistance from the compiler 
 side).  So please tell, what are your experiences? Successes?  
 Failures?  What tips would you give to someone wanting to port 
 to their own architecture?
 
 Regards
 Iain

(This is mostly about porting to a different C library. I don't remember many issues when porting to a different CPU architecture) Issues I hit with druntime: * Adapting the core.stdc bindings to something different than the currently supported C libraries sucks: The version blocks are sometimes completely wrong. For example Android's bionic is a C library based on BSD code, but running on Linux. As a result sometimes the version(FreeBSD) blocks apply for bionic, but sometimes the version(linux) blocks are right. I basically had to rewrite the complete core.stdc bindings. This is an issue because druntime and phobos do not distinguish between OS/Kernel and C library. * Wrong constants or macros in the C bindings are very hard to spot - you'll only notice those at runtime * When statically linking the phobos/druntime library you are no warned about missing symbols - For shared libraries -Wl,--no-undefined can be used, however, there are some issues with that as well: (http://stackoverflow.com/questions/2356168/force-gcc-to-notify-about-undefined-references-in-shared-libraries second answer) * Bionic just implements some functions as macros and never exports those as functions (htons, etc). Because of the last point it's easy to miss that Ideally all of the core.stdc bindings should be generated automatically. This is possible if we can run code (using offsetof, alignof, etc) but it's not that easy for cross compilation. I thought about hooking into the GCC C frontend to do that, but I had no time to look at it yet. * All those issues also apply to phobos, where phobos uses custom C bindings / extern(C) declarations. * I had to edit some stuff in std.stdio (because Android has no wide character/fwide support). Templates can be annoying in this case: some if(isOutputRange!T) chains hid an error in the IO code, it took me some time to find that problem. The reported error was completely misleading (cannot put dchar[] into LockingTextWriter or something) * When adding new, system specific code to a module and using selective imports, that may affect other modules (can't remember which compiler bug this was). This means that adding an import in one module might break another module on another architecture. * Porting the GC doesn't seem to be too difficult, but some care is needed to get stack scanning/TLS scanning right (If you have random crashes, it's either the GC not working(probably not scanning stack/tls) or fno-section-anchors missing) * Always use "-fno-section-anchors". It's not needed for simple code, but I was chasing a weird bug in derelict, till I realized I didn't compile derelict with "-fno-section-anchors". * Right now, issue 284 is a little annoying. At least unittest and phobos/druntime as shared libraries won't work at all till that's fixed. * AFAIK the unittests cannot be run when cross-compiling right now? * There might be more issues like this one where phobos is checking for a wrong status code: (https://github.com/D-Programming-Language/phobos/pull/487) * For systems where long double isn't available, fixing core.stdc.math is annoying. I have to implement a proper solution which works for all systems without long double. However, all that considered most issues are when interfacing C. The D code most of the time 'just works'.

planning on submitting any of these patches back upstream?
Dec 03 2013
prev sibling next sibling parent Johannes Pfau <nospam example.com> writes:
Am Tue, 03 Dec 2013 12:26:57 +0100
schrieb "Joakim" <joakim airpost.net>:

 Seems like you got pretty far with your Android port: are you 
 planning on submitting any of these patches back upstream?

At some point, probably yes. However, I want to get the 'easy' stuff working first. This means a stable and well-tested ARM/Linux/glibc build should be available first. (It makes more sense this way as we still have/had some ARM codegen bugs in the compiler.)
Dec 03 2013
prev sibling next sibling parent Iain Buclaw <ibuclaw gdcproject.org> writes:
On 3 December 2013 16:13, Johannes Pfau <nospam example.com> wrote:
 Am Tue, 03 Dec 2013 12:26:57 +0100
 schrieb "Joakim" <joakim airpost.net>:

 Seems like you got pretty far with your Android port: are you
 planning on submitting any of these patches back upstream?

At some point, probably yes. However, I want to get the 'easy' stuff working first. This means a stable and well-tested ARM/Linux/glibc build should be available first. (It makes more sense this way as we still have/had some ARM codegen bugs in the compiler.)

That need testing / approval. ;-)
Dec 03 2013
prev sibling next sibling parent "Joakim" <joakim airpost.net> writes:
On Tuesday, 3 December 2013 at 15:13:36 UTC, Johannes Pfau wrote:
 Am Tue, 03 Dec 2013 12:26:57 +0100
 schrieb "Joakim" <joakim airpost.net>:

 Seems like you got pretty far with your Android port: are you 
 planning on submitting any of these patches back upstream?

At some point, probably yes. However, I want to get the 'easy' stuff working first. This means a stable and well-tested ARM/Linux/glibc build should be available first. (It makes more sense this way as we still have/had some ARM codegen bugs in the compiler.)

OK, anything against putting your incomplete Android port online someplace, say on github? I'm working on an Android/x86 port now, that way we could work together and submit our patches together. This would especially allay Iain's concerns about future conflicts between Android/x86 and Android/ARM patches. My work is publicly available here: https://github.com/joakim-noah/druntime/tree/android
Dec 03 2013
prev sibling next sibling parent Iain Buclaw <ibuclaw gdcproject.org> writes:
On 3 December 2013 15:30, Joakim <joakim airpost.net> wrote:
 On Tuesday, 3 December 2013 at 15:13:36 UTC, Johannes Pfau wrote:
 Am Tue, 03 Dec 2013 12:26:57 +0100
 schrieb "Joakim" <joakim airpost.net>:

 Seems like you got pretty far with your Android port: are you planning on
 submitting any of these patches back upstream?

At some point, probably yes. However, I want to get the 'easy' stuff working first. This means a stable and well-tested ARM/Linux/glibc build should be available first. (It makes more sense this way as we still have/had some ARM codegen bugs in the compiler.)

OK, anything against putting your incomplete Android port online someplace, say on github? I'm working on an Android/x86 port now, that way we could work together and submit our patches together. This would especially allay Iain's concerns about future conflicts between Android/x86 and Android/ARM patches. My work is publicly available here: https://github.com/joakim-noah/druntime/tree/android

They are also the concerns of Walter, David and Kai. But yes I'm a bit more vocal about it than some. :~)
Dec 03 2013
prev sibling next sibling parent Johannes Pfau <nospam example.com> writes:
Am Tue, 3 Dec 2013 15:23:54 +0000
schrieb Iain Buclaw <ibuclaw gdcproject.org>:

 On 3 December 2013 16:13, Johannes Pfau <nospam example.com> wrote:
 Am Tue, 03 Dec 2013 12:26:57 +0100
 schrieb "Joakim" <joakim airpost.net>:

 Seems like you got pretty far with your Android port: are you
 planning on submitting any of these patches back upstream?

At some point, probably yes. However, I want to get the 'easy' stuff working first. This means a stable and well-tested ARM/Linux/glibc build should be available first. (It makes more sense this way as we still have/had some ARM codegen bugs in the compiler.)

That need testing / approval. ;-)

I know ;-) I found yet another codegen issue when testing dub, currently reducing that issue.
Dec 03 2013
prev sibling next sibling parent Johannes Pfau <nospam example.com> writes:
Am Tue, 03 Dec 2013 16:30:10 +0100
schrieb "Joakim" <joakim airpost.net>:

 On Tuesday, 3 December 2013 at 15:13:36 UTC, Johannes Pfau wrote:
 Am Tue, 03 Dec 2013 12:26:57 +0100
 schrieb "Joakim" <joakim airpost.net>:

 Seems like you got pretty far with your Android port: are you 
 planning on submitting any of these patches back upstream?

At some point, probably yes. However, I want to get the 'easy' stuff working first. This means a stable and well-tested ARM/Linux/glibc build should be available first. (It makes more sense this way as we still have/had some ARM codegen bugs in the compiler.)

OK, anything against putting your incomplete Android port online someplace, say on github? I'm working on an Android/x86 port now, that way we could work together and submit our patches together. This would especially allay Iain's concerns about future conflicts between Android/x86 and Android/ARM patches. My work is publicly available here: https://github.com/joakim-noah/druntime/tree/android

Sure, here's the code: Android stuff, but I guess 99% of it is obsolete now https://github.com/jpf91/GDC/commits/android NDK buildscripts integration: https://github.com/jpf91/ndk-build/commits/master But that stuff is really old and probably not very interesting. Here's the current ARM/GlibC work: https://github.com/jpf91/GDC/commits/arm
Dec 03 2013
prev sibling next sibling parent "Mike" <none none.com> writes:
On Tuesday, 3 December 2013 at 09:28:11 UTC, Iain Buclaw wrote:
 On 3 December 2013 02:43, Mike <none none.com> wrote:
 Personally I feel that people porting to specific 
 architectures should
 maintain their differences in separate files under a /ports 
 directory
 structure - lets say core.stdc.stdio as a cod example. The 
 version for
 bionic would be under /ports/bionic/core/stdc/stdio.d, and 
 that is the
 module that gets compiled into the library when building for 
 bionic.
 When installing, the build process generates a header file of 
 the
 bionic version of core.stdc.stdio and puts the file in the 
 correct
 /include/core/stdc/stdio.di location.

 Though it is fine to say using version {} else version {} 
 else static
 assert(false); when dealing with a small set of 
 architectures.  I feel
 strongly this is not practical when considering there are 23+
 architectures and 12+ platforms that could be in mixed 
 combination.
 The result would either be lots of code duplications 
 everywhere, or
 just a wiry long block of spaghetti code.  Every port in one 
 file
 would (eventually) make it difficult for maintainers IMO.

I agree. Submitted an enhancement here: https://d.puremagic.com/issues/show_bug.cgi?id=11666

Thanks. My name is Iain.

Iain (sorry for the misspelling), I'm wondering if you could please elaborate on the following statement: "When installing, the build process generates a header file of the bionic version of core.stdc.stdio and puts the file in the correct /include/core/stdc/stdio.di location." Being new to D and the D Runtime build process, while I understand the general principle, I'm not seeing exactly how this can be implemented. * How are header files generated in the build process? * Once all the necessary files are generated, what need to be done to tell the compiler to "use this one"? Also, D has "header files"? Tell me it isn't so. Did you mean a .di file? Are these called "header files"?
Dec 03 2013
prev sibling next sibling parent "Joakim" <joakim airpost.net> writes:
On Tuesday, 3 December 2013 at 21:39:20 UTC, Johannes Pfau wrote:
 Am Tue, 03 Dec 2013 16:30:10 +0100
 schrieb "Joakim" <joakim airpost.net>:

 On Tuesday, 3 December 2013 at 15:13:36 UTC, Johannes Pfau 
 wrote:
 Am Tue, 03 Dec 2013 12:26:57 +0100
 schrieb "Joakim" <joakim airpost.net>:

 Seems like you got pretty far with your Android port: are 
 you planning on submitting any of these patches back 
 upstream?

At some point, probably yes. However, I want to get the 'easy' stuff working first. This means a stable and well-tested ARM/Linux/glibc build should be available first. (It makes more sense this way as we still have/had some ARM codegen bugs in the compiler.)

OK, anything against putting your incomplete Android port online someplace, say on github? I'm working on an Android/x86 port now, that way we could work together and submit our patches together. This would especially allay Iain's concerns about future conflicts between Android/x86 and Android/ARM patches. My work is publicly available here: https://github.com/joakim-noah/druntime/tree/android

Sure, here's the code: Android stuff, but I guess 99% of it is obsolete now https://github.com/jpf91/GDC/commits/android

mentioned earlier that you "had to rewrite the complete core.stdc bindings," but there are only three small patches to core.stdc in the above branch: https://github.com/jpf91/GDC/commit/d807f9d8a1cc51efe8b14a48d6a294174dc92168 You also mentioned porting the GC, but there are no patches for that. I thought you'd gotten further along on an Android port based on your earlier comments. It appears that we hit the same stderr issues though. :)
Dec 03 2013
prev sibling next sibling parent Iain Buclaw <ibuclaw gdcproject.org> writes:
On 4 December 2013 01:38, Mike <none none.com> wrote:
 On Tuesday, 3 December 2013 at 09:28:11 UTC, Iain Buclaw wrote:
 On 3 December 2013 02:43, Mike <none none.com> wrote:
 Personally I feel that people porting to specific architectures should
 maintain their differences in separate files under a /ports directory
 structure - lets say core.stdc.stdio as a cod example. The version for
 bionic would be under /ports/bionic/core/stdc/stdio.d, and that is the
 module that gets compiled into the library when building for bionic.
 When installing, the build process generates a header file of the
 bionic version of core.stdc.stdio and puts the file in the correct
 /include/core/stdc/stdio.di location.

 Though it is fine to say using version {} else version {} else static
 assert(false); when dealing with a small set of architectures.  I feel
 strongly this is not practical when considering there are 23+
 architectures and 12+ platforms that could be in mixed combination.
 The result would either be lots of code duplications everywhere, or
 just a wiry long block of spaghetti code.  Every port in one file
 would (eventually) make it difficult for maintainers IMO.

I agree. Submitted an enhancement here: https://d.puremagic.com/issues/show_bug.cgi?id=11666

Thanks. My name is Iain.

Iain (sorry for the misspelling), I'm wondering if you could please elaborate on the following statement: "When installing, the build process generates a header file of the bionic version of core.stdc.stdio and puts the file in the correct /include/core/stdc/stdio.di location." Being new to D and the D Runtime build process, while I understand the general principle, I'm not seeing exactly how this can be implemented. * How are header files generated in the build process? * Once all the necessary files are generated, what need to be done to tell the compiler to "use this one"? Also, D has "header files"? Tell me it isn't so. Did you mean a .di file? Are these called "header files"?

1. There are interface files installed in header locations on your filesystem. ;-) They used to be generated in the build process (ie: gdc -fsyntax-only -fintfc) but now they are just installed from *.d -> *.di 2. You don't need to tell the compiler to *use this one* because ~only one is installed~. Which one to install depends on whatever some configure script/make rules decides based on your target system when you build/install the library.
Dec 03 2013
prev sibling next sibling parent Johannes Pfau <nospam example.com> writes:
Am Wed, 04 Dec 2013 08:29:11 +0100
schrieb "Joakim" <joakim airpost.net>:

 On Tuesday, 3 December 2013 at 21:39:20 UTC, Johannes Pfau wrote:
 Am Tue, 03 Dec 2013 16:30:10 +0100
 schrieb "Joakim" <joakim airpost.net>:

 On Tuesday, 3 December 2013 at 15:13:36 UTC, Johannes Pfau 
 wrote:
 Am Tue, 03 Dec 2013 12:26:57 +0100
 schrieb "Joakim" <joakim airpost.net>:

 Seems like you got pretty far with your Android port: are 
 you planning on submitting any of these patches back 
 upstream?

At some point, probably yes. However, I want to get the 'easy' stuff working first. This means a stable and well-tested ARM/Linux/glibc build should be available first. (It makes more sense this way as we still have/had some ARM codegen bugs in the compiler.)

OK, anything against putting your incomplete Android port online someplace, say on github? I'm working on an Android/x86 port now, that way we could work together and submit our patches together. This would especially allay Iain's concerns about future conflicts between Android/x86 and Android/ARM patches. My work is publicly available here: https://github.com/joakim-noah/druntime/tree/android

Sure, here's the code: Android stuff, but I guess 99% of it is obsolete now https://github.com/jpf91/GDC/commits/android

mentioned earlier that you "had to rewrite the complete core.stdc bindings," but there are only three small patches to core.stdc in the above branch: https://github.com/jpf91/GDC/commit/d807f9d8a1cc51efe8b14a48d6a294174dc92168

Hmm, it's possible that I didn't push those changes to github. I might have some backups of that stuff, but I'm not sure. (IIRC I had another git project for that where I experimented with the 'ports' idea)
 
 You also mentioned porting the GC, but there are no patches for
 that.  I thought you'd gotten further along on an Android port
 based on your earlier comments.  It appears that we hit the same
 stderr issues though. :)

'porting' is an overstatement. The GC is already quite portable, all I did was adding functions to get the stack top / bottom on Android. (Stack top is actually handled in a generic way in gdc). See https://github.com/jpf91/GDC/blob/428a7573896962acfdd8132465b1f4af0ff3aea8/d/druntime/rt/memory.d#L119 and search for Android. (Of course no real testing was done and the GC could still be broken in other ways. But stack-scanning should work, TLS scanning is portable anyway and heap scanning should work as well)
Dec 04 2013
prev sibling next sibling parent "Joakim" <joakim airpost.net> writes:
On Tuesday, 3 December 2013 at 02:43:34 UTC, Mike wrote:
 I agree.  Submitted an enhancement here:  
 https://d.puremagic.com/issues/show_bug.cgi?id=11666

What new platform are you porting druntime to? I'm guessing linux/ARM Cortex-M based on your previous posts. Hopefully, I can reuse some of your work when I try ARM out.
Jan 05 2014
prev sibling next sibling parent "Mike" <none none.com> writes:
On Sunday, 5 January 2014 at 17:07:10 UTC, Joakim wrote:
 What new platform are you porting druntime to?  I'm guessing 
 linux/ARM Cortex-M based on your previous posts.  Hopefully, I 
 can reuse some of your work when I try ARM out.

I'm porting to an STM32F4 (http://www.st.com/web/en/catalog/mmc/FM141/SC1169/SS1577) MCU, simply because that's the only hardware I have, and I'm familiar with it, but I hope it will be portable to any ARM Cortex-M MCU with little or no modification. I'm doing a bare-metal port (no OS) so Linux way out of scope for me. My goal to make something like the Arduino. So the end product would be end up with the following: * The hardware (I/O board). Think Arduino DUE (http://store.arduino.cc/index.php?main_page=product_info&cPath=11&products_id=243) * A compiler specific to the hardware (LCD with ARM Thumb backend, or GDC cross-compiled for arm-none-eabi * A port of the D runtime. Definitely not a complete port. Just enough to work the I/O and make use D programming constructs (classes, structs, exceptions, thread local storage, etc...) * A library, or set of libraries, to make programming the IO convenient. See the Arduino reference to get a an idea (http://arduino.cc/en/Reference/HomePage) * Possibly an IDE specific to microcontroller development. I'd like to eventually create a very tiny real-time OS like FreeRTOS(http://www.freertos.org/) and the like, but that is a very long-term goal. Other goals are: * Assist the LDC and GDC folks with supporting this platform, as this adventure truly stands on their shoulders and we're not going to get anywhere without them. * Show that the D language can replace C/C++ in the 32-bit MCU realm. Timo Sintonen is also well on his way to an ARM Cortex-M port (https://bitbucket.org/timosi/minlibd), but he's taking quite a different approach than I. I'm taking more of a bottom up approach and hope to discard the peripheral library, the C library, and anything else C/C++ and do everything in D. I've created a simple "hello world" tutorial here (http://wiki.dlang.org/Extremely_minimal_semihosted_%22Hello_World%22) so people interested in this or similar endeavors can have a convenient place to start. At the moment, I don't have any source code to publish, as everything I've built so far is simply for academic purposes (I have a lot to learn). But, I'd be happy to share what I have and what I've learned, and I hope in the coming months I'll have a repository with something useful. Mike
Jan 05 2014
prev sibling next sibling parent "Joakim" <joakim airpost.net> writes:
On Monday, 6 January 2014 at 01:32:09 UTC, Mike wrote:
 On Sunday, 5 January 2014 at 17:07:10 UTC, Joakim wrote:
 What new platform are you porting druntime to?  I'm guessing 
 linux/ARM Cortex-M based on your previous posts.  Hopefully, I 
 can reuse some of your work when I try ARM out.

I'm porting to an STM32F4 (http://www.st.com/web/en/catalog/mmc/FM141/SC1169/SS1577) MCU, simply because that's the only hardware I have, and I'm familiar with it, but I hope it will be portable to any ARM Cortex-M MCU with little or no modification. I'm doing a bare-metal port (no OS) so Linux way out of scope for me. My goal to make something like the Arduino. So the end product would be end up with the following: * The hardware (I/O board). Think Arduino DUE (http://store.arduino.cc/index.php?main_page=product_info&cPath=11&products_id=243) * A compiler specific to the hardware (LCD with ARM Thumb backend, or GDC cross-compiled for arm-none-eabi * A port of the D runtime. Definitely not a complete port. Just enough to work the I/O and make use D programming constructs (classes, structs, exceptions, thread local storage, etc...) * A library, or set of libraries, to make programming the IO convenient. See the Arduino reference to get a an idea (http://arduino.cc/en/Reference/HomePage) * Possibly an IDE specific to microcontroller development. I'd like to eventually create a very tiny real-time OS like FreeRTOS(http://www.freertos.org/) and the like, but that is a very long-term goal. Other goals are: * Assist the LDC and GDC folks with supporting this platform, as this adventure truly stands on their shoulders and we're not going to get anywhere without them. * Show that the D language can replace C/C++ in the 32-bit MCU realm. Timo Sintonen is also well on his way to an ARM Cortex-M port (https://bitbucket.org/timosi/minlibd), but he's taking quite a different approach than I. I'm taking more of a bottom up approach and hope to discard the peripheral library, the C library, and anything else C/C++ and do everything in D. I've created a simple "hello world" tutorial here (http://wiki.dlang.org/Extremely_minimal_semihosted_%22Hello_World%22) so people interested in this or similar endeavors can have a convenient place to start. At the moment, I don't have any source code to publish, as everything I've built so far is simply for academic purposes (I have a lot to learn). But, I'd be happy to share what I have and what I've learned, and I hope in the coming months I'll have a repository with something useful. Mike

Great, nice to hear about a pure D approach to embedded. :) Keep us updated on your progress. I suggest that you publish your work-in-progress patches to a public repo somewhere, as I've been doing with my Android/x86 port, so that those interested can follow your work.
Jan 05 2014
prev sibling next sibling parent reply "Dwhatever" <not real.com> writes:
I'm trying to do the same, trying to compile OS free code but I 
haven't so far been successful because D requires the runtime and 
then also Phobos. Compared to C/C++ where you can create pretty 
advanced stand alone code without even include any standard 
libraries, this because much of C++ is part of the compiler.

With D this is not the case. I don't really have a good picture 
what is part of the compiler or the runtime. Even creating a 
simple sample class and create a stack object requires that I 
include *everything*, that hairy runtime phobos nest. The runtime 
also has dependencies into Phobos which I find to be very 
inconvenient and I don't think it is a good idea. The runtime 
should be stand alone and Phobos should depended on the runtime 
and not vice versa.

Also, I am not too happy about the change "scope for allocating 
classes on the stack", 
http://dlang.org/deprecate.html#scope%20for%20allocating%20classes%20on%20the%20stack.

This change requires you to include std.typecons from Phobos, 
which means you have include a library for a simple operation 
like stack allocation. C++ does not require this as it is a part 
of the language.

Compared to C/C++, D is very hard to get to work in the 
embedded/OS less environment because of this.
Jan 06 2014
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/6/14 4:22 AM, Mike wrote:
 On Monday, 6 January 2014 at 11:47:57 UTC, Dwhatever wrote:
 I'm trying to do the same, trying to compile OS free code but I
 haven't so far been successful because D requires the runtime and then
 also Phobos. Compared to C/C++ where you can create pretty advanced
 stand alone code without even include any standard libraries, this
 because much of C++ is part of the compiler.

I created a bare-metal freestanding (OS Free) hello world program and wrote a wiki about it here (http://wiki.dlang.org/Extremely_minimal_semihosted_%22Hello_World%22)

Sounds like a good topic for a DConf talk. You may want to consider submitting one. Andrei
Jan 06 2014
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2014-01-07 05:03, Sean Kelly wrote:

 Those wikis exist (albeit somewhat outdated) but I think they're still
 on Dsource. Is the site still online?

Yes, Dsource is still online. -- /Jacob Carlborg
Jan 07 2014
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2014-01-07 09:20, Mike wrote:

 I think it's OK to emit code that is not used, as long as the linker can
 safely strip it out if it can't find a path to it.

With the ModuleInfo and ClassInfo you can do some form of runtime reflection: auto foo = Object.create("bar.Foo"); How would the compiler know that bar.Foo is actually used anywhere in the code? -- /Jacob Carlborg
Jan 07 2014
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/6/2014 3:47 AM, Dwhatever wrote:
 Compared to C/C++, D is very hard to get to work in the embedded/OS less
 environment because of this.

It's not that hard. Use the -betterC switch to prevent ModuleInfo records and their dependencies from being emitted, and then you can write D code that has zero dependence on druntime or phobos. I've used this to port D to new platforms that have no druntime or phobos, because I need a working & tested compiler to compile those two.
Jan 07 2014
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 1/7/2014 6:22 PM, Adam D. Ruppe wrote:
 Maybe -betterC can stop outputting those references too and then we'd really be
 cooking.

-betterC does sometimes suffer from "bit rot" as a lot of people work on the front end and inadvertently subvert it. Nevertheless, the idea is sound, and it works, and it doesn't seem to have taken you long to figure it out :-)
Jan 07 2014
prev sibling next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
I took the liberty:

https://d.puremagic.com/issues/show_bug.cgi?id=11881
Jan 07 2014
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2014-01-09 19:39, Sean Kelly wrote:

 I do think we may need to stick with manually written headers though,
 as much for copyright reasons as anything.

Why not an automatic solution? Why would the copyright matter if it's manually or automatically translated?
 The old wiki pages are here:

 http://dsource.org/projects/druntime

Here's a list of runtime hooks: http://wiki.dlang.org/Runtime_Hooks -- /Jacob Carlborg
Jan 09 2014
parent reply Jacob Carlborg <doob me.com> writes:
On 2014-01-10 01:02, Sean Kelly wrote:

 Because of this clause from the Boost license page:

 "The conceptual interface to a library isn't covered. The
 particular representation expressed in the header is covered, as
 is the documentation, examples, test programs, and all the other
 material that goes with the library. A different implementation
 is free to use the same logical interface, however. Interface
 issues have been fought out in court several times; ask a lawyer
 for details."

 I suspect that an automatic translation might be subject to the
 "representation" issue, while a manual rewrite should not.

I see, that sucks :(. I guess this is a fine line since you very well could en up with the same result of a manual and automatic translation. -- /Jacob Carlborg
Jan 09 2014
parent Jacob Carlborg <doob me.com> writes:
On 2014-01-10 19:35, Sean Kelly wrote:

 Yeah, this is also one reason why I chose the header format I
 did.  It works directly from the spec and so imposes a structure
 of its own rather than inviting direct ports of existing headers.
    But I'm really just guessing here, since I don't have any legal
 background.

 One issue I'm not sure about regarding automatic translation
 (assuming this is deemed acceptable).  Say you're trying to
 translate pthread.h on a Posix system.  The declarations for that
 file might be required to live in sys/types.h, but in actuality
 live in a smattering of files in other locations (usually
 bits/whatever) all selected by a pretty complex chain of
 preprocessor conditions.  If we were to do an automatic
 translation, how would this all work?  One thing I like about the
 current approach is that the result is succinct and legible,
 despite requiring more work to generate.

Yeah, I know. That's one of my big problems with DStep, what to do about includes. Many types are hidden behind a typdefs to make it cross-platform. The actual type is usually in an internal architecture specific header file. The advantage would be to have all the declarations automatically translated but then manually put in the correct files. It does not necessarily need to be, run one command and it's all translated. Automatic translation could help even if it's not completely automatic. Just to avoid any manual errors. -- /Jacob Carlborg
Jan 10 2014
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2014-01-08 02:57, Adam D. Ruppe wrote:

 But put a single struct and it complains
 Error: TypeInfo not found. object.d may be incorrectly installed or
 corrupt, compile with -v switch


 even with -betterC :-( :-( :-(

Even if the struct isn't used or void-initialized? -- /Jacob Carlborg
Jan 07 2014
parent Jacob Carlborg <doob me.com> writes:
On 2014-01-08 16:25, Johannes Pfau wrote:

 The compiler can not know that a struct isn't used/void initialized.
 With separate compilation TypeInfo is output in object A.o and could be
 used from any other object. So only the linker actually knows for sure
 if the TypeInfo is used (There are exceptions where even the linker
 can't know that. Think of dynamic libraries).

Right, separate compilation always ruins the day :( -- /Jacob Carlborg
Jan 09 2014
prev sibling next sibling parent "Mike" <none none.com> writes:
On Monday, 6 January 2014 at 11:47:57 UTC, Dwhatever wrote:
 I'm trying to do the same, trying to compile OS free code but I 
 haven't so far been successful because D requires the runtime 
 and then also Phobos. Compared to C/C++ where you can create 
 pretty advanced stand alone code without even include any 
 standard libraries, this because much of C++ is part of the 
 compiler.

I created a bare-metal freestanding (OS Free) hello world program and wrote a wiki about it here (http://wiki.dlang.org/Extremely_minimal_semihosted_%22Hello_World%22) The problem is that there's not really much D in that D program, but it is a working foundation on which to build. If you plan on writing D code for embedded systems, you probably don't want all of the runtime anyway (at least I don't). Using GDC compiled for arm-none-eabi, I created a malloc/free implementation in D and am now able to create classes that are allocated on the heap with "new" and deallocated using "destroy". I also created a reference counted implementation too that automatically deallocates when the reference count reaches 0. But, I'm still struggling with the memory management as a whole. Unfortunately once I started creating structs and classes I had to implement a bunch of "TypeInfo.." stuff in object.d even though it appears none of it gets used. My object.d is now only about 140 lines. This was only possible with a recent change to GDC (http://forum.dlang.org/post/mailman.3.1387377012.2938.d.gnu puremagic.com) and the -fno-emit-moduleinfo switch. I tried with LDC too, but LDC requires the world, so I gave up for now. If any compiler implementers are reading this, please try to generate only code that is used by the program being compiled. I need to get this sorted out and submit some bug/enhancement requests to the compiler writers, but I first need to understand things better so I can articulate it well.
 With D this is not the case. I don't really have a good picture 
 what is part of the compiler or the runtime. Even creating a 
 simple sample class and create a stack object requires that I 
 include *everything*, that hairy runtime phobos nest. The 
 runtime also has dependencies into Phobos which I find to be 
 very inconvenient and I don't think it is a good idea. The 
 runtime should be stand alone and Phobos should depended on the 
 runtime and not vice versa.

I sympathize and agree. I can't see the abstraction in the phobos/runtime hairball. That's why I'm throwing the baby out with the bathwater and making a new baby. I also wish the runtime wasn't so coupled to the garbage collector. The garbage collector is just one of several ways to manage memory, and I don't think the runtime should be so dependent on it, but that's probably easier said than done.
 Compared to C/C++, D is very hard to get to work in the 
 embedded/OS less environment because of this.

I agree, but it won't get any better until you and I step up and do something about it.
Jan 06 2014
prev sibling next sibling parent "David Nadlinger" <code klickverbot.at> writes:
On Monday, 6 January 2014 at 12:22:37 UTC, Mike wrote:
 If any compiler implementers are reading this, please try to 
 generate only code that is used by the program being compiled.  
 I need to get this sorted out and submit some bug/enhancement 
 requests to the compiler writers, but I first need to 
 understand things better so I can articulate it well.

The issue with ModuleInfo is that you can actually query the list of all loaded D modules at runtime (ModuleInfo.opApply(), mainly for things like running static constructors, unit tests, Object.factory, …). So, even if your module is not using static constructors or unit tests, you can't just not emit the ModuleInfo in general. In theory, LDC allows you to selectively disable ModuleInfo generation for some modules (http://wiki.dlang.org/LDC-specific_language_changes), but it looks like the pragmas actually don't have any effect in current LDC2 builds (https://github.com/ldc-developers/ldc/issues/571). David
Jan 06 2014
prev sibling next sibling parent "Sean Kelly" <sean invisibleduck.org> writes:
On Monday, 9 April 2012 at 10:19:47 UTC, Iain Buclaw wrote:
 Personally I feel that people porting to specific architectures 
 should maintain their differences in separate files under a 
 /ports directory structure - lets say core.stdc.stdio as a cod 
 example. The version for bionic would be under 
 /ports/bionic/core/stdc/stdio.d, and that is the module that 
 gets compiled into the library when building for bionic. When 
 installing, the build process generates a header file of the 
 bionic version of core.stdc.stdio and puts the file in the 
 correct /include/core/stdc/stdio.di location.

This would really be fine. I'd even say just put them in core.sys.bionic or whatever, except that this location is for platform-specific imports and so core.sys.bionic.stdio could contain nonstandard extensions, while anything exposed by core.stdc should not. For all of these issues, bugzilla tickets and/or pull requests are much appreciated. I know that the design I chose for my own ease of maintenance may not scale past the handful of platforms DMD targets.
Jan 06 2014
prev sibling next sibling parent "Sean Kelly" <sean invisibleduck.org> writes:
On Monday, 6 January 2014 at 11:47:57 UTC, Dwhatever wrote:
 I'm trying to do the same, trying to compile OS free code but I 
 haven't so far been successful because D requires the runtime 
 and then also Phobos.

D doesn't require Phobos. The compiler implicitly links against libphobos.a, but that isn't the same thing. All a D app actually needs is Druntime. And you can replace the GC in Druntime with gcstub (backed by malloc) and stub out the thread API if these aren't appropriate for your target. This still means using a custom Druntime, but making the changes should really be pretty straightforward in most cases. The most complicated aspect of porting will be how you handle core.stdc, since Druntime still relies on C library calls for various things.
Jan 06 2014
prev sibling next sibling parent Iain Buclaw <ibuclaw gdcproject.org> writes:
--047d7bf0e03a4cee2904ef5175de
Content-Type: text/plain; charset=ISO-8859-1

On 6 Jan 2014 17:55, "Sean Kelly" <sean invisibleduck.org> wrote:
 On Monday, 6 January 2014 at 11:47:57 UTC, Dwhatever wrote:
 I'm trying to do the same, trying to compile OS free code but I haven't


 D doesn't require Phobos.  The compiler implicitly links against
 libphobos.a, but that isn't the same thing.  All a D app actually
 needs is Druntime.  And you can replace the GC in Druntime with
 gcstub (backed by malloc) and stub out the thread API if these
 aren't appropriate for your target.

GDC provides a configure flag to compile in gcstub instead of the standard GC. There's also a pthread inspired thread API (gcc.gthreads) that provides a common interface to the platform-specific implementation - not all gcc supported thread models have been ported in yet (only posix and win32) though on bare metal you may just want to compile with thread model=single. --047d7bf0e03a4cee2904ef5175de Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable <p dir=3D"ltr">On 6 Jan 2014 17:55, &quot;Sean Kelly&quot; &lt;<a href=3D"m= ailto:sean invisibleduck.org">sean invisibleduck.org</a>&gt; wrote:<br> &gt;<br> &gt; On Monday, 6 January 2014 at 11:47:57 UTC, Dwhatever wrote:<br> &gt;&gt;<br> &gt;&gt; I&#39;m trying to do the same, trying to compile OS free code but = I haven&#39;t so far been successful because D requires the runtime and the= n also Phobos.<br> &gt;<br> &gt;<br> &gt; D doesn&#39;t require Phobos. =A0The compiler implicitly links against= <br> &gt; libphobos.a, but that isn&#39;t the same thing. =A0All a D app actuall= y<br> &gt; needs is Druntime. =A0And you can replace the GC in Druntime with<br> &gt; gcstub (backed by malloc) and stub out the thread API if these<br> &gt; aren&#39;t appropriate for your target.</p> <p dir=3D"ltr">GDC provides a configure flag to compile in gcstub instead o= f the standard GC.</p> <p dir=3D"ltr">There&#39;s also a pthread inspired thread API (gcc.gthreads= ) that provides a common interface to the platform-specific implementation = - not all gcc supported thread models have been ported in yet (only posix a= nd win32) though on bare metal you may just want to compile with thread mod= el=3Dsingle.</p> --047d7bf0e03a4cee2904ef5175de--
Jan 06 2014
prev sibling next sibling parent "Dwhatever" <not real.com> writes:
On Monday, 6 January 2014 at 17:52:44 UTC, Sean Kelly wrote:
 On Monday, 6 January 2014 at 11:47:57 UTC, Dwhatever wrote:
 I'm trying to do the same, trying to compile OS free code but 
 I haven't so far been successful because D requires the 
 runtime and then also Phobos.

D doesn't require Phobos. The compiler implicitly links against libphobos.a, but that isn't the same thing. All a D app actually needs is Druntime. And you can replace the GC in Druntime with gcstub (backed by malloc) and stub out the thread API if these aren't appropriate for your target. This still means using a custom Druntime, but making the changes should really be pretty straightforward in most cases. The most complicated aspect of porting will be how you handle core.stdc, since Druntime still relies on C library calls for various things.

I'm using LDC and unfortunately the -mtriple=arm-eabi isn't supported so I have to use -mtriple=arm-linux which automatically sets the "Posix" version string. I can always try to stub all the Posix calls in the library but what I think would work is that you have a stubbed version when there isn't any system name like Windows or Posix. For example. version (Windows) { ... } else version (Posix) { ... } else { //Stubbed interface } However, it isn't really obvious what is missing when stubbing all the interfaces. Perhaps some stubbed interfaces should have a reference to some function so that the programmer understands that this must be implemented. For example malloc and free would be obvious.
Jan 06 2014
prev sibling next sibling parent "Mike" <none none.com> writes:
On Monday, 6 January 2014 at 17:04:57 UTC, Sean Kelly wrote:
 On Monday, 9 April 2012 at 10:19:47 UTC, Iain Buclaw wrote:
 Personally I feel that people porting to specific 
 architectures should maintain their differences in separate 
 files under a /ports directory structure - lets say 
 core.stdc.stdio as a cod example. The version for bionic would 
 be under /ports/bionic/core/stdc/stdio.d, and that is the 
 module that gets compiled into the library when building for 
 bionic. When installing, the build process generates a header 
 file of the bionic version of core.stdc.stdio and puts the 
 file in the correct /include/core/stdc/stdio.di location.

This would really be fine. I'd even say just put them in core.sys.bionic or whatever, except that this location is for platform-specific imports and so core.sys.bionic.stdio could contain nonstandard extensions, while anything exposed by core.stdc should not. For all of these issues, bugzilla tickets and/or pull requests are much appreciated. I know that the design I chose for my own ease of maintenance may not scale past the handful of platforms DMD targets.

I would submit tickets requests for these things, but in order to create a meaningful ticket that can be acted upon, one must know precisely what should be done, and articulate that in the ticket. But in order to understand what must be done, one must understand the current organization of the runtime. But in order to understand the current organization of the runtime, one needs to be able to see the abstraction. But the crux of this problem is that one can't see the abstraction because of the way the runtime is organized (catch 22). The same goes for pull requests. Those already familiar with the runtime could accelerate contribution from others by 1) documenting the current abstraction on the wiki or 2) following up on https://d.puremagic.com/issues/show_bug.cgi?id=11666 so one can see the abstraction. Finally the original runtime authors and porters could both begin writing a porting guide.
Jan 06 2014
prev sibling next sibling parent "Mike" <none none.com> writes:
On Monday, 6 January 2014 at 17:52:44 UTC, Sean Kelly wrote:
 And you can replace the GC in Druntime with
 gcstub (backed by malloc) and stub out the thread API if these
 aren't appropriate for your target.  This still means using a
 custom Druntime, but making the changes should really be pretty
 straightforward in most case.

A wiki post desribing exactly this would be most helpful.
Jan 06 2014
prev sibling next sibling parent "Mike" <none none.com> writes:
On Monday, 6 January 2014 at 20:23:23 UTC, Dwhatever wrote:
 On Monday, 6 January 2014 at 17:52:44 UTC, Sean Kelly wrote:
 On Monday, 6 January 2014 at 11:47:57 UTC, Dwhatever wrote:

supported so I have to use -mtriple=arm-linux which automatically sets the "Posix" version string. I can always try to stub all the Posix calls in the library but what I think would work is that you have a stubbed version when there isn't any system name like Windows or Posix. For example. version (Windows) { ... } else version (Posix) { ... } else { //Stubbed interface }

For LDC with an ARM backend, you only need to compile with -march= and/or -mcpu= if you wish to compile for bare-metal. The version strings are listed here (http://dlang.org/version.html). So if I understand your objective, you would only need... else version(ARM_Thumb) // or version(ARM) if targeting Cortex-A and the like { ... } And it may actually need to look more like... version(X86) { version(Windows) { } else version(Linux) { } else { } } else version(ARM_Thumb) { ... } of course the hard part is filling in the (...). What CPU/MCU are you targeting? Are you building for bare-metal?
Jan 06 2014
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
I haven't really been following this thread, but I did a minimal 
D for x86 too

http://arsdnet.net/dcode/minimal.zip

you can compile it on linux or for bare metal. I got a fair chunk 
of the language working with my custom object.d, or you can back 
off and use less of the lang:

http://arsdnet.net/dcode/minimal.d


It might not work on the newest dmd, I did this some months ago. 
But it wasn't really too hard and much of it should be usable on 
arm too.
Jan 06 2014
prev sibling next sibling parent "Sean Kelly" <sean invisibleduck.org> writes:
On Tuesday, 7 January 2014 at 03:02:51 UTC, Adam D. Ruppe wrote:
 I haven't really been following this thread, but I did a 
 minimal D for x86 too

 http://arsdnet.net/dcode/minimal.zip

 you can compile it on linux or for bare metal. I got a fair 
 chunk of the language working with my custom object.d, or you 
 can back off and use less of the lang:

 http://arsdnet.net/dcode/minimal.d


 It might not work on the newest dmd, I did this some months 
 ago. But it wasn't really too hard and much of it should be 
 usable on arm too.

Those wikis exist (albeit somewhat outdated) but I think they're still on Dsource. Is the site still online?
Jan 06 2014
prev sibling next sibling parent "Mike" <none none.com> writes:
On Monday, 6 January 2014 at 16:25:08 UTC, David Nadlinger wrote:
 On Monday, 6 January 2014 at 12:22:37 UTC, Mike wrote:
 If any compiler implementers are reading this, please try to 
 generate only code that is used by the program being compiled.
  I need to get this sorted out and submit some bug/enhancement 
 requests to the compiler writers, but I first need to 
 understand things better so I can articulate it well.

The issue with ModuleInfo is that you can actually query the list of all loaded D modules at runtime (ModuleInfo.opApply(), mainly for things like running static constructors, unit tests, Object.factory, …). So, even if your module is not using static constructors or unit tests, you can't just not emit the ModuleInfo in general.

I think it's OK to emit code that is not used, as long as the linker can safely strip it out if it can't find a path to it. I'm still a novice with the GNU toolchain, but I believe ld does this this with --gc-sections in collusion with GCC's -ffunction-sections and -fdata-sections. A recent experiment with GDC showed that if unused symbols are stripped out by the linker, some of the used symbols get dislocated (at least that's what I think is happening). Discussion here (http://forum.dlang.org/post/wrekpqefiswstqhhrhoh forum.dlang.org). My experiments with LDC last month seemed require a huge amount snowballing implementations just to compile, when they were all just going to get stripped out by the linker with --gc-sections anyway. Again, I would like to submit enhancement/bug reports for these things, but I need to learn more about what's actually going on, and what's actually needed by D, so I my report is meaningful and actionable.
 In theory, LDC allows you to selectively disable ModuleInfo 
 generation for some modules 
 (http://wiki.dlang.org/LDC-specific_language_changes), but it 
 looks like the pragmas actually don't have any effect in 
 current LDC2 builds 
 (https://github.com/ldc-developers/ldc/issues/571).

 David

I actually wasn't aware of these features. These will be quite helpful when implemented. Thank you.
Jan 07 2014
prev sibling next sibling parent "Dwhatever" <not real.com> writes:
 For LDC with an ARM backend, you only need to compile with 
 -march= and/or -mcpu= if you wish to compile for bare-metal.

 The version strings are listed here 
 (http://dlang.org/version.html).  So if I understand your 
 objective, you would only need...

 else version(ARM_Thumb) // or version(ARM) if targeting 
 Cortex-A and the like
 {
    ...
 }

 And it may actually need to look more like...
 version(X86)
 {
     version(Windows)
     { }
     else version(Linux)
     { }
     else
     { }
 }
 else version(ARM_Thumb)
 {
     ...
 }

 of course the hard part is filling in the (...).

 What CPU/MCU are you targeting?  Are you building for 
 bare-metal?

Yes, for bare metal, no OS, nothing. I think I have to specify my target architecture and CPU otherwise the compiler cannot know that I'm cross compiling for ARM. -march= -mcpu= will not work for me. I'm trying to compile a simple stand alone object file. class TestClass { ubyte member; this(ubyte m) { member = m; } ubyte Get() { return member; } }; extern(C) void main() { // stack class scope test = new TestClass(0); // simple inline asm test __asm("mov r0,#1; mov r1,#2", "~{r0,r1}"); } This test should be simple enough I thought but it turned out that to compile and link this, D requires almost everything from the runtime. So the challenge is, compile and link the simple code above targeting ARM using LDC, no OS allowed.
Jan 07 2014
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Tuesday, 7 January 2014 at 21:52:54 UTC, Dwhatever wrote:
 So the challenge is, compile and link the simple code above 
 targeting ARM using LDC, no OS allowed.

My minimal.zip is x86 but should be able to compile that and might not be hard to port to arm. http://arsdnet.net/dcode/minimal.zip
Jan 07 2014
prev sibling next sibling parent "Mike" <none none.com> writes:
On Tuesday, 7 January 2014 at 21:52:54 UTC, Dwhatever wrote:
 What CPU/MCU are you targeting?  Are you building for 
 bare-metal?

Yes, for bare metal, no OS, nothing. I think I have to specify my target architecture and CPU otherwise the compiler cannot know that I'm cross compiling for ARM. -march= -mcpu= will not work for me. I'm trying to compile a simple stand alone object file. class TestClass { ubyte member; this(ubyte m) { member = m; } ubyte Get() { return member; } }; extern(C) void main() { // stack class scope test = new TestClass(0); // simple inline asm test __asm("mov r0,#1; mov r1,#2", "~{r0,r1}"); } This test should be simple enough I thought but it turned out that to compile and link this, D requires almost everything from the runtime. So the challenge is, compile and link the simple code above targeting ARM using LDC, no OS allowed.

First of all, there's no such thing as a class allocated on the stack in D. See here ( http://dlang.org/deprecate.html#scope for allocating classes on the stack). You have to go with structs if you want that feature, but there are other ways like Scoped(T), RefeCounted(T), scope(exit), etc... It feels as little half-ass to do things this way, but D is quite powerful, and an alternate, elegant solution is probably just waiting to be discovered. Anyway, once you add the word "class" or "struct" you release an avalanche of snowballing required runtime stuff that you are likely not even using, directly or indirectly, in your program. And if using --gc-sections in your linker, it all just gets stripped out in the end. I was trying to articulate that here (http://forum.dlang.org/post/zewevdmburppufkjxdje forum.dlang.org) IMO this is a problem with the compiler, not D, and LDC suffers from it much more than GDC. Iain Buclaw recently made a change to GDC (http://forum.dlang.org/post/mailman.3.1387377012.2938.d.gnu puremagic.com) that allowed me to reduce my object.d to 140 lines, and with an accompanying malloc and free, I was able to create classes. I don't know what the change was, but it was most beneficial (Thanks Iain!). I hope the LDC folks can do the same, but I probably need to file an enhancement request and make a case for it. I'm sorry I can't share my code at the moment; I haven't decided yet where to put it. I was studying Adam Ruppe's minimal X86 port last night and found it most useful (Thumbs Up!), but with a 2000 line object.d, it's stretching the word "minimal", at least when compared with C. But if you really want all the cool stuff D has to offer, this is probably what will be required in the end. All the compilers seem to have been written mostly with the PC in mind, and seem to expect too much of the D runtime accompanying the compiler, which is definitely very different from C/C++ where you only pay for what you use. It also seems some argue that if you aren't using the full runtime, you aren't using D, and they have a point, but only one that limits D's appeal. The compiler folks have been most helpful so far as I try to "go minimal" with D, but I hope they will see the benefit of an "only require what is actually used" approach. D is, however, different from C, and this may have some consequences as mentioned here ( http://forum.dlang.org/post/lagsn7$2lqc$1 digitalmars.com). In further constrast with C, much of the language is implemented in the runtime, as you previously alluded to. Even the switch...case statement seems to be, at least partially, implemented in the runtime (https://github.com/D-Programming-Language/druntime/blob/maste /src/rt/switch_.d). I think this is great, because this means we don't have to know how to build compilers to port the language to a given platform. And with a "pay as you go" compiler, this could make D suitable for even the tiniest of microcontrollers. Anyway, if you want to go minimal, go with GDC for now. I hope to file some enhancement requests with the compilers soon that will help with this very issue, but I don't know how interested they will be in addressing them. I do see progress being made, so I'm hopeful. Mike
Jan 07 2014
prev sibling next sibling parent Iain Buclaw <ibuclaw gdcproject.org> writes:
2014/1/8 Mike <none none.com>:
 On Tuesday, 7 January 2014 at 21:52:54 UTC, Dwhatever wrote:
 What CPU/MCU are you targeting?  Are you building for bare-metal?

Yes, for bare metal, no OS, nothing. I think I have to specify my target architecture and CPU otherwise the compiler cannot know that I'm cross compiling for ARM. -march= -mcpu= will not work for me. I'm trying to compile a simple stand alone object file. class TestClass { ubyte member; this(ubyte m) { member = m; } ubyte Get() { return member; } }; extern(C) void main() { // stack class scope test = new TestClass(0); // simple inline asm test __asm("mov r0,#1; mov r1,#2", "~{r0,r1}"); } This test should be simple enough I thought but it turned out that to compile and link this, D requires almost everything from the runtime. So the challenge is, compile and link the simple code above targeting ARM using LDC, no OS allowed.

First of all, there's no such thing as a class allocated on the stack in D. See here ( http://dlang.org/deprecate.html#scope for allocating classes on the stack). You have to go with structs if you want that feature, but there are other ways like Scoped(T), RefeCounted(T), scope(exit), etc... It feels as little half-ass to do things this way, but D is quite powerful, and an alternate, elegant solution is probably just waiting to be discovered.

Yeah, I'd like to block that proposal for deprecation until Scoped(T) doesn't rely on DMD-specific NRVO behaviour.
 Anyway, once you add the word "class" or "struct" you release an avalanche
 of snowballing required runtime stuff that you are likely not even using,
 directly or indirectly, in your program. And if using --gc-sections in your
 linker, it all just gets stripped out in the end. I was trying to articulate
 that here (http://forum.dlang.org/post/zewevdmburppufkjxdje forum.dlang.org)

If you use a struct, it generates typeinfo and initialiser symbols that go on the comdat on the basis that you *may* access the typeinfo at runtime. These are weak decls so the linker merges/removes duplicates. There's currently no solution in which the compiler decides whether to omit sending the symbols to object file, but this can be worked upon.
 IMO this is a problem with the compiler, not D, and LDC suffers from it much
 more than GDC. Iain Buclaw recently made a change to GDC
 (http://forum.dlang.org/post/mailman.3.1387377012.2938.d.gnu puremagic.com)
 that allowed me to reduce my object.d to 140 lines, and with an accompanying
 malloc and free, I was able to create classes. I don't know what the change
 was, but it was most beneficial (Thanks Iain!). I hope the LDC folks can do
 the same, but I probably need to file an enhancement request and make a case
 for it.

I could have sworn that I spoke to David about this a couple months back, and it was in LDC already - unless they have some rules as to whether a static array literal is loaded on the stack or heap...
 I'm sorry I can't share my code at the moment; I haven't decided yet where
 to put it.  I was studying Adam Ruppe's minimal X86 port last night and
 found it most useful (Thumbs Up!), but with a 2000 line object.d, it's
 stretching the word "minimal", at least when compared with C. But if you
 really want all the cool stuff D has to offer, this is probably what will be
 required in the end.

So it's only double the size of C runtime (crt.o, crtend.o :-) I think the best logical steps to go down, is that you should write a replacement for the core library functions that the compiler implicitly calls (_d_arrayliteralX, _d_arraycopy, _d_newclass, _d_newitemT, etc), but omit using the TypeInfo parameter. Once you feel that it is ready, then we can add a switch into the compiler that: 1) Doesn't generate typeinfo 2) Passes a null pointer as the typeinfo parameter to the Druntime library calls. After testing and verifying it's all going sound, I'd be happy to omit the typeinfo altogether from the compiler side. However, before we get there, this will have some interesting hurdles to go over, for instance: Object _d_newclass(const ClassInfo ci) -> Object _d_newclass() Means that: 1) There's no way to tell the GC what we are allocating (eg: classes with no pointer fields require no scanning). However being bare bones, you probably won't mind this. There is no stop-the-world GC in the background. 2) The new'd class is not default initialised _d_newclass (no typeinfo, no known initialiser). So either you accept as a reasonable trade-off that there's no default initialisation of classes in bare-bones, or we alter the compiler codegen to take care of the default initialisation *after* calling _d_newclass. Others will have similar problems to consider.
Jan 07 2014
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Wednesday, 8 January 2014 at 00:21:15 UTC, Mike wrote:
 I'm sorry I can't share my code at the moment; I haven't 
 decided yet where to put it.  I was studying Adam Ruppe's 
 minimal X86 port last night and found it most useful (Thumbs 
 Up!), but with a 2000 line object.d, it's stretching the word 
 "minimal", at least when compared with C. But if you really 
 want all the cool stuff D has to offer, this is probably what 
 will be required in the end.

Sorry if I'm late to reply, I'm barely following this thread, just poking in when I see me mentioned :) But I started with a truly minimal D: the standalone file here was my first go <http://arsdnet.net/dcode/minimal.d> but indeed, that hits limits fast. I'd be ok with it as a "better C"... and the 3KB statically linked elf executable was awesome, but not even structs work, which takes a lot of fun out of things. So I started adding stuff. Note that about 500 lines of the object.d in there aren't strictly needed, but since I had a custom druntime, I wanted to play a bit :) Some things are needed so the compiler doesn't complain. Some stuff, typeinfo especially, are there so the linker doesn't complain. Then classes and exceptions were a lot of code (which I copy/pasted from druntime). About line 900 - line 2000 are these. You could cut a *lot* of that out if you didn't want to use exceptions, but I like them so I wanted that to work. But exceptions need stack unwinding (BTW gdc uses a totally different stack unwinding thing, at least on ARM/raspberry pi. This is dmd's code) and classes so it kinda ballooned the size a bit. I just fixed this to work with newer dmd btw. http://arsdnet.net/dcode/minimal.zip The executable was ~30 KB statically linked, and a good majority of the language worked. So I was pretty happy with that trade off. A lot of lines, but not a huge amount of bloat. Anywho, let me get it back to the whole "minimal" thing.... But put a single struct and it complains Error: TypeInfo not found. object.d may be incorrectly installed or corrupt, compile with -v switch even with -betterC :-( :-( :-( I don't even care about typeinfo; doing my own runtime, I'd be ok with any function that needs it to simply fail to compile. I can do it myself with templates. Basic structs should really work regardless.
 Even the switch...case statement seems to be, at least 
 partially, implemented in the runtime

Yea, for strings. My thing did a stupid loop for string cases :P If you don't try to use a string switch though, those references aren't emitted at all.
Jan 07 2014
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Wednesday, 8 January 2014 at 01:27:28 UTC, Walter Bright wrote:
 It's not that hard. Use the -betterC switch to prevent 
 ModuleInfo records and their dependencies from being emitted, 
 and then you can write D code that has zero dependence on 
 druntime or phobos.

...until you declare a struct. I find the -betterC switch to be totally useless. If you are just toying, there's no module info anyway: extern(C) { void _d_run_main() {} // why is this required now???? void* _d_dso_registry; void _start() { } } dmd -c *.d -defaultlib= -debuglib= gcc -m32 *.o -nostdlib success. Add "struct Test{}" and: Error: ModuleInfo not found. object.d may be incorrectly installed or corrupt. OK, add -betterC: Error: TypeInfo not found. object.d may be incorrectly installed or corrupt, compile with -v switch If you do use an object.d with it defined to get past the compiler, it doesn't help much as the linker complains anyway: test.o:(.data._D20TypeInfo_S4test4Test6__initZ+0x0): undefined reference to `_D15TypeInfo_Struct6__vtblZ' So my verdict is -betterC is pretty useless. If it got past the TypeInfo stuff the same way it gets past ModuleInfo though, then we'd be cooking with gas. That potentially gives a language that is legitimately a better C. But without structs, bah.
Jan 07 2014
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Wednesday, 8 January 2014 at 02:03:57 UTC, Adam D. Ruppe wrote:
 OK, add -betterC:

 Error: TypeInfo not found. object.d may be incorrectly 
 installed or corrupt, compile with -v switch

This is a filthy hack, but it is easy to change this in the compiler. (Isn't -betterC a filthy hack in the first place anyway :) ) In Expression *Type::getTypeInfo(Scope *sc) Add if(global.params.betterC) return NULL; right to the top. And same for struct decl void TypeInfoStructDeclaration::toDt(dt_t **pdt) { if(global.params.betterC) return; Then it compiles with the struct! ~5 KB executable, two line object.d. ....but adding some methods to the struct complain about missing ModuleInfo again. Looks like it wants to do assert(this !is null); and the assert thing takes a module info reference. Ugh.... that said, adding: void _d_assert_msg() {} void _d_assertm() {} void* _D4test12__ModuleInfoZ; to my extern(C) boilerplate did manage to work - structs with postblits and dtors yay! - and trimmed the exe down to 1.3K. Maybe -betterC can stop outputting those references too and then we'd really be cooking.
Jan 07 2014
prev sibling next sibling parent "Mike" <none none.com> writes:
On Wednesday, 8 January 2014 at 01:27:28 UTC, Walter Bright wrote:
 I've used this to port D to new platforms that have no druntime 
 or phobos, because I need a working & tested compiler to 
 compile those two.

But DMD doesn't support ARM in any flavor (Cortex-A nor Cortex-M - thumb), and that is the platform Dwhatever and myself are working on. Have you tried this with LDC and GDC? These compilers are quickly improving, but their current in their current state, it's a very different story when compared with C/C++.
 It's not that hard. Use the -betterC switch to prevent 
 ModuleInfo records and their dependencies from being emitted, 
 and then you can write D code that has zero dependence on 
 druntime or phobos.

As the creator of D and a significant author of the runtime, I think you are taking a lot for granted. I think from your perpective, it probably isn't all that hard, and certainly as I study D and the runtime, it is becoming significantly easier. But in comparison with C, Dwhatever is right... it's a bear, but I don't necessarily believe that that's bad. Mostly, it is due to the current expectations of the LDC and GDC compiler. Currently they require a large part of the runtime, that has no hope of every being called, just to get the simplest thing to compile. GDC has made significant improvement here recently, and I hope LDC will join the trend. There's some very constructive discussion going on about this right now in other parts of this thread. Furthtermore, the way the D Runtime is organizized, it's quite difficult to see the abstractions (if there are any). Putting some weight behind this issue (https://d.puremagic.com/issues/show_bug.cgi?id=11666) would certainly help. The origin of this thread was really about a porting guide; an excellent idea! You can wait several months for me learn D and the runtime and I'll create one, or you and the other D Runtime authors can create one and show us just how easy it is. A simple wiki post like this (http://wiki.osdev.org/Porting_Newlib) would be an excellent start. That being said, Thanks for D. I'm quite excited about this language. Mike
Jan 07 2014
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Wednesday, 8 January 2014 at 02:54:40 UTC, Mike wrote:
 But DMD doesn't support ARM in any flavor (Cortex-A nor 
 Cortex-M - thumb), and that is the platform Dwhatever and 
 myself are working on.  Have you tried this with LDC and GDC?

The switches should still work - the stuff we're talking about here is done with the frontend, which is (mostly) shared across all three compilers.
Jan 07 2014
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Jan 08, 2014 at 01:57:21AM +0000, Adam D. Ruppe wrote:
[...]
 Anywho, let me get it back to the whole "minimal" thing....
 
 But put a single struct and it complains
 Error: TypeInfo not found. object.d may be incorrectly installed or
 corrupt, compile with -v switch
 
 
 even with -betterC :-( :-( :-(
 
 
 I don't even care about typeinfo; doing my own runtime, I'd be ok
 with any function that needs it to simply fail to compile. I can do
 it myself with templates.

Couldn't you create some TypeInfo's with stubbed-out methods? Or does that not work either? [...]
Even the switch...case statement seems to be, at least partially,
implemented in the runtime

Yea, for strings. My thing did a stupid loop for string cases :P

Isn't that what the druntime string switch function does too? :-P Last I checked, it was also using a linear search through the cases. T -- There are 10 kinds of people in the world: those who can count in binary, and those who can't.
Jan 07 2014
prev sibling next sibling parent "Mike" <none none.com> writes:
On Wednesday, 8 January 2014 at 01:15:45 UTC, Iain Buclaw wrote:
 Anyway, once you add the word "class" or "struct" you release 
 an avalanche
 of snowballing required runtime stuff that you are likely not 
 even using,
 directly or indirectly, in your program. And if using 
 --gc-sections in your
 linker, it all just gets stripped out in the end. I was trying 
 to articulate
 that here 
 (http://forum.dlang.org/post/zewevdmburppufkjxdje forum.dlang.org)

If you use a struct, it generates typeinfo and initialiser symbols that go on the comdat on the basis that you *may* access the typeinfo at runtime. These are weak decls so the linker merges/removes duplicates. There's currently no solution in which the compiler decides whether to omit sending the symbols to object file, but this can be worked upon.

Thanks Iain, things are starting to make sense. Can't the compiler just emit whatever the runtime provides for the TypeInfo stuff, right or wrong? So, if the runtime does not provide a TypeInfo_x implementation, it doesn't emit one. If the runtime does provide a TypeInfo_x, it emits exactly what the runtime provided, right or wrong. If the runtime only provides some TypeInfo_x implementations and not others, the compiler only emits the ones that it finds. If the program provides a faulty TypeInfo_x implementation, the program behaves in a faulty manner, just as the programmer specified. If the program references a missing TypeInfo_x (or one of it's properties) implicitly/indirectly, the linker will emit and undefined reference error, won't it? Is this implementation possible?
Jan 08 2014
prev sibling next sibling parent Iain Buclaw <ibuclaw gdcproject.org> writes:
On 8 January 2014 11:13, Mike <none none.com> wrote:
 On Wednesday, 8 January 2014 at 01:15:45 UTC, Iain Buclaw wrote:
 Anyway, once you add the word "class" or "struct" you release an
 avalanche
 of snowballing required runtime stuff that you are likely not even using,
 directly or indirectly, in your program. And if using --gc-sections in
 your
 linker, it all just gets stripped out in the end. I was trying to
 articulate
 that here
 (http://forum.dlang.org/post/zewevdmburppufkjxdje forum.dlang.org)

If you use a struct, it generates typeinfo and initialiser symbols that go on the comdat on the basis that you *may* access the typeinfo at runtime. These are weak decls so the linker merges/removes duplicates. There's currently no solution in which the compiler decides whether to omit sending the symbols to object file, but this can be worked upon.

Thanks Iain, things are starting to make sense. Can't the compiler just emit whatever the runtime provides for the TypeInfo stuff, right or wrong? So, if the runtime does not provide a TypeInfo_x implementation, it doesn't emit one. If the runtime does provide a TypeInfo_x, it emits exactly what the runtime provided, right or wrong. If the program provides a faulty TypeInfo_x implementation, the program behaves in a faulty manner, just as the programmer specified. If the program references a missing TypeInfo_x (or one of it's properties) implicitly/indirectly, the linker will emit and undefined reference error, won't it? Is this implementation possible?

Half and half. The TypeInfo support code is a separation layer between runtime and compiler and the compiler is not fully aware of what is implemented by runtime. On the one hand, there's the internal TypeInfo you see in object.d, these are required to be known at compile time for certain operations. If those aren't present then you'll get mismatch errors from the compiler, and in the case of GDC, a handy little ICE to end the compilation immediately. On the other hand there's the TypeInfo support under rt.typeinfo (eg: TypeInfo_AC). For these, only the symbol is generated by the compiler on the fly and referenced as 'extern'. So you get linker errors if they aren't implemented. This is why I suggested the 'all or nothing' approach.
Jan 08 2014
prev sibling next sibling parent Iain Buclaw <ibuclaw gdcproject.org> writes:
On 8 January 2014 12:13, Iain Buclaw <ibuclaw gdcproject.org> wrote:
 On 8 January 2014 11:13, Mike <none none.com> wrote:
 On Wednesday, 8 January 2014 at 01:15:45 UTC, Iain Buclaw wrote:
 Anyway, once you add the word "class" or "struct" you release an
 avalanche
 of snowballing required runtime stuff that you are likely not even using,
 directly or indirectly, in your program. And if using --gc-sections in
 your
 linker, it all just gets stripped out in the end. I was trying to
 articulate
 that here
 (http://forum.dlang.org/post/zewevdmburppufkjxdje forum.dlang.org)

If you use a struct, it generates typeinfo and initialiser symbols that go on the comdat on the basis that you *may* access the typeinfo at runtime. These are weak decls so the linker merges/removes duplicates. There's currently no solution in which the compiler decides whether to omit sending the symbols to object file, but this can be worked upon.

Thanks Iain, things are starting to make sense. Can't the compiler just emit whatever the runtime provides for the TypeInfo stuff, right or wrong? So, if the runtime does not provide a TypeInfo_x implementation, it doesn't emit one. If the runtime does provide a TypeInfo_x, it emits exactly what the runtime provided, right or wrong. If the program provides a faulty TypeInfo_x implementation, the program behaves in a faulty manner, just as the programmer specified. If the program references a missing TypeInfo_x (or one of it's properties) implicitly/indirectly, the linker will emit and undefined reference error, won't it? Is this implementation possible?

Half and half. The TypeInfo support code is a separation layer between runtime and compiler and the compiler is not fully aware of what is implemented by runtime. On the one hand, there's the internal TypeInfo you see in object.d, these are required to be known at compile time for certain operations. If those aren't present then you'll get mismatch errors from the compiler, and in the case of GDC, a handy little ICE to end the compilation immediately. On the other hand there's the TypeInfo support under rt.typeinfo (eg: TypeInfo_AC). For these, only the symbol is generated by the compiler on the fly and referenced as 'extern'. So you get linker errors if they aren't implemented. This is why I suggested the 'all or nothing' approach.

Note, the struct of the internal TypeInfo symbols are known at compile time because object.d is implicitly imported.
Jan 08 2014
prev sibling next sibling parent Iain Buclaw <ibuclaw gdcproject.org> writes:
On 8 January 2014 12:14, Iain Buclaw <ibuclaw gdcproject.org> wrote:
 On 8 January 2014 12:13, Iain Buclaw <ibuclaw gdcproject.org> wrote:
 On 8 January 2014 11:13, Mike <none none.com> wrote:
 On Wednesday, 8 January 2014 at 01:15:45 UTC, Iain Buclaw wrote:
 Anyway, once you add the word "class" or "struct" you release an
 avalanche
 of snowballing required runtime stuff that you are likely not even using,
 directly or indirectly, in your program. And if using --gc-sections in
 your
 linker, it all just gets stripped out in the end. I was trying to
 articulate
 that here
 (http://forum.dlang.org/post/zewevdmburppufkjxdje forum.dlang.org)

If you use a struct, it generates typeinfo and initialiser symbols that go on the comdat on the basis that you *may* access the typeinfo at runtime. These are weak decls so the linker merges/removes duplicates. There's currently no solution in which the compiler decides whether to omit sending the symbols to object file, but this can be worked upon.

Thanks Iain, things are starting to make sense. Can't the compiler just emit whatever the runtime provides for the TypeInfo stuff, right or wrong? So, if the runtime does not provide a TypeInfo_x implementation, it doesn't emit one. If the runtime does provide a TypeInfo_x, it emits exactly what the runtime provided, right or wrong. If the program provides a faulty TypeInfo_x implementation, the program behaves in a faulty manner, just as the programmer specified. If the program references a missing TypeInfo_x (or one of it's properties) implicitly/indirectly, the linker will emit and undefined reference error, won't it? Is this implementation possible?

Half and half. The TypeInfo support code is a separation layer between runtime and compiler and the compiler is not fully aware of what is implemented by runtime. On the one hand, there's the internal TypeInfo you see in object.d, these are required to be known at compile time for certain operations. If those aren't present then you'll get mismatch errors from the compiler, and in the case of GDC, a handy little ICE to end the compilation immediately. On the other hand there's the TypeInfo support under rt.typeinfo (eg: TypeInfo_AC). For these, only the symbol is generated by the compiler on the fly and referenced as 'extern'. So you get linker errors if they aren't implemented. This is why I suggested the 'all or nothing' approach.

Note, the struct of the internal TypeInfo symbols are known at compile time because object.d is implicitly imported.

s/struct/structure/
Jan 08 2014
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Wednesday, 8 January 2014 at 06:10:38 UTC, H. S. Teoh wrote:
 Couldn't you create some TypeInfo's with stubbed-out methods?

Not quite, see Iain's post, it is half and half. The compiler checks the runtime definition to ensure it is there and the correct size, so you can sorta stub it (my minimal.zip did a void*[16]) but it still takes some care. And the compiler outputs the data anyway, so you might as well define it correctly and actually use it.
Jan 08 2014
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Wednesday, 8 January 2014 at 07:57:29 UTC, Jacob Carlborg
wrote:
 Even if the struct isn't used or void-initialized?

Yeah.
Jan 08 2014
prev sibling next sibling parent Johannes Pfau <nospam example.com> writes:
Am Wed, 8 Jan 2014 01:15:32 +0000
schrieb Iain Buclaw <ibuclaw gdcproject.org>:

 
 If you use a struct, it generates typeinfo and initialiser symbols
 that go on the comdat on the basis that you *may* access the typeinfo
 at runtime.
 
 These are weak decls so the linker merges/removes duplicates.  There's
 currently no solution in which the compiler decides whether to omit
 sending the symbols to object file, but this can be worked upon.

I think there are three steps to make TypeInfo completely optional * -fno-typeinfo as a global switch instructing the compiler that it _never_ has typeinfo and should never output typeinfo. In this case all files must be compiled consistently with -fno-typeinfo or without it Useful to disable all TypeInfo generation even if the runtime has typeinfo support * pragma(notypeinfo)/ attribute(notypeinfo) as in LDC Useful to exclude typeinfo for specific types * A way for the runtime to say 'I don't support TypeInfo' Basically introducing 'core.config' with 'enum RuntimeSupportsTypeInfo = false;'. If the compiler finds this declaration it should automatically use '-fno-typeinfo' 'core.config' could then also be used with RuntimeSupportsGC, RuntimeSupportsUTF (switches over strings, IIRC) and similar things. A minimal runtime would just set all those variables to false and the compiler could automatically use a restricted set of D (nogc/no string switches/...)
 
 I think the best logical steps to go down, is that you should write a
 replacement for the core library functions that the compiler
 implicitly calls (_d_arrayliteralX, _d_arraycopy, _d_newclass,
 _d_newitemT, etc), but omit using the TypeInfo parameter.  Once you
 feel that it is ready, then we can add a switch into the compiler
 that:
 1) Doesn't generate typeinfo
 2) Passes a null pointer as the typeinfo parameter to the Druntime
 library calls.
 

 Object _d_newclass(const ClassInfo ci)  ->  Object _d_newclass()
 

making it a template). I think this came up when we talked about replacing 'new' with a library template or in a custom allocator discussion or something. There were already approaches making associative arrays a templated library type. Then we could just as well make normal arrays a templated library type and avoid typeinfo there. Runtime variadic arguments need typeinfo IIRC. We'd have to ban calling those functions with -fno-typeinfo or if any of the arguments was declared with attribute(notypeinfo)
Jan 08 2014
prev sibling next sibling parent Johannes Pfau <nospam example.com> writes:
Am Wed, 8 Jan 2014 12:13:33 +0000
schrieb Iain Buclaw <ibuclaw gdcproject.org>:

 On 8 January 2014 11:13, Mike <none none.com> wrote:
 Can't the compiler just emit whatever the runtime provides for the
 TypeInfo stuff, right or wrong?  So, if the runtime does not
 provide a TypeInfo_x implementation, it doesn't emit one.  If the
 runtime does provide a TypeInfo_x, it emits exactly what the
 runtime provided, right or wrong.

 If the program provides a faulty TypeInfo_x implementation, the
 program behaves in a faulty manner, just as the programmer
 specified.

 If the program references a missing TypeInfo_x (or one of it's
 properties) implicitly/indirectly, the linker will emit and
 undefined reference error, won't it?

 Is this implementation possible?

Half and half. The TypeInfo support code is a separation layer between runtime and compiler and the compiler is not fully aware of what is implemented by runtime. On the one hand, there's the internal TypeInfo you see in object.d, these are required to be known at compile time for certain operations. If those aren't present then you'll get mismatch errors from the compiler, and in the case of GDC, a handy little ICE to end the compilation immediately.

However, the compiler does not actually parse and use the definitions in object.d. It always uses it's own hardcoded definitions. So you can't simply remove fields from the TypeInfo classes and then expect the compiler to stop outputting the data for this field. Although the implementation could be changed to allow for that I don't think it's worth the trouble. I think if we can simply disable/enable TypeInfo that should be good enough. Having many different TypeInfo formats could bring it's own problems.
Jan 08 2014
prev sibling next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 07/01/14 03:50, Andrei Alexandrescu wrote:
 On 1/6/14 4:22 AM, Mike wrote:
 On Monday, 6 January 2014 at 11:47:57 UTC, Dwhatever wrote:
 I'm trying to do the same, trying to compile OS free code but I
 haven't so far been successful because D requires the runtime and then
 also Phobos. Compared to C/C++ where you can create pretty advanced
 stand alone code without even include any standard libraries, this
 because much of C++ is part of the compiler.

I created a bare-metal freestanding (OS Free) hello world program and wrote a wiki about it here (http://wiki.dlang.org/Extremely_minimal_semihosted_%22Hello_World%22)

Sounds like a good topic for a DConf talk. You may want to consider submitting one.

Yes, please. It's not my personal area of work, but I'd be _really_ interested in seeing a good talk on embedded/from the ground up D -- and it will certainly be of professional interest to friends and colleagues of mine.
Jan 08 2014
prev sibling next sibling parent Johannes Pfau <nospam example.com> writes:
Am Tue, 7 Jan 2014 22:08:54 -0800
schrieb "H. S. Teoh" <hsteoh quickfur.ath.cx>:

Even the switch...case statement seems to be, at least partially,
implemented in the runtime

Yea, for strings. My thing did a stupid loop for string cases :P

Isn't that what the druntime string switch function does too? :-P Last I checked, it was also using a linear search through the cases.

But IIRC druntime does some UTF normalization or something on strings or was that only for foreach over strings? D's 'UTF support' is a little to heavy for small runtimes. We need to make that opt-out for the runtime as I suggested in this reply: http://forum.dlang.org/thread/fdsovuxnffmnpqhfmpts forum.dlang.org?page=6#post-lajpgk:242jkc:241:40digitalmars.com
Jan 08 2014
prev sibling next sibling parent Johannes Pfau <nospam example.com> writes:
Am Wed, 08 Jan 2014 13:31:07 +0000
schrieb "Adam D. Ruppe" <destructionator gmail.com>:

 On Wednesday, 8 January 2014 at 07:57:29 UTC, Jacob Carlborg
 wrote:
 Even if the struct isn't used or void-initialized?

Yeah.

The compiler can not know that a struct isn't used/void initialized. With separate compilation TypeInfo is output in object A.o and could be used from any other object. So only the linker actually knows for sure if the TypeInfo is used (There are exceptions where even the linker can't know that. Think of dynamic libraries). There are 2 ways to solve this: * Disable TypeInfo globally, everywhere (useful for embedded systems) * Disable TypeInfo for specific types using some kind of annotation in the source code (pragma, UDA). This way the compiler knows if a type has TypeInfo or not and can complain if it'd need TypeInfo
Jan 08 2014
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Wednesday, 8 January 2014 at 15:05:56 UTC, Johannes Pfau wrote:
 IIRC Andrei wants _d_newclass to be independent of ClassInfo 
 anyway (by making it a template).

me too
 Runtime variadic arguments need typeinfo IIRC. We'd have to ban 
 calling those functions with -fno-typeinfo or if any of the 
 arguments

...you know, they don't strictly *need* the data. Most variadic args are checked by doing if args[1] == typeid(int)... which is just a magic number comparison. The actual classes and other info aren't needed. (Indeed, I think the classes aren't needed anyway since we can get that info through __traits anyway nowadays. In fact in my minimal.zip thing I have a template that makes typeinfo objects out of trait info to prove it.)
Jan 08 2014
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Wednesday, 8 January 2014 at 15:21:21 UTC, Johannes Pfau wrote:
 But IIRC druntime does some UTF normalization or something on 
 strings or was that only for foreach over strings?

No, all it does is if you ask for foreach(dchar c; string){}, it will decode the multi-byte sequences. This does not require druntime, the compiler just outputs the code for it. Decoding the multi byte sequences isn't much code, it is just some loads, shifts, and bit masks. Fairly lightweight.
Jan 08 2014
prev sibling next sibling parent Johannes Pfau <nospam example.com> writes:
Am Wed, 08 Jan 2014 15:59:25 +0000
schrieb "Adam D. Ruppe" <destructionator gmail.com>:

 On Wednesday, 8 January 2014 at 15:21:21 UTC, Johannes Pfau wrote:
 But IIRC druntime does some UTF normalization or something on 
 strings or was that only for foreach over strings?

No, all it does is if you ask for foreach(dchar c; string){}, it will decode the multi-byte sequences. This does not require druntime, the compiler just outputs the code for it.

Are all the aApplay methods deprecated then? See https://github.com/D-Programming-Language/druntime/blob/master/src/rt/aApply.d which also imports rt.util.utf.
 
 Decoding the multi byte sequences isn't much code, it is just 
 some loads, shifts, and bit masks. Fairly lightweight.

But it is _some_ code. It's always a matter of perspective, but on a 8bit AVR with 4-16KB for code you probably don't want to implement these functions in the runtime. Instead you just want the compiler to complain if someone does foreach(dchar, "string"). If this is really needed on small systems it's always possible to implement a range to do this: foreach(dchar x; decode("string")) Don't get me wrong, I think this is just fine in normal D, but in embedded D it would be nice if the runtime could disable that feature.
Jan 08 2014
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Wednesday, 8 January 2014 at 16:22:55 UTC, Johannes Pfau wrote:
 Are all the aApplay methods deprecated then? See
 https://github.com/D-Programming-Language/druntime/blob/master/src/rt/aApply.d

Oh, oops, you are right, my link command was pulling that in. Here's the correct thing: test.d:(.text._D4test9write_rawFxAaZv+0x39): undefined reference to `_aApplycd1'
 But it is _some_ code.

Yeah, and since you're right about it needing a runtime function, as long as you use the correct linker command the function will not be found. I like the link errors more than compile errors btw because that makes it easy to opt-in to these things if you do want them: simply implement the missing function.
Jan 08 2014
prev sibling next sibling parent "Dwhatever" <not real.com> writes:
My question is what do all these TypeInfo and ModuleInfo have to 
do with operating systems? Assuming that TypeInfo and ModuleInfo 
has really nothing to do with the OS, I see no problem with it. 
Sure it is nice to be able to remove it for certain embedded 
system where size is critical but for many embedded systems this 
is not necessary. Why remove key parts of the language when it is 
the OS dependency that is the problem?

My view is that there is no clear distinction what is independent 
of the operating system and what is operating system dependent in 
the runtime. What is part of the langauge and what is not?
Jan 08 2014
prev sibling next sibling parent "Mike" <none none.com> writes:
On Wednesday, 8 January 2014 at 15:05:56 UTC, Johannes Pfau wrote:
 I think there are three steps to make TypeInfo completely 
 optional

 * -fno-typeinfo as a global switch instructing the compiler 
 that it
   _never_ has typeinfo and should never output typeinfo. In 
 this case
   all files must be compiled consistently with -fno-typeinfo or 
 without
   it
   Useful to disable all TypeInfo generation even if the runtime 
 has
   typeinfo support
 * pragma(notypeinfo)/ attribute(notypeinfo) as in LDC
   Useful to exclude typeinfo for specific types
 * A way for the runtime to say 'I don't support TypeInfo'
   Basically introducing 'core.config' with 'enum
   RuntimeSupportsTypeInfo = false;'. If the compiler finds this
   declaration it should automatically use '-fno-typeinfo'

This has been an interesting discussion, and has cause me to come full circle with a different perspective. First, I want to dispell a potential myth: "If we don't elminate TypeInfo stuff we'll have code bloat" This is not true if compiling with -ffunction-section, -fdata-sections, and linking with --gc-sections. All the TypeInfo stuff will eventually just get stripped out by the linker, as long as it's not used by the program. However, the compilers will have to generated code that can be safely --gc'ed. GDC, at the moment, doesn't. I need to minimize my code and report an issue on this. It's first on my list, so please stand by. Discussion here (http://forum.dlang.org/post/wrekpqefiswstqhhrhoh forum.dlang.org). As long as the TypeInfo and other unused stuff can be safely --gc'd, D will be suitable for the tiniest of microcontrollers. "So what's the problem?", you might ask. The problem is really convenience. I want to make a simple 10-line "Hello World" with a struct containing 1 measly property, and I have to create 1000 lines of runtime stuff just to get it to compile. And worst of all, none of that runtime stuff has any hope of every being called, and --gc-sections discards it anyway. How unfortunate! Actually, for the aforementioned contrived example, it's only about 150 lines in GDC at the moment, but LDC appears to be much more (LDC folks, I'll make an issue report soon. It's second on my list). So, why would one want to make a "Hello Word" program with a single struct containing 1 measly property? Answer: to learn. But in the end, after one learns the runtime and adds all of the features they want, what will the end up with?... A 10,000 line runtime with all the TypeInfo stuff and more...and they'll be really happy with it. And worst of all, the -fno-typeinfo switch that they were using while they were learning, was removed from their makefile in the first month of study. "So, what's the REAL problem?", you might ask. Answer: Lack of information about the runtime. e.g. no porting guide and a tightly-coupled hairball of d runtime with very little modularity, and very little platform, hardware, and feature abstraction. I now think that parts of this current conversation would not even exist had this information existed. There doesn't seem to be much interest from the runtime folks in doing something about this, so I intend to. The porting guide is 3rd on my list. And it's going to stink, because I don't know what I'm doing yet. But maybe people irritated by the smell will help clean it up. I think the -fno-typeinfo switch and the other proposed changes will be quite useful for the student of D and the D runtime, but I think they will only be used temporarily in their initial study. I'm not voting against it, but I'm beginning to see it as not-so-important. Maybe others have different needs for it than I, and it may be useful to them. I'm quite thankful however, for the engligtening discussion, and the willingness to be so accomodating. Thank you!
 I think the best logical steps to go down, is that you should 
 write a
 replacement for the core library functions that the compiler
 implicitly calls (_d_arrayliteralX, _d_arraycopy, _d_newclass,
 _d_newitemT, etc), but omit using the TypeInfo parameter.  
 Once you
 feel that it is ready, then we can add a switch into the 
 compiler
 that:
 1) Doesn't generate typeinfo
 2) Passes a null pointer as the typeinfo parameter to the 
 Druntime
 library calls.
 

 Object _d_newclass(const ClassInfo ci)  ->  Object 
 _d_newclass()
 

anyway (by making it a template). I think this came up when we talked about replacing 'new' with a library template or in a custom allocator discussion or something.

A templated _d_newclass, and possibly other runtime hooks would help greatly with the tight-coupling of the runtime. It's not fourth on my list, yet, but it could use an issue report.
Jan 08 2014
prev sibling next sibling parent "Sean Kelly" <sean invisibleduck.org> writes:
On Wednesday, 8 January 2014 at 01:27:28 UTC, Walter Bright wrote:
 On 1/6/2014 3:47 AM, Dwhatever wrote:
 Compared to C/C++, D is very hard to get to work in the 
 embedded/OS less environment because of this.

It's not that hard. Use the -betterC switch to prevent ModuleInfo records and their dependencies from being emitted, and then you can write D code that has zero dependence on druntime or phobos.

-betterC? That flag isn't even documented. Though I suggest renaming it to something that doesn't reference a different language. -bare or -minimal or -noemit or something.
Jan 09 2014
prev sibling next sibling parent "Sean Kelly" <sean invisibleduck.org> writes:
On Wednesday, 8 January 2014 at 02:54:40 UTC, Mike wrote:
 On Wednesday, 8 January 2014 at 01:27:28 UTC, Walter Bright 
 wrote:

 It's not that hard. Use the -betterC switch to prevent 
 ModuleInfo records and their dependencies from being emitted, 
 and then you can write D code that has zero dependence on 
 druntime or phobos.

As the creator of D and a significant author of the runtime, I think you are taking a lot for granted. I think from your perpective, it probably isn't all that hard, and certainly as I study D and the runtime, it is becoming significantly easier. But in comparison with C, Dwhatever is right... it's a bear, but I don't necessarily believe that that's bad.

The runtime contains years of accumulated cruft, and it obviously biased towards the way DMD works. I'd personally like to see the "rt" portion at least get a thorough once-over along the lines of what you're doing to see if parts can be simplified or discarded, and to determine whether there are any hidden costs to using language features.
 Furthtermore, the way the D Runtime is organizized, it's quite 
 difficult to see the abstractions (if there are any). Putting 
 some weight behind this issue 
 (https://d.puremagic.com/issues/show_bug.cgi?id=11666) would 
 certainly help.

We'll have to rely mostly on input from GDC and LDC here. The current stdc header design is the one that was the easiest for me to maintain when I was the sole runtime developer and we were targeting a handful of platforms. That's clearly changed, but I don't know what workflow is ideal for the other compilers. I do think we may need to stick with manually written headers though, as much for copyright reasons as anything. That said, if the runtime at least were modified to not use the C library at all, we may eventually be able to detach it completely. I think we're probably already down to a small number of functions (memset, memcpy, and a few maths routines).
 The origin of this thread was really about a porting guide; an 
 excellent idea!  You can wait several months for me learn D and 
 the runtime and I'll create one, or you and the other D Runtime 
 authors can create one and show us just how easy it is.  A 
 simple wiki post like this 
 (http://wiki.osdev.org/Porting_Newlib) would be an excellent 
 start.

The old wiki pages are here: http://dsource.org/projects/druntime Also, this may still be relevant: http://dsource.org/projects/tango/wiki/ChapterC The information is pretty sparing, but it should provide a decent starting point and explain at least a bit of the rationale behind the current design. The most notable design change since these were written is that I decided it's reasonable for all portions of the runtime to import anything in "core", so the separate compilation aspect isn't quite as true as it once was. This also means that not every extern (C) function defined in, say, core.exception may actually be needed any more from a design perspective, though I still very much like having the GC interact with the thread library via the provided extern (C) thread_* routines. I think one could easily argue that if the runtime ever needs to create a Thread, it should do so via a proxy function instead of directly instantiating the class to retain ease of porting D to platforms where threads aren't desired.
Jan 09 2014
prev sibling next sibling parent "Sean Kelly" <sean invisibleduck.org> writes:
On Thursday, 9 January 2014 at 20:16:14 UTC, Jacob Carlborg wrote:
 On 2014-01-09 19:39, Sean Kelly wrote:

 I do think we may need to stick with manually written headers 
 though, as much for copyright reasons as anything.

Why not an automatic solution? Why would the copyright matter if it's manually or automatically translated?

Because of this clause from the Boost license page: "The conceptual interface to a library isn't covered. The particular representation expressed in the header is covered, as is the documentation, examples, test programs, and all the other material that goes with the library. A different implementation is free to use the same logical interface, however. Interface issues have been fought out in court several times; ask a lawyer for details." I suspect that an automatic translation might be subject to the "representation" issue, while a manual rewrite should not.
Jan 09 2014
prev sibling next sibling parent "Joakim" <joakim airpost.net> writes:
On Friday, 10 January 2014 at 00:02:22 UTC, Sean Kelly wrote:
 On Thursday, 9 January 2014 at 20:16:14 UTC, Jacob Carlborg 
 wrote:
 On 2014-01-09 19:39, Sean Kelly wrote:

 I do think we may need to stick with manually written headers 
 though, as much for copyright reasons as anything.

Why not an automatic solution? Why would the copyright matter if it's manually or automatically translated?

Because of this clause from the Boost license page: "The conceptual interface to a library isn't covered. The particular representation expressed in the header is covered, as is the documentation, examples, test programs, and all the other material that goes with the library. A different implementation is free to use the same logical interface, however. Interface issues have been fought out in court several times; ask a lawyer for details." I suspect that an automatic translation might be subject to the "representation" issue, while a manual rewrite should not.

IANAL and I agree that this may be a somewhat legally murky topic, but Android extensively uses automatically translated headers and I don't think it has caused them much of a problem. Well, other than that whole Java mess with Oracle, ;) which Google won. I count 632 header files in the platform headers that I'm using for Android/x86 that contain the following notice: *** This header was automatically generated from a Linux kernel header *** of the same name, to make information necessary for userspace to *** call into the kernel available to libc. It contains only constants, *** structures, and macros generated from the original header, and thus, *** contains no copyrightable information.
Jan 10 2014
prev sibling parent "Sean Kelly" <sean invisibleduck.org> writes:
On Friday, 10 January 2014 at 07:32:49 UTC, Jacob Carlborg wrote:
 On 2014-01-10 01:02, Sean Kelly wrote:

 Because of this clause from the Boost license page:

 "The conceptual interface to a library isn't covered. The
 particular representation expressed in the header is covered, 
 as
 is the documentation, examples, test programs, and all the 
 other
 material that goes with the library. A different implementation
 is free to use the same logical interface, however. Interface
 issues have been fought out in court several times; ask a 
 lawyer
 for details."

 I suspect that an automatic translation might be subject to the
 "representation" issue, while a manual rewrite should not.

I see, that sucks :(. I guess this is a fine line since you very well could en up with the same result of a manual and automatic translation.

Yeah, this is also one reason why I chose the header format I did. It works directly from the spec and so imposes a structure of its own rather than inviting direct ports of existing headers. But I'm really just guessing here, since I don't have any legal background. One issue I'm not sure about regarding automatic translation (assuming this is deemed acceptable). Say you're trying to translate pthread.h on a Posix system. The declarations for that file might be required to live in sys/types.h, but in actuality live in a smattering of files in other locations (usually bits/whatever) all selected by a pretty complex chain of preprocessor conditions. If we were to do an automatic translation, how would this all work? One thing I like about the current approach is that the result is succinct and legible, despite requiring more work to generate.
Jan 10 2014