www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - Linker-hacking out the D runtime

reply sarn <sarn theartofmachinery.com> writes:
As it stands, the -betterC flag is still immature and only 
removes a bit of the D runtime.  I've been playing around a bit 
to see what could be possible.  To do that, I've had to do some 
linker hacking to make code that's completely free of D runtime 
dependencies.

I thought I'd write something up to help other people experiment 
with this stuff:
https://theartofmachinery.com/2016/12/18/d_without_runtime.html

Let's make a better -betterC :)
Dec 17 2016
next sibling parent Iain Buclaw via Digitalmars-d-announce writes:
On 18 December 2016 at 01:04, sarn via Digitalmars-d-announce
<digitalmars-d-announce puremagic.com> wrote:
 As it stands, the -betterC flag is still immature and only removes a bit of
 the D runtime.
-betterC removes module info and module helpers, not the D runtime. You will find it in gdc with the more appropriately named command line switch -fno-moduleinfo. ;-)
Dec 17 2016
prev sibling parent reply Mike <none none.com> writes:
On Sunday, 18 December 2016 at 00:04:54 UTC, sarn wrote:

 I thought I'd write something up to help other people 
 experiment with this stuff:
 https://theartofmachinery.com/2016/12/18/d_without_runtime.html
Thanks for this. I abandoned D sometime ago largely because of https://issues.dlang.org/show_bug.cgi?id=14758 (but there were other reasons), so your blog post is interesting to me. It is unfortunate that we have to resort to such hackery, but its nice to have such tools at our disposal regardless. I proposed another idea for giving users more control over D Runtime by moving runtime hook definitions to .di header files. If you're interested, you can read about it here: http://forum.dlang.org/post/psssnzurlzeqeneagora forum.dlang.org. I'd much rather have something like that over a -betterC; you can read more about some disadvantages to expanding on -betterC (e.g. removing RTTI) here: http://forum.dlang.org/post/nevipjrkdqxivoerftlw forum.dlang.org. I've largely embraced Rust now for its "no runtime" and "no dependencies libcore" features (and a few other safety/robustness features), but I miss the modeling power and compile-time features of D greatly. Anyway, thanks for the post; it's given me a few ideas. Mike
Dec 17 2016
next sibling parent sarn <sarn theartofmachinery.com> writes:
On Sunday, 18 December 2016 at 02:37:22 UTC, Mike wrote:
 I abandoned D sometime ago largely because of 
 https://issues.dlang.org/show_bug.cgi?id=14758 (but there were 
 other reasons), so your blog post is interesting to me.  It is 
 unfortunate that we have to resort to such hackery, but its 
 nice to have such tools at our disposal regardless.
Yeah, the TypeInfo spam is the biggest pain point.
 I proposed another idea for giving users more control over D 
 Runtime by moving runtime hook definitions to .di header files.
  If you're interested, you can read about it here:  
 http://forum.dlang.org/post/psssnzurlzeqeneagora forum.dlang.org.
  I'd much rather have something like that over a -betterC; you 
 can read more about some disadvantages to expanding on -betterC 
 (e.g. removing RTTI) here: 
 http://forum.dlang.org/post/nevipjrkdqxivoerftlw forum.dlang.org.
I think D can still be very usable without TypeInfo (especially if the unnecessary language dependence on it improves). But I'm also wary of "solving" the problem with a hundred compiler flags and causing fragmentation.
 Anyway, thanks for the post; it's given me a few ideas.
Thanks for letting me know. I wasn't totally sure anyone would be interested in that hack.
Dec 18 2016
prev sibling parent reply Iain Buclaw via Digitalmars-d-announce writes:
On 18 December 2016 at 03:37, Mike via Digitalmars-d-announce
<digitalmars-d-announce puremagic.com> wrote:
 On Sunday, 18 December 2016 at 00:04:54 UTC, sarn wrote:

 I thought I'd write something up to help other people experiment with this
 stuff:
 https://theartofmachinery.com/2016/12/18/d_without_runtime.html
Thanks for this. I abandoned D sometime ago largely because of https://issues.dlang.org/show_bug.cgi?id=14758 (but there were other reasons), so your blog post is interesting to me. It is unfortunate that we have to resort to such hackery, but its nice to have such tools at our disposal regardless.
As a response to my last message in that thread, the changes for classinfo generation is now ready to go in master. However I already have ideas for future implementation that is not so naive when it comes to looking up fields to assign data. How does a kind of tagging system sound? i.e: class TypeInfo_Class { name string classname; init byte[] initializer; }
Dec 18 2016
next sibling parent reply Mike <none none.com> writes:
On Sunday, 18 December 2016 at 12:57:08 UTC, Iain Buclaw wrote:

 As a response to my last message in that thread, the changes 
 for classinfo generation is now ready to go in master.
Ok, I'll give it a test in the next week or so.
 However I already have ideas for future implementation that is 
 not so naive when it comes to looking up fields to assign data. 
  How does a kind of tagging system sound?

 i.e:

 class TypeInfo_Class
 {
      name
     string classname;

      init
     byte[] initializer;
 }
I'm assuming this idea addresses the issue with porting druntime and leaving some features unimplemented. If I understand your intention, this looks akin to Rust's Lang Items feature (https://doc.rust-lang.org/book/lang-items.html). Please correct me if I'm wrong. I'm not sure what you're envisioning here, so I'd like to hear more. Would the user get compile-time errors if they used a D feature that required a druntime implementation, but the implementation did not exist? Mike
Dec 18 2016
parent reply Iain Buclaw via Digitalmars-d-announce writes:
On 19 December 2016 at 01:36, Mike via Digitalmars-d-announce
<digitalmars-d-announce puremagic.com> wrote:
 On Sunday, 18 December 2016 at 12:57:08 UTC, Iain Buclaw wrote:

 As a response to my last message in that thread, the changes for classinfo
 generation is now ready to go in master.
Ok, I'll give it a test in the next week or so.
 However I already have ideas for future implementation that is not so
 naive when it comes to looking up fields to assign data.  How does a kind of
 tagging system sound?

 i.e:

 class TypeInfo_Class
 {
      name
     string classname;

      init
     byte[] initializer;
 }
I'm assuming this idea addresses the issue with porting druntime and leaving some features unimplemented. If I understand your intention, this looks akin to Rust's Lang Items feature (https://doc.rust-lang.org/book/lang-items.html). Please correct me if I'm wrong.
Heh, I've never looked into Rust, so I didn't know that. But in a way, yes, it is quite similar.
 I'm not sure what you're envisioning here, so I'd like to hear more.  Would
 the user get compile-time errors if they used a D feature that required a
 druntime implementation, but the implementation did not exist?

 Mike
The compiler doesn't actually generate any code that peeks inside TypeInfo. It only generates the reference to the right typeinfo to pass to library runtime functions - or on request via typeid(). It doesn't actually care about the data stored inside. And that's the angle I've taken when laying out the actual data - if you provide the fields we want to populate, then we'll populate them. If you omit a few, then the compiler won't bother with them. Because at the end of the day, it's druntime library that uses and makes sense of the TypeInfo data provided. The compile just says: "Well, this is as much as I'm willing to tell you about the type." Iain
Dec 19 2016
parent reply Mike <none none.com> writes:
On Monday, 19 December 2016 at 19:53:06 UTC, Iain Buclaw wrote:

 The compiler doesn't actually generate any code that peeks 
 inside TypeInfo.  It only generates the reference to the right 
 typeinfo to pass to library runtime functions -  or on request 
 via typeid().  It doesn't actually care about the data stored 
 inside.  And that's the angle I've taken when laying out the 
 actual data - if you provide the fields we want to populate, 
 then we'll populate them.  If you omit a few, then the compiler 
 won't bother with them.  Because at the end of the day, it's 
 druntime library that uses and makes sense of the TypeInfo data 
 provided.  The compile just says: "Well, this is as much as I'm 
 willing to tell you about the type."
Ok, that's interesting, but what if you don't want TypeInfo at all? Can you omit the TypeInfo structure entirely from object.d? Or perhaps you still need to declare it in object.d, but since the compiler doesn't find any fields to populate, it results in an empty struct? I'd really hate to have to add empty TypeInfo_XXX classes to my object.d though. I'm still wondering about what the programmer would see if they tried to do a dyamic cast (for example) and there was no (or an incomplete) TypeInfo in the runtime. You see, when I started with D, I had a grand plan that I could create a product like Arduino. I would provide a PCB and an accompanying API/SDK to my customers, and they would do the programming. I would have wanted them to have a polished experience when programming my product, so if they were doing something that wasn't supported by the platform, they would receive a friendly compiler message telling them so. I thought that this would be possible if the compiler resolved druntime symbols at compile-time in the same way it does any other library: by reading .di header files. If a symbol was not found, they would get a compiler error telling them so. I could also add templates, static-ifs, and static asserts to the .di files for, not only generating highly optimized code, but also to notify the programmer with more helpful and precise compile-time messages. I also envisioned products with very high resource constraints (like ARM Cortex-M0) without threading, but for high performance microcontrollers (like ARM Cortex-M4/7) I could leverage D's built-in understanding of thead as my RTOS. Each product would, therefore, have very different druntime implementations. I think what you're proposing here by tagging fields is certainly better than what we currently have, and would be desirable for other D users (especially those of late), but does it scale well if I want to deliver a polished programming experience (with optimized code generation) to my customers when my platform only has a subset of D's rich feature set? Mike
Dec 20 2016
parent reply Iain Buclaw via Digitalmars-d-announce writes:
On 20 December 2016 at 12:24, Mike via Digitalmars-d-announce
<digitalmars-d-announce puremagic.com> wrote:
 On Monday, 19 December 2016 at 19:53:06 UTC, Iain Buclaw wrote:

 The compiler doesn't actually generate any code that peeks inside
 TypeInfo.  It only generates the reference to the right typeinfo to pass to
 library runtime functions -  or on request via typeid().  It doesn't
 actually care about the data stored inside.  And that's the angle I've taken
 when laying out the actual data - if you provide the fields we want to
 populate, then we'll populate them.  If you omit a few, then the compiler
 won't bother with them.  Because at the end of the day, it's druntime
 library that uses and makes sense of the TypeInfo data provided.  The
 compile just says: "Well, this is as much as I'm willing to tell you about
 the type."
Ok, that's interesting, but what if you don't want TypeInfo at all? Can you omit the TypeInfo structure entirely from object.d? Or perhaps you still need to declare it in object.d, but since the compiler doesn't find any fields to populate, it results in an empty struct? I'd really hate to have to add empty TypeInfo_XXX classes to my object.d though.
Yeah, there are two logical steps that need ratifying. 1) -fno-rtti should be a flag that is honoured by the compiler. 2) Runtime functions that "lower" to a C function call should not generate RTTI if all information about the type is known at compile-time. There are ideas floating around to improve [2], but for the time being however, marking all functions as nogc catches almost all cases where this happens.
Jan 08
parent reply Mike <none none.com> writes:
On Sunday, 8 January 2017 at 22:14:36 UTC, Iain Buclaw wrote:

 1) -fno-rtti should be a flag that is honoured by the compiler.
The more I think about it the more I dislike the whole idea of -fno-rtti. All I've ever wanted from the D compiler is to not put code in my binary that has not chance of ever being used. Or, to generate it in a way that can be garbage collected by the linker. If that dream came true, I could avoid RTTI by simply avoiding any feature that needed it (REALLY needed it!) and -fno-rtti would be of no value, except to enforce policy.
Jan 09
parent sarn <sarn theartofmachinery.com> writes:
On Monday, 9 January 2017 at 13:42:01 UTC, Mike wrote:
 On Sunday, 8 January 2017 at 22:14:36 UTC, Iain Buclaw wrote:

 1) -fno-rtti should be a flag that is honoured by the compiler.
The more I think about it the more I dislike the whole idea of -fno-rtti. All I've ever wanted from the D compiler is to not put code in my binary that has not chance of ever being used. Or, to generate it in a way that can be garbage collected by the linker.
I've actually come to the same conclusion. Different combinations of compiler flags for different codebases is just a recipe for pain. We already have things like nothrow, and I'm pretty sure we can solve at least most of the problem just by exploiting language features to do what you said.
Jan 09
prev sibling parent reply Mike <none none.com> writes:
On Sunday, 18 December 2016 at 12:57:08 UTC, Iain Buclaw wrote:
 On 18 December 2016 at 03:37, Mike via Digitalmars-d-announce 
 <digitalmars-d-announce puremagic.com> wrote:
 On Sunday, 18 December 2016 at 00:04:54 UTC, sarn wrote:

 I thought I'd write something up to help other people 
 experiment with this
 stuff:
 https://theartofmachinery.com/2016/12/18/d_without_runtime.html
Thanks for this. I abandoned D sometime ago largely because of https://issues.dlang.org/show_bug.cgi?id=14758 (but there were other reasons), so your blog post is interesting to me. It is unfortunate that we have to resort to such hackery, but its nice to have such tools at our disposal regardless.
As a response to my last message in that thread, the changes for classinfo generation is now ready to go in master.
Just built GDC from the gdc-6 branch. The commit history says master was merged in 24 hours ago, so I'm assuming your ClassInfo changes are there. Anyway the build produces the same result: TypeInfo strings for each an every type in the .rodata section. The binary should be about 6K, but is instead about 600K. Perhaps I'll try again at a later date. Mike
Dec 19 2016
parent Iain Buclaw via Digitalmars-d-announce writes:
On 19 December 2016 at 12:41, Mike via Digitalmars-d-announce
<digitalmars-d-announce puremagic.com> wrote:
 On Sunday, 18 December 2016 at 12:57:08 UTC, Iain Buclaw wrote:
 On 18 December 2016 at 03:37, Mike via Digitalmars-d-announce
 <digitalmars-d-announce puremagic.com> wrote:
 On Sunday, 18 December 2016 at 00:04:54 UTC, sarn wrote:

 I thought I'd write something up to help other people experiment with
 this
 stuff:
 https://theartofmachinery.com/2016/12/18/d_without_runtime.html
Thanks for this. I abandoned D sometime ago largely because of https://issues.dlang.org/show_bug.cgi?id=14758 (but there were other reasons), so your blog post is interesting to me. It is unfortunate that we have to resort to such hackery, but its nice to have such tools at our disposal regardless.
As a response to my last message in that thread, the changes for classinfo generation is now ready to go in master.
Just built GDC from the gdc-6 branch. The commit history says master was merged in 24 hours ago, so I'm assuming your ClassInfo changes are there. Anyway the build produces the same result: TypeInfo strings for each an every type in the .rodata section. The binary should be about 6K, but is instead about 600K. Perhaps I'll try again at a later date. Mike
Yeah, I'm just wrapping up the last touches, PR will land for master in days, or tonight maybe... May be a delay before it gets merged into the gdc-6 branch.
Dec 19 2016