www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - RFC: Pay-as-you-go, Portable D Runtime for Microcontrollers (and

reply "Mike" <none none.com> writes:
Read on GitHub: 
https://github.com/JinShil/minimal_druntime_experiment

There was recently a discussion about how we could create a 
portable, pay-as-you-go, D runtime to help bring the promise of D 
to free-standing platforms and devices with tight resource 
constraints (e.g. microcontrollers).  Thread started here:  
http://forum.dlang.org/post/mhrs4p$31id$1 digitalmars.com

The primary motivation is to create an arm-none-eabi GDC 
cross-compiler toolchain for programming ARM Cortex-M 
microcontrollers in D, but I think there's a way to achieve 
broader platform support by delegating implementation details 
down the "supply chain".  I hope to articulate that strategy in 
this post.

To prove the concept, provide a place to start, and discuss 
ideas, I created the most minimal D runtime I could:  
https://github.com/JinShil/minimal_druntime_experiment.  I've 
also included Phobos, so we could still have `std.stdio.write` 
and `std.stdio.writeln` for console output, as every device needs 
a console for development.

Overview
**********
d
├── phobos
│   └── std
└── runtime
     └── rt
         └── typeinfo

ports
├── arm
│   └── cortexm4
│       ├── phobosWe
│       │   └── phobosPort.d
│       └── runtime
│           └── runtimePort.d
├── posix
│   └── linux
│       ├── phobos
│       │   └── phobosPort.d
│       └── runtime
│           ├── c_types.d
│           └── runtimePort.d


There are two main folders: "d" and "ports".  "d" provides the 
patform-agnostic code, or code that is relevant to a large number 
of platforms.  The "ports" directory provides the 
platform-specific implementation.  Building simply requires 
importing "d/runtime", "d/phobos", and your platform's hierarchy 
in the "ports" folder.  At the moment, I've only implemented a 
Linux 64-bit port and an ARM Cortex-M4 port to illustrate the 
concept.  This is roughly how I wish the official runtime could 
be structured in the spirit of Issue 11666.

The official runtime includes platform-specific bindings in 
`core.sys` and `stdc`, but I think that is a design anomaly.  
While a port may find it convenient to use those bindings, they 
should not be part of the D language's public API.  Rather, they 
should be deported to Deimos, and if needed, imported privately 
by a platform's port.  For the Linux port, I chose to write the 
platform's system calls in assembly to illustrate that it is not 
even necessary to use those bindings if one wants to do the due 
diligence in D.

Porting to a New Platform
******************************
The platform-agnostic code in "d" delgates implementation details 
to the platform-specific code using `extern(C) extern 
_d_sys_name` "system calls" (for lack of a better term).  This is 
how the newlib C library delegates platform-specific 
implementations:  See 
http://wiki.osdev.org/Porting_Newlib#newlib.2Flibc.2Fsys.2Fmyos.2Fsyscalls.c

At the moment, for the sake of demonstration, only 2 system calls 
have been defined:

* `__d_sys_write` - Equivalent to C's `write` system call
* `__d_sys_exit` - Equivalent to C's `exit` system call

These two system calls allow us to create a simple Hello World.  
"runtimePort.d" implements `__d_sys_exit` and "phobosPort.d" 
implements the `__d_sys_write`.

Putting it all Together
************************
Users are not expected to use this code directly.  Rather, I 
envision toolchain, silicon, and board vendors will use this code 
as a small, but essential part of, their platform's D programming 
package.  For example, a package for the ARM Cortex-M family of 
MCUs might contain the following:
* arm-none-eabi GDC cross-compiler
* a library containing compiled code for the "d" folder in this 
repository
* a library containing the compiled code for the 
"ports/arm/cortexm" folders in this repository
* cortex-m core startup files
* the newlib C library
* the C library bindings from Deimos
* multilibs from the GNU toolchain.

A silicon vendor like ST, may then take that package and add more 
platform-specific for their family of products:
* startup files with interrupt vectors for their peripherals
* linker scripts for each of their MCUs
* flash memory programmer
* library for on-dye peripherals
* etc..

A board vendor may choose to create yet another layer on top of 
the silicon vendor's package.
* library for peripherals external to the MCU (external RAM, IO 
expanders, gyroscope, LCD, etc...)

In short, this repository just provides just the foundation to 
"get D working", and delegates to toolchain, silicon, and board 
vendors to fill in the details for their products.

RFC
*****
For those who have stake in this code base (kernel developers, 
embedded developers, toolchain and package maintainers, etc...) 
your constructive criticism and ideas are most welcome.  I am 
unsure yet how this will play out, and what roles players will 
assume, but let's give it a try.

Plea to Compiler Implementers
***********************************
We need better control over codegen.  TypeInfo and dead-code 
removal is preventing me from making progress 
(http://forum.dlang.org/post/quemhwpgijwmqtpxukiv forum.dlang.org). 
  I've resorted to compiling to assembly, using sed to hack the 
assembly, and then compiling the assembly.  Things like that make 
me want to not use D at all.  Here are some ideas from myself and 
others:
*  Move TypeInfo to the runtime: 
https://issues.dlang.org/show_bug.cgi?id=12270
*  Move D runtime hooks out of the compiler and into *.di files 
so they can be decorated with attributes and `version`ed so the 
compiler can see what features of D are supported by the port and 
generate smarter code:  
http://forum.dlang.org/post/psssnzurlzeqeneagora forum.dlang.org
*  Add -fno-rtti compiler switch
*  Add attribute support so programmer can choose which types to 
generate runtime-time info for.

I don't know what the right solution is.  Let's have that 
discussion.

LDC Folks:  https://github.com/ldc-developers/ldc/issues/781 is 
currently preventing me from supporting LDC with this runtime.

Mike
May 04 2015
next sibling parent reply Dan Olson <zans.is.for.cans yahoo.com> writes:
"Mike" <none none.com> writes:

 Read on GitHub: https://github.com/JinShil/minimal_druntime_experiment

 There was recently a discussion about how we could create a portable,
 pay-as-you-go, D runtime to help bring the promise of D to
 free-standing platforms and devices with tight resource constraints
 (e.g. microcontrollers).  Thread started here:
 http://forum.dlang.org/post/mhrs4p$31id$1 digitalmars.com
This is cool stuff. Embedded has been the bulk of my software life and is most fun. I see there has been a lot of discussion that I'll have to read up on before I say much more though.
 * the newlib C library
newlib has treated me well in the past, especially being so highly customizable. I think it is a good choice. It would be fun to rewrite it in D, but that perhaps is much work for little immediate benefit.
 Plea to Compiler Implementers
 ***********************************
 We need better control over codegen.  TypeInfo and dead-code removal
 is preventing me from making progress
 (http://forum.dlang.org/post/quemhwpgijwmqtpxukiv forum.dlang.org). I've
 resorted to compiling to assembly, using sed to hack the assembly, and
 then compiling the assembly.  Things like that make me want to not use
 D at all.
Yeah, this should be improved. How about a -disable-tls option so that when there are no threads and thus no TLS, you can compile exising D code as-is. Otherwise you have to rewrite normal variables to __gshared everywhere (I actually have this option in a ldc fork).
 LDC Folks:  https://github.com/ldc-developers/ldc/issues/781 is
 currently preventing me from supporting LDC with this runtime.
I looked at this and found a workaround and noted it in the above issue link. -- Dan Olson
May 04 2015
next sibling parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 5 May 2015 at 08:39, Dan Olson via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 "Mike" <none none.com> writes:

 Read on GitHub: https://github.com/JinShil/minimal_druntime_experiment

 There was recently a discussion about how we could create a portable,
 pay-as-you-go, D runtime to help bring the promise of D to
 free-standing platforms and devices with tight resource constraints
 (e.g. microcontrollers).  Thread started here:
 http://forum.dlang.org/post/mhrs4p$31id$1 digitalmars.com
This is cool stuff. Embedded has been the bulk of my software life and is most fun. I see there has been a lot of discussion that I'll have to read up on before I say much more though.
 * the newlib C library
newlib has treated me well in the past, especially being so highly customizable. I think it is a good choice. It would be fun to rewrite it in D, but that perhaps is much work for little immediate benefit.
 Plea to Compiler Implementers
 ***********************************
 We need better control over codegen.  TypeInfo and dead-code removal
 is preventing me from making progress
 (http://forum.dlang.org/post/quemhwpgijwmqtpxukiv forum.dlang.org). I've
 resorted to compiling to assembly, using sed to hack the assembly, and
 then compiling the assembly.  Things like that make me want to not use
 D at all.
Yeah, this should be improved. How about a -disable-tls option so that when there are no threads and thus no TLS, you can compile exising D code as-is. Otherwise you have to rewrite normal variables to __gshared everywhere (I actually have this option in a ldc fork).
Configure GDC with --disable-tls --disable-threads to get the desired behaviour (disabling only TLS just makes codegen fallback to emulated thread support).
May 04 2015
next sibling parent reply "Mike" <none none.com> writes:
On Tuesday, 5 May 2015 at 06:56:52 UTC, Iain Buclaw wrote:
 Configure GDC with --disable-tls --disable-threads to get the 
 desired behaviour (disabling only TLS just makes codegen 
 fallback to emulated thread support).
I didn't know this. You mean if I configure with those options, TLS variables will be treated as __gshared? Mike
May 05 2015
parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 5 May 2015 at 09:48, Mike via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Tuesday, 5 May 2015 at 06:56:52 UTC, Iain Buclaw wrote:
 Configure GDC with --disable-tls --disable-threads to get the desired
 behaviour (disabling only TLS just makes codegen fallback to emulated thread
 support).
I didn't know this. You mean if I configure with those options, TLS variables will be treated as __gshared? Mike
I have no control over what the backend does, but if it generates control variables for them (emulated tls), then __get_address would incur a small heap cost when first retrieved. If the cost is too much (ie: there is no heap), then we should be able to stop emutls from kicking in by preventing the D frontend from returning true on isThreadLocal calls.
May 05 2015
parent reply Dan Olson <zans.is.for.cans yahoo.com> writes:
Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:

 If the cost is too much (ie: there is no heap), then we should be able
 to stop emutls from kicking in by preventing the D frontend from
 returning true on isThreadLocal calls.
Iain, I think something like this would be good in GDC. Makes it so much easier to compile for no thread environments.
May 05 2015
parent reply Johannes Pfau <nospam example.com> writes:
Am Tue, 05 May 2015 08:41:13 -0700
schrieb Dan Olson <zans.is.for.cans yahoo.com>:

 Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
 
 If the cost is too much (ie: there is no heap), then we should be
 able to stop emutls from kicking in by preventing the D frontend
 from returning true on isThreadLocal calls.
Iain, I think something like this would be good in GDC. Makes it so much easier to compile for no thread environments.
I'm not really sure about this. It's trivial to implement but it generates a semantic difference which harms code portability (e.g. if you have code where you want a global variable, don't mark it as __gshared and then move to an environment with threads).
May 05 2015
parent reply "Mike" <none none.com> writes:
On Tuesday, 5 May 2015 at 17:27:05 UTC, Johannes Pfau wrote:
 Am Tue, 05 May 2015 08:41:13 -0700
 schrieb Dan Olson <zans.is.for.cans yahoo.com>:

 Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> 
 writes:
 
 If the cost is too much (ie: there is no heap), then we 
 should be
 able to stop emutls from kicking in by preventing the D 
 frontend
 from returning true on isThreadLocal calls.
Iain, I think something like this would be good in GDC. Makes it so much easier to compile for no thread environments.
I'm not really sure about this. It's trivial to implement but it generates a semantic difference which harms code portability (e.g. if you have code where you want a global variable, don't mark it as __gshared and then move to an environment with threads).
I'm coming around and feeling inclined to agree with this. One of the things I want to avoid is creatIng an embedded dialect of the D. I hope D for microcontrollers, kernels and such looks very much like the idiomatic D in the application programming domain. What I would like to see, however, is a friendly compiler error like "This platform does not support thread-local storage. Please decorate your global variables with __gshared." rather than "undefined reference '_d_get_address'" from the linker. Mike
May 05 2015
next sibling parent reply "Suliman" <evermind live.ru> writes:
Maybe it's stupid question, but is there any chance to get D on 
AVR(8/16 bit)?
May 05 2015
parent reply Johannes Pfau <nospam example.com> writes:
Am Wed, 06 May 2015 04:47:53 +0000
schrieb "Suliman" <evermind live.ru>:

 Maybe it's stupid question, but is there any chance to get D on 
 AVR(8/16 bit)?
Yes! I've written small D test programs for AVR. The only problem could be far pointers. However, far pointers are very rare in AVR code and it's hard to find documentation. It seems there's no compiler support for far pointers and everything is implemented in the library[1]. In that case there's no issue for D. [1] http://www.nongnu.org/avr-libc/user-manual/group__avr__pgmspace.html
May 06 2015
parent reply "Suliman" <evermind live.ru> writes:
Yes! I've written small D test programs for AVR.
you mean 8-bit AVR? It's very cool! Could you show sources code example?
May 06 2015
parent Johannes Pfau <nospam example.com> writes:
Am Wed, 06 May 2015 07:44:03 +0000
schrieb "Suliman" <evermind live.ru>:

Yes! I've written small D test programs for AVR.
you mean 8-bit AVR? It's very cool! Could you show sources code example?
Yes [1] [2] [3]. Only Hello-World / proof of concept kind of code for now. There are no fundamental issues, just a lack of time to finish this. I've focused on small and optimized codegen rather than nice examples for now. It also uses the avr-libc instead of custom startup code (because of lack of time). The next step is generating all the register wrappers. It's a very tedious task but I've got a script 90% ready to parse the register tables out of the Atmel pdfs. The output should still be checked manually though. Unfortunately I don't have time to work on this for now, but I'll get back to it at some point ;-) [1] https://github.com/D-Programming-microD/avr-playground/blob/master/src/test.d [2] https://github.com/D-Programming-microD/avr-playground/blob/master/src/util/delay.d [3] (mixin generator is ready but not yet on github) https://github.com/D-Programming-microD/avr-playground/blob/master/src/util/mmio.d
May 06 2015
prev sibling next sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Wed, 06 May 2015 02:44:53 +0000
schrieb "Mike" <none none.com>:

 On Tuesday, 5 May 2015 at 17:27:05 UTC, Johannes Pfau wrote:
 Am Tue, 05 May 2015 08:41:13 -0700
 schrieb Dan Olson <zans.is.for.cans yahoo.com>:

 Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> 
 writes:
 
 If the cost is too much (ie: there is no heap), then we 
 should be
 able to stop emutls from kicking in by preventing the D 
 frontend
 from returning true on isThreadLocal calls.
Iain, I think something like this would be good in GDC. Makes it so much easier to compile for no thread environments.
I'm not really sure about this. It's trivial to implement but it generates a semantic difference which harms code portability (e.g. if you have code where you want a global variable, don't mark it as __gshared and then move to an environment with threads).
I'm coming around and feeling inclined to agree with this. One of the things I want to avoid is creatIng an embedded dialect of the D. I hope D for microcontrollers, kernels and such looks very much like the idiomatic D in the application programming domain. What I would like to see, however, is a friendly compiler error like "This platform does not support thread-local storage. Please decorate your global variables with __gshared." rather than "undefined reference '_d_get_address'" from the linker. Mike
That's easy: https://github.com/D-Programming-microD/GDC/commit/d5677e03f40f4eb548e520728c5071a0b4445d24 (Well, the error message could be a little bit friendlier ;-)
May 06 2015
parent "Mike" <none none.com> writes:
On Wednesday, 6 May 2015 at 07:20:38 UTC, Johannes Pfau wrote:

 
 What I would like to see, however, is a friendly compiler 
 error like "This platform does not support thread-local 
 storage.  Please decorate your global variables with 
 __gshared." rather than "undefined reference '_d_get_address'" 
 from the linker.
 
 
 Mike
That's easy: https://github.com/D-Programming-microD/GDC/commit/d5677e03f40f4eb548e520728c5071a0b4445d24 (Well, the error message could be a little bit friendlier ;-)
You know, that fork of yours has some really interesting stuff in it :0 The -ftls-support option would be really great, but I'd call it -fno-tls :) I think such an option would be most useful at configure-time. That is, if GDC is configured with --disable-tls or --disable-threads, then -fno-tls is ON by default. That way it's more a definition of the target platform than user choice. And now that I think of it, tying some of those compiler options to the configure options might be a better way to define platform support rather than reading .di files or enums from a special "configuration" module. Mike
May 06 2015
prev sibling parent Dan Olson <zans.is.for.cans yahoo.com> writes:
The progress here is nice.  I'd like to do similar stuff with LDC but
have a couple other projects to finish up first.

More thoughts on supporting systems without threads:

"Mike" <none none.com> writes:
 On Tuesday, 5 May 2015 at 17:27:05 UTC, Johannes Pfau wrote:
 I'm not really sure about this. It's trivial to implement but it
 generates a semantic difference which harms code portability
 (e.g. if
 you have code where you want a global variable, don't mark it as
 __gshared and then move to an environment with threads).
I think this could happen anyway. I write some D code on linux, but my program doesn't create additional threads so I don't realize I should have used __gshared somewhere. Sometime later when I use code in program with multi threads, I discover my problem that I should have used __gshared on a var.
 I'm coming around and feeling inclined to agree with this.  One of the
 things I want to avoid is creatIng an embedded dialect of the D.  I
 hope D for microcontrollers, kernels and such looks very much like the
 idiomatic D in the application programming domain.
I don't think it changes D, does it? I want to try to convince here because I think it opens up a lot of D code for no-thread systems that would otherwise need to be edited and have __gshared added here and there. The language just says a non-immutable global declaration [without shared or __gshared] lives in thread local storage. In a single thread environment, a thread local degenerates to a global, doesn't it? Looking at it another way. I could design a "TLS" model for a non-threaded system that has the compiler puts all "thread local" vars in section named ".tls" instead of ".data". Because there is only one thread, there is only one ".tls" section. But I could then have linker merge .tls and .data sections. And instructions to reference tls and non-tls vars are identical. I don't think going the other way is safe. If a system supports threads but not TLS, then turning D thread local vars into globals is not good (however I did do that so I could get initial work done on iOS before I added TLS to backend). -- Dan
May 06 2015
prev sibling parent reply "Jens Bauer" <doctor who.no> writes:
On Tuesday, 5 May 2015 at 06:56:52 UTC, Iain Buclaw wrote:
 On 5 May 2015 at 08:39, Dan Olson via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 How about a -disable-tls option so that when there are no 
 threads and
 thus no TLS, you can compile exising D code as-is.  Otherwise 
 you have
 to rewrite normal variables to __gshared everywhere (I 
 actually have this
 option in a ldc fork).
Configure GDC with --disable-tls --disable-threads to get the desired behaviour (disabling only TLS just makes codegen fallback to emulated thread support).
I already have supplied those options in my toolchain. But does anyone know if this is advisable when using FreeRTOS (or any other RTOS for that matter) ? -I'm asking, because I'm not using any RTOS myself, but there are loads of people who do.
May 07 2015
parent reply "Mike" <none none.com> writes:
On Thursday, 7 May 2015 at 14:48:08 UTC, Jens Bauer wrote:
 I already have supplied those options in my toolchain.
 But does anyone know if this is advisable when using FreeRTOS 
 (or any other RTOS for that matter) ?
 -I'm asking, because I'm not using any RTOS myself, but there 
 are loads of people who do.
In order to make the full stack in D, I think we eventually will need to make 2 toolchains: a bare-metal kernel toolchain, and an application programming toolchain. The bare-metal kernel toolchain would not have some of the high-level features of D, like threading and synchronization, as that has yet to be built. However, once a D RTOS is created with all necessary features for theading and synchronization, then the application programming toolchain can be made with a druntime ported the D RTOS's API. I've also considered another interesting approach. It seems possible to port all features of D right to the metal, essentially embedding the RTOS directly into the runtime. Then D is your RTOS :-) Mike
May 09 2015
parent reply "Jens Bauer" <doctor who.no> writes:
On Sunday, 10 May 2015 at 01:55:53 UTC, Mike wrote:
 On Thursday, 7 May 2015 at 14:48:08 UTC, Jens Bauer wrote:
 I already have supplied those options in my toolchain.
 But does anyone know if this is advisable when using FreeRTOS 
 (or any other RTOS for that matter) ?
 -I'm asking, because I'm not using any RTOS myself, but there 
 are loads of people who do.
In order to make the full stack in D, I think we eventually will need to make 2 toolchains: a bare-metal kernel toolchain, and an application programming toolchain.
That does not sound too appealing, because as far as I know, you have your bare-metal arm-none-eabi toolchain with C and C++, which can build RTOS. Then you'll need another arm-rtos-eabi, which can build RTOS, but cannot build bare-metal. I think people will not like this, because they don't want to switch toolchains when they work on different projects. The arm-linux-eabi toolchain will be a third toolchain, because RTOS is not Linux, though Linux may be some kind of RTOS. ;)
 The bare-metal kernel toolchain would not have some of the 
 high-level features of D, like threading and synchronization, 
 as that has yet to be built.  However, once a D RTOS is created 
 with all necessary features for theading and synchronization, 
 then the application programming toolchain can be made with a 
 druntime ported the D RTOS's API.
Another (annoying) input: If I decide to write a context-switcher in assembly language, I suddenly have threads, which I'd of course like to be able to use with my bare-metal toolchain.
 I've also considered another interesting approach.  It seems 
 possible to port all features of D right to the metal, 
 essentially embedding the RTOS directly into the runtime.  Then 
 D is your RTOS :-)
I do like this approach better, and that resembles the way I've been thinking until now. Yes, it might require more work, but strongly I think it's worth it. I believe this would also give the user the most convenient D-compiler (and toolchain).
May 09 2015
parent reply "Adrian Matoga" <epi atari8.info> writes:
I somehow missed this topic earlier but I played a bit with
embedded runtime last month so I'll share some thoughts.

On Sunday, 10 May 2015 at 06:55:07 UTC, Jens Bauer wrote:
 On Sunday, 10 May 2015 at 01:55:53 UTC, Mike wrote:
 I've also considered another interesting approach.  It seems 
 possible to port all features of D right to the metal, 
 essentially embedding the RTOS directly into the runtime.  
 Then D is your RTOS :-)
I do like this approach better, and that resembles the way I've been thinking until now. Yes, it might require more work, but strongly I think it's worth it. I believe this would also give the user the most convenient D-compiler (and toolchain).
I'm also strongly in favor of integrating an RTOS with the runtime, especially if we publish it under a liberal license, just as the D runtime is distributed now. Since most of existing RTOSes are distributed under either proprietary license or GPL, having a ready-to-use RTOS without legal restrictions could work towards the adoption of D in the embedded market. I would start with evaluating the existing concurrency APIs in terms of their applicability in such RTOS. Generally, if we stick to the pay-as-you-go approach most features of D runtime (even exceptions and RTTI) can be ported. They will not imply any costs when not used, but will be ready to use out-of-the-box when they're needed. We could advertise it using e.g. automatically generated charts showing the cost of each feature. Also, there are already compiler switches to show GC allocations and TLS variables, so a similar approach could be used for other potentially costly features. As for TLS, I think we should not change D semantics by implicitly making all globals __gshared, but instead follow Dan's suggestion to create a specific TLS model which statically resolves thread-local variables to regular globals in single-threaded builds. I did some experiments with porting small parts of Phobos and druntime. Initial tests show that e.g. semihosted writefln costs about 4KB of flash initially + about 0.5KB per each new argument type list (GDC, -Os, Cortex-M3). Sure, it may seem a lot if your uC has 16KB or less, but in such case you probably wouldn't use formatted output much in C either. You don't pay for it when you don't call it, and when you need it - it's there and it's fully functional and type safe (unlike C's printf). And of course there're probably ways to optimize it. Also I think exceptions could be possible to implement without large costs. I've already got scope(exit) and scope(success) working, and IMHO this is already a huge advantage over manual cleanup. I started working on unwinding using the libgcc support but other duties stopped me from finishing it. I'll try to push this work on github later this week. I've run the above on LM3S6965 (qemu) and STM32F103 (Nucleo-F103RB).
Jun 10 2015
parent reply "Mike" <none none.com> writes:
On Wednesday, 10 June 2015 at 10:06:19 UTC, Adrian Matoga wrote:

 Generally, if we stick to the pay-as-you-go approach most
 features of D runtime (even exceptions and RTTI) can be ported.
 They will not imply any costs when not used, but will be ready 
 to
 use out-of-the-box when they're needed.
I haven't had a pay-as-you-go experience with RTTI. In fact TypeInfo has become my mortal enemy [1]. I submitted a bug report for now [2]. [1] TypeInfo not garbage collected (discussion) - http://forum.dlang.org/post/quemhwpgijwmqtpxukiv forum.dlang.org [2] TypeInfo not garbage collected (bug report) - http://bugzilla.gdcproject.org/show_bug.cgi?id=184 There is an implementation of an -fno-rtti switch [3], but unfortunately, I have found it compromises on a few things (slicing, postblit, and maybe others). I think the best way forward is to move TypeInfo to the runtime as described in [4]. I'm currently working on an initial pull request for it, but I have to admit that I don't know much about what I'm doing in the compiler and am struggling with it. But I'm afraid if I don't do it, it won't happen. I can't even continue with my work without it. [3] -fno-rtti implementation - https://github.com/D-Programming-GDC/GDC/pull/100 [4] Move TypeInfo to the D Runtime - https://issues.dlang.org/show_bug.cgi?id=12270
 I'll try to push this work on github later this week.
I look forward to seeing it. It's encouraging to see more interest in using D for this Domain. Mike
Jun 10 2015
parent reply "Adrian Matoga" <epi atari8.info> writes:
On Thursday, 11 June 2015 at 00:26:39 UTC, Mike wrote:
 On Wednesday, 10 June 2015 at 10:06:19 UTC, Adrian Matoga wrote:

 Generally, if we stick to the pay-as-you-go approach most
 features of D runtime (even exceptions and RTTI) can be ported.
 They will not imply any costs when not used, but will be ready 
 to
 use out-of-the-box when they're needed.
I haven't had a pay-as-you-go experience with RTTI. In fact TypeInfo has become my mortal enemy [1]. I submitted a bug report for now [2]. [1] TypeInfo not garbage collected (discussion) - http://forum.dlang.org/post/quemhwpgijwmqtpxukiv forum.dlang.org [2] TypeInfo not garbage collected (bug report) - http://bugzilla.gdcproject.org/show_bug.cgi?id=184 There is an implementation of an -fno-rtti switch [3], but unfortunately, I have found it compromises on a few things (slicing, postblit, and maybe others). I think the best way forward is to move TypeInfo to the runtime as described in [4]. I'm currently working on an initial pull request for it, but I have to admit that I don't know much about what I'm doing in the compiler and am struggling with it. But I'm afraid if I don't do it, it won't happen. I can't even continue with my work without it. [3] -fno-rtti implementation - https://github.com/D-Programming-GDC/GDC/pull/100 [4] Move TypeInfo to the D Runtime - https://issues.dlang.org/show_bug.cgi?id=12270
If there's anything I can help with, let me know. My experience with GDC is close to non-existent (I managed to make some one-line hacks to make it compile for AVR or to show the sizes of TypeInfos, but that was pretty easy), and I didn't even bother to build LDC, but I want to write stuff for STMs in D so I'm motivated to learn by doing.
 I'll try to push this work on github later this week.
I look forward to seeing it. It's encouraging to see more interest in using D for this Domain.
Here it is: https://github.com/epi/dirt It's just a playground where I try adding different features by trial and error, so it's very far from being as clean and organized as your projects. I wouldn't even start without your and Adam Ruppe's work, so thank you for that. Apart from first steps towards formatted print and exception support I mentioned earlier, there's a try on running static constructors (which is most likely broken anyway) in the correct order. The machinery used to make it work wastes some text and data memory to do it in run time, but since we're linking everything statically in the end, I wonder if the linker could be forced to put it all together instead.
Jun 14 2015
parent "Jens Bauer" <doctor who.no> writes:
On Sunday, 14 June 2015 at 19:44:48 UTC, Adrian Matoga wrote:
 On Thursday, 11 June 2015 at 00:26:39 UTC, Mike wrote:
 On Wednesday, 10 June 2015 at 10:06:19 UTC, Adrian Matoga 
 wrote:
If there's anything I can help with, let me know. My experience with GDC is close to non-existent (I managed to make some one-line hacks to make it compile for AVR or to show the sizes of TypeInfos, but that was pretty easy), and I didn't even bother to build LDC, but I want to write stuff for STMs in D so I'm motivated to learn by doing.
Today I'll start using launchpad's GCC with GDC, because it has support for the Cortex-M7, plus that it's the official ARM toolchain, and it seems to be slightly ahead of the official GCC releases.
 Here it is: https://github.com/epi/dirt
I like small; I like simple - so I like it. :) Feel free to grab the startup-files here: https://github.com/jens-gpio/MCU (see http://d.gpio.dk/ for more info and how to contact me if you need to).
Jun 18 2015
prev sibling parent reply "Mike" <none none.com> writes:
On Tuesday, 5 May 2015 at 06:39:42 UTC, Dan Olson wrote:

 LDC Folks:  https://github.com/ldc-developers/ldc/issues/781 is
 currently preventing me from supporting LDC with this runtime.
I looked at this and found a workaround and noted it in the above issue link.
Many Thanks! I've gotten further thanks to your help, but now I'm stuck on https://github.com/ldc-developers/ldc/issues/925. Mike
May 05 2015
next sibling parent Dan Olson <zans.is.for.cans yahoo.com> writes:
"Mike" <none none.com> writes:

 On Tuesday, 5 May 2015 at 06:39:42 UTC, Dan Olson wrote:

 LDC Folks:  https://github.com/ldc-developers/ldc/issues/781 is
 currently preventing me from supporting LDC with this runtime.
I looked at this and found a workaround and noted it in the above issue link.
Many Thanks! I've gotten further thanks to your help, but now I'm stuck on https://github.com/ldc-developers/ldc/issues/925. Mike
Mike, Just updated the issue (you probably got an email). It is caused by a change from 2.066 to 2.067 in DMD front end versions, not specific to LDC.
May 05 2015
prev sibling parent reply "Mike" <none none.com> writes:
On Tuesday, 5 May 2015 at 08:13:21 UTC, Mike wrote:
 On Tuesday, 5 May 2015 at 06:39:42 UTC, Dan Olson wrote:

 LDC Folks:  https://github.com/ldc-developers/ldc/issues/781 
 is
 currently preventing me from supporting LDC with this runtime.
I looked at this and found a workaround and noted it in the above issue link.
Many Thanks! I've gotten further thanks to your help, but now I'm stuck on https://github.com/ldc-developers/ldc/issues/925. Mike
I've gotten even further thanks to everyone's help, but LDC is still giving me a little grief. Take a look at these undefined references (abbreviated for this forum): (_D10TypeInfo_l6__vtblZ+0x8): undef ref `_D6object8TypeInfo8toStringMxFNaNbNfZAya' (_D10TypeInfo_l6__vtblZ+0x10): undef ref `_D6object8TypeInfo6toHashMxFNbNeZm' (_D10TypeInfo_l6__vtblZ+0x18): undef ref `_D6object8TypeInfo5opCmpMFC6ObjectZi' (_D10TypeInfo_l6__vtblZ+0x20): undef ref `_D6object8TypeInfo8opEqualsMFC6ObjectZb' (_D10TypeInfo_l6__vtblZ+0x28): undef ref `_D6object8TypeInfo7getHashMxFNbNexPvZm' (_D10TypeInfo_l6__vtblZ+0x30): undef ref `_D6object8TypeInfo6equalsMxFxPvxPvZb' (_D10TypeInfo_l6__vtblZ+0x38): undef ref `_D6object8TypeInfo7compareMxFxPvxPvZi' (_D10TypeInfo_l6__vtblZ+0x40): undef ref `_D6object8TypeInfo5tsizeMxFNaNbNdNiNfZm' (_D10TypeInfo_l6__vtblZ+0x48): undef ref `_D6object8TypeInfo4swapMxFPvPvZv' (_D10TypeInfo_l6__vtblZ+0x50): undef ref `_D6object8TypeInfo4nextMNgFNaNbNdNiZNgC8TypeInfo' (_D10TypeInfo_l6__vtblZ+0x58): undef ref `_D6object8TypeInfo4initMxFNaNbNiNfZAxv' (_D10TypeInfo_l6__vtblZ+0x60): undef ref `_D6object8TypeInfo5flagsMxFNaNbNdNiNfZk' (_D10TypeInfo_l6__vtblZ+0x68): undef ref `_D6object8TypeInfo5offTiMxFZAxS6object14OffsetTypeInfo' (_D10TypeInfo_l6__vtblZ+0x70): undef ref `_D6object8TypeInfo7destroyMxFPvZv' (_D10TypeInfo_l6__vtblZ+0x78): undef ref `_D6object8TypeInfo8postblitMxFPvZv' (_D10TypeInfo_l6__vtblZ+0x80): undef ref `_D6object8TypeInfo6talignMxFNaNbNdNiNfZm' (_D10TypeInfo_l6__vtblZ+0x88): undef ref `_D6object8TypeInfo8argTypesMFNbNfJC8TypeInfoJC8TypeInfoZi' (_D10TypeInfo_l6__vtblZ+0x90): undef ref `_D6object8TypeInfo6rtInfoMxFNaNbNdNiNfZPyv' In function `_D3std5stdio16__T7writelnTAyaZ7writelnFAyaZv': (.text._D3std5stdio16__T7writelnTAyaZ7writelnFAyaZv[_D3std5stdio16__T7writelnTAyaZ7w itelnFAyaZv]+0x1c): undef ref `_D3std5stdio6stdoutS3std5stdio4File' (.text._D3std5stdio16__T7writelnTAyaZ7writelnFAyaZv[_D3std5stdio16__T7writelnTAyaZ7w itelnFAyaZv]+0x41): undef ref `fprintf' (.text._D3std5stdio16__T7writelnTAyaZ7writelnFAyaZv[_D3std5stdio16__T7writelnTAyaZ7w itelnFAyaZv]+0x81): undef ref `_D3std9exception14__T7enforceTbZ7enforceFNaNfbLAxaAyamZb' In function `_Dmain': (.text._Dmain+0x36): undef ref `memset' (_D27TypeInfo_S4main10TestStruct6__initZ[_D27TypeInfo_S4main10TestSt uct6__initZ]+0x70): undef ref `_D10TypeInfo_g6__initZ' (_D4main9TestClass6__vtblZ+0x8): undef ref `_D6object6Object8toStringMFZAya' (_D4main9TestClass6__vtblZ+0x10): undef ref `_D6object6Object6toHashMFNbNeZm' (_D4main9TestClass6__vtblZ+0x18): undef ref `_D6object6Object5opCmpMFC6ObjectZi' (_D4main9TestClass6__vtblZ+0x20): undef ref `_D6object6Object8opEqualsMFC6ObjectZb' (_D12TypeInfo_Axm6__initZ[_D12TypeInfo_Axm6__initZ]+0x0): undef ref `_D14TypeInfo_Array6__vtblZ' (_D11TypeInfo_xm6__initZ[_D11TypeInfo_xm6__initZ]+0x10): undef ref `_D10TypeInfo_m6__initZ' (_D11TypeInfo_ya6__initZ[_D11TypeInfo_ya6__initZ]+0x10): undef ref `_D10TypeInfo_a6__initZ' (_D12TypeInfo_xAa6__initZ[_D12TypeInfo_xAa6__initZ]+0x10): undef ref `_D12TypeInfo_Axa6__initZ' (_D11TypeInfo_xb6__initZ[_D11TypeInfo_xb6__initZ]+0x10): undef ref `_D10TypeInfo_b6__initZ' (_D11TypeInfo_xi6__initZ[_D11TypeInfo_xi6__initZ]+0x10): undef ref `_D10TypeInfo_i6__initZ' (_D11TypeInfo_xa6__initZ[_D11TypeInfo_xa6__initZ]+0x10): undef ref `_D10TypeInfo_a6__initZ' (_D11TypeInfo_xh6__initZ[_D11TypeInfo_xh6__initZ]+0x10): undef ref `_D10TypeInfo_h6__initZ' (_D11TypeInfo_xk6__initZ[_D11TypeInfo_xk6__initZ]+0x10): undef ref `_D10TypeInfo_k6__initZ' (_D13TypeInfo_xAya6__initZ[_D13TypeInfo_xAya6__initZ]+0x10): undef ref `_D12TypeInfo_Aya6__initZ' This is far more than what I get with DMD and GDC. You can see why TypeInfo is a major pain, and as far as I can tell, I don't have any code that needs it. But what's most surprising is the call to fprintf for std.writeln, even though my implementation is actually a system call in assembly: https://github.com/JinShil/minimal_druntime_experiment/blob/master/source/ports/posix/linux/phobos/phobosPort.d#L6 I'm compiling with: ldc2 -c -defaultlib= -release Are there any compiler flags I can add to help trim the fat? Thanks, Mike
May 06 2015
parent reply Dan Olson <zans.is.for.cans yahoo.com> writes:
"Mike" <none none.com> writes:
 I've gotten even further thanks to everyone's help, but LDC is still
 giving me a little grief.  Take a look at these undefined references
 (abbreviated for this forum):

 (_D10TypeInfo_l6__vtblZ+0x8): undef ref
 `_D6object8TypeInfo8toStringMxFNaNbNfZAya'
-- snip --
 Are there any compiler flags I can add to help trim the fat?
Mike, try creating an empty ldc2.conf file where you compile from (I just tried so it should work). That will override the one that came with your ldc2 installation that has searches the real druntime/phobos before your versions. It looks like there will be an option -conf with 0.16.0 to override the default conf file. https://github.com/ldc-developers/ldc/issues/879 You can put -v on your command ldc2 command line to see where it is picking up files.
May 07 2015
parent reply "Mike" <none none.com> writes:
On Thursday, 7 May 2015 at 07:00:58 UTC, Dan Olson wrote:

 Mike, try creating an empty ldc2.conf file where you compile 
 from (I
 just tried so it should work). That will override the one that 
 came with
 your ldc2 installation that has searches the real 
 druntime/phobos before
 your versions. It looks like there will be an option -conf with 
 0.16.0
 to override the default conf file.
That did it! But I thought defaultLib= did the same thing. Anyway, Thanks for all your help, Dan. I now have support for all 3 compilers porting to 2 different platforms (one bare metal microcontroller, one modern OS). It can't do anything more than "Hello, World!", but it's a start :-) Mike
May 07 2015
parent Dan Olson <zans.is.for.cans yahoo.com> writes:
"Mike" <none none.com> writes:
 That did it!  But I thought defaultLib= did the same thing.  Anyway,
 Thanks for all your help, Dan.

 I now have support for all 3 compilers porting to 2 different
 platforms (one bare metal microcontroller, one modern OS).  It can't
 do anything more than "Hello, World!", but it's a start :-)
You may not have h/w to test this on, but support for x86_64 OSX could be added easily. Syscall assembly is same as linux x86_64, just the syscall # is different: exit rax=0x2000001 write rax=0x2000004 I just tried in a test D program and it worked. Anyway, probably little value to have OSX here as a platform. I think interesting platforms will be no OS, where end user plugs in their write handler to write fd 1,2 to serial port or to go through gdbstub protocol over serial, or ... -- Dan
May 07 2015
prev sibling next sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Tue, 05 May 2015 02:26:26 +0000
schrieb "Mike" <none none.com>:

 Read on GitHub:=20
 https://github.com/JinShil/minimal_druntime_experiment
=20
 There was recently a discussion about how we could create a=20
 portable, pay-as-you-go, D runtime to help bring the promise of D=20
 to free-standing platforms and devices with tight resource=20
 constraints (e.g. microcontrollers).  Thread started here: =20
 http://forum.dlang.org/post/mhrs4p$31id$1 digitalmars.com
=20
 The primary motivation is to create an arm-none-eabi GDC=20
 cross-compiler toolchain for programming ARM Cortex-M=20
 microcontrollers in D, but I think there's a way to achieve=20
 broader platform support by delegating implementation details=20
 down the "supply chain".  I hope to articulate that strategy in=20
 this post.
=20
 To prove the concept, provide a place to start, and discuss=20
 ideas, I created the most minimal D runtime I could: =20
 https://github.com/JinShil/minimal_druntime_experiment.  I've=20
 also included Phobos, so we could still have `std.stdio.write`=20
 and `std.stdio.writeln` for console output, as every device needs=20
 a console for development.
=20
 Overview
 **********
 d
 =E2=94=9C=E2=94=80=E2=94=80 phobos
 =E2=94=82   =E2=94=94=E2=94=80=E2=94=80 std
 =E2=94=94=E2=94=80=E2=94=80 runtime
      =E2=94=94=E2=94=80=E2=94=80 rt
          =E2=94=94=E2=94=80=E2=94=80 typeinfo
=20
 ports
 =E2=94=9C=E2=94=80=E2=94=80 arm
 =E2=94=82   =E2=94=94=E2=94=80=E2=94=80 cortexm4
 =E2=94=82       =E2=94=9C=E2=94=80=E2=94=80 phobosWe
 =E2=94=82       =E2=94=82   =E2=94=94=E2=94=80=E2=94=80 phobosPort.d
 =E2=94=82       =E2=94=94=E2=94=80=E2=94=80 runtime
 =E2=94=82           =E2=94=94=E2=94=80=E2=94=80 runtimePort.d
 =E2=94=9C=E2=94=80=E2=94=80 posix
 =E2=94=82   =E2=94=94=E2=94=80=E2=94=80 linux
 =E2=94=82       =E2=94=9C=E2=94=80=E2=94=80 phobos
 =E2=94=82       =E2=94=82   =E2=94=94=E2=94=80=E2=94=80 phobosPort.d
 =E2=94=82       =E2=94=94=E2=94=80=E2=94=80 runtime
 =E2=94=82           =E2=94=9C=E2=94=80=E2=94=80 c_types.d
 =E2=94=82           =E2=94=94=E2=94=80=E2=94=80 runtimePort.d
=20
=20
 There are two main folders: "d" and "ports".  "d" provides the=20
 patform-agnostic code, or code that is relevant to a large number=20
 of platforms.  The "ports" directory provides the=20
 platform-specific implementation.  Building simply requires=20
 importing "d/runtime", "d/phobos", and your platform's hierarchy=20
 in the "ports" folder.  At the moment, I've only implemented a=20
 Linux 64-bit port and an ARM Cortex-M4 port to illustrate the=20
 concept.  This is roughly how I wish the official runtime could=20
 be structured in the spirit of Issue 11666.
=20
 The official runtime includes platform-specific bindings in=20
 `core.sys` and `stdc`, but I think that is a design anomaly. =20
 While a port may find it convenient to use those bindings, they=20
 should not be part of the D language's public API.  Rather, they=20
 should be deported to Deimos, and if needed, imported privately=20
 by a platform's port.  For the Linux port, I chose to write the=20
 platform's system calls in assembly to illustrate that it is not=20
 even necessary to use those bindings if one wants to do the due=20
 diligence in D.
=20
 Porting to a New Platform
 ******************************
 The platform-agnostic code in "d" delgates implementation details=20
 to the platform-specific code using `extern(C) extern=20
 _d_sys_name` "system calls" (for lack of a better term).  This is=20
 how the newlib C library delegates platform-specific=20
 implementations:  See=20
 http://wiki.osdev.org/Porting_Newlib#newlib.2Flibc.2Fsys.2Fmyos.2Fsyscall=
s.c
=20
 At the moment, for the sake of demonstration, only 2 system calls=20
 have been defined:
=20
 * `__d_sys_write` - Equivalent to C's `write` system call
 * `__d_sys_exit` - Equivalent to C's `exit` system call
=20
 These two system calls allow us to create a simple Hello World. =20
 "runtimePort.d" implements `__d_sys_exit` and "phobosPort.d"=20
 implements the `__d_sys_write`.
=20
 Putting it all Together
 ************************
 Users are not expected to use this code directly.  Rather, I=20
 envision toolchain, silicon, and board vendors will use this code=20
 as a small, but essential part of, their platform's D programming=20
 package.  For example, a package for the ARM Cortex-M family of=20
 MCUs might contain the following:
 * arm-none-eabi GDC cross-compiler
 * a library containing compiled code for the "d" folder in this=20
 repository
 * a library containing the compiled code for the=20
 "ports/arm/cortexm" folders in this repository
 * cortex-m core startup files
 * the newlib C library
 * the C library bindings from Deimos
 * multilibs from the GNU toolchain.
=20
 A silicon vendor like ST, may then take that package and add more=20
 platform-specific for their family of products:
 * startup files with interrupt vectors for their peripherals
 * linker scripts for each of their MCUs
 * flash memory programmer
 * library for on-dye peripherals
 * etc..
=20
 A board vendor may choose to create yet another layer on top of=20
 the silicon vendor's package.
 * library for peripherals external to the MCU (external RAM, IO=20
 expanders, gyroscope, LCD, etc...)
=20
 In short, this repository just provides just the foundation to=20
 "get D working", and delegates to toolchain, silicon, and board=20
 vendors to fill in the details for their products.
=20
 RFC
 *****
 For those who have stake in this code base (kernel developers,=20
 embedded developers, toolchain and package maintainers, etc...)=20
 your constructive criticism and ideas are most welcome.  I am=20
 unsure yet how this will play out, and what roles players will=20
 assume, but let's give it a try.
=20
I would probably split the runtime into at least two, probably three parts: * Compiler support library (object.d, gcc/*.d, exception implementation) (module rt/ no module name) * Higher level library (portable, micro) * Hardware specific library (not portable, avr/ stm/) I think some basic portability between microcontrollers is useful. The compiler support library should be only a few 100 lines of code and should be without external dependencies. This way it's also useful for very small platforms (8bit) or use-cases where you inject code into other processes. It's trivial to port: we should have an ansi-c port and native ports should be implementable in 1-2 hours. The higher level library should contain stuff like 'core.time.Duration',=20 emplace, allocator framework, Volatile!T, register wrapper types, etc. The implementation could be hardware specific, but it should provide a common interface (e.g. a delay!Duration function has got a common interface but different implementation, malloc is similar). Namespace micro =3D> import micro.time; import micro.memory; So everything in the micro namespace is portable. The hardware library the provides access to hardware specific peripherals. In reality the high-level-library might require the hardware library or they could actually be merged. The important part is portable vs platform specific module API for user code. A radically different approach instead of using ports directories is using git features: Have a base repository with a master branch which only includes the interfaces, probably with static assert wherever platform specific code is necessary. Then have AVR/STM32/LPC/... branches where you simply implement these functions. (We could also have different repositories instead of branches) + you can modify all code + you already start with a common interface + changes are easy to compare by comparing git branches + changes have descriptions (in their git commits) + it's easy to merge further generic changes Phobos and core.stdc should then also be separate libraries.
 Plea to Compiler Implementers
 ***********************************
 We need better control over codegen.  TypeInfo and dead-code=20
 removal is preventing me from making progress=20
 (http://forum.dlang.org/post/quemhwpgijwmqtpxukiv forum.dlang.org).=20
   I've resorted to compiling to assembly, using sed to hack the=20
 assembly, and then compiling the assembly.  Things like that make=20
 me want to not use D at all.  Here are some ideas from myself and=20
 others:
 *  Move TypeInfo to the runtime:=20
 https://issues.dlang.org/show_bug.cgi?id=3D12270
 *  Move D runtime hooks out of the compiler and into *.di files=20
 so they can be decorated with attributes and `version`ed so the=20
 compiler can see what features of D are supported by the port and=20
 generate smarter code: =20
 http://forum.dlang.org/post/psssnzurlzeqeneagora forum.dlang.org
 *  Add -fno-rtti compiler switch
This is something I could probably finish up in a few hours next weekend.
 *  Add attribute support so programmer can choose which types to=20
 generate runtime-time info for.
I can add an attribute(notypeinfo) in GDC but it'll be implemented in the backend. This means we can prevent TypeInfo output but we can't reliably warn on TypeInfo usage. You'll get linker errors instead.
May 05 2015
next sibling parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 5 May 2015 at 19:38, Johannes Pfau via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 I would probably split the runtime into at least two, probably three
 parts:

 * Compiler support library (object.d, gcc/*.d, exception
   implementation) (module rt/ no module name)
 * Higher level library (portable, micro)
 * Hardware specific library (not portable, avr/ stm/)

 I think some basic portability between microcontrollers is useful. The
 compiler support library should be only a few 100 lines of code and
 should be without external dependencies. This way it's also useful for
 very small platforms (8bit) or use-cases where you inject code into
 other processes. It's trivial to port: we should have an ansi-c port
 and native ports should be implementable in 1-2 hours.
It should also be possible for compilations to succeed without rt/typeinfo - and for the compiler to not assume they exist (Type::builtinTypeInfo).
May 05 2015
parent "Mike" <none none.com> writes:
On Tuesday, 5 May 2015 at 18:13:03 UTC, Iain Buclaw wrote:

 It should also be possible for compilations to succeed without
 rt/typeinfo - and for the compiler to not assume they exist
 (Type::builtinTypeInfo).
That would also be most welcome.
May 05 2015
prev sibling parent reply "Mike" <none none.com> writes:
On Tuesday, 5 May 2015 at 17:38:38 UTC, Johannes Pfau wrote:

 I would probably split the runtime into at least two, probably 
 three
 parts:

 * Compiler support library (object.d, gcc/*.d, exception
   implementation) (module rt/ no module name)
 * Higher level library (portable, micro)
 * Hardware specific library (not portable, avr/ stm/)

 I think some basic portability between microcontrollers is 
 useful. The
 compiler support library should be only a few 100 lines of code 
 and
 should be without external dependencies. This way it's also 
 useful for
 very small platforms (8bit) or use-cases where you inject code 
 into
 other processes. It's trivial to port: we should have an ansi-c 
 port
 and native ports should be implementable in 1-2 hours.


 The higher level library should contain stuff like 
 'core.time.Duration',
 emplace, allocator framework, Volatile!T, register wrapper 
 types, etc.
 The implementation could be hardware specific, but it should 
 provide a
 common interface (e.g. a delay!Duration function has got a 
 common
 interface but different implementation, malloc is similar).
 Namespace micro => import micro.time; import micro.memory;
 So everything in the micro namespace is portable.


 The hardware library the provides access to hardware specific
 peripherals. In reality the high-level-library might require the
 hardware library or they could actually be merged. The 
 important part
 is portable vs platform specific module API for user code.
Sorry for the late reply, but I'm still chewing and digesting this. How do you propose delegating implementation down the supply chain? I suppose you may have answered that with your git idea below, but did you have something else in mind? I think that's really what I'm trying to work out with this experiment: How to delegate the implementation so it's obvious what porters need to do without a lot of explanation, and they can implement just the features they need.
 A radically different approach instead of using ports 
 directories is
 using git features: Have a base repository with a master branch 
 which
 only includes the interfaces, probably with static assert 
 wherever
 platform specific code is necessary. Then have AVR/STM32/LPC/...
 branches where you simply implement these functions. (We could 
 also
 have different repositories instead of branches)

 + you can modify all code
 + you already start with a common interface
 + changes are easy to compare by comparing git branches
 + changes have descriptions (in their git commits)
 + it's easy to merge further generic changes
I'll have to think about this and give it a try. I fear, however, that it might be a little too radical for what people are used to, but maybe not.
 Phobos and core.stdc should then also be separate libraries.
Agreed.
 *  Add -fno-rtti compiler switch
This is something I could probably finish up in a few hours next weekend.
That would be an immense help and productivity boost. Thanks for your support.
 *  Add attribute support so programmer can choose which types 
 to generate runtime-time info for.
I can add an attribute(notypeinfo) in GDC but it'll be implemented in the backend. This means we can prevent TypeInfo output but we can't reliably warn on TypeInfo usage. You'll get linker errors instead.
Probably a good idea to table that for now. Mike
May 07 2015
parent reply Johannes Pfau <nospam example.com> writes:
Am Thu, 07 May 2015 11:52:29 +0000
schrieb "Mike" <none none.com>:

 On Tuesday, 5 May 2015 at 17:38:38 UTC, Johannes Pfau wrote:
 
 I would probably split the runtime into at least two, probably 
 three
 parts:

 * Compiler support library (object.d, gcc/*.d, exception
   implementation) (module rt/ no module name)
 * Higher level library (portable, micro)
 * Hardware specific library (not portable, avr/ stm/)

 I think some basic portability between microcontrollers is 
 useful. The
 compiler support library should be only a few 100 lines of code 
 and
 should be without external dependencies. This way it's also 
 useful for
 very small platforms (8bit) or use-cases where you inject code 
 into
 other processes. It's trivial to port: we should have an ansi-c 
 port
 and native ports should be implementable in 1-2 hours.


 The higher level library should contain stuff like 
 'core.time.Duration',
 emplace, allocator framework, Volatile!T, register wrapper 
 types, etc.
 The implementation could be hardware specific, but it should 
 provide a
 common interface (e.g. a delay!Duration function has got a 
 common
 interface but different implementation, malloc is similar).
 Namespace micro => import micro.time; import micro.memory;
 So everything in the micro namespace is portable.


 The hardware library the provides access to hardware specific
 peripherals. In reality the high-level-library might require the
 hardware library or they could actually be merged. The 
 important part
 is portable vs platform specific module API for user code.
Sorry for the late reply, but I'm still chewing and digesting this. How do you propose delegating implementation down the supply chain? I suppose you may have answered that with your git idea below, but did you have something else in mind?
Mostly the git idea. It makes sense to keep the 'customization' points to a minimum and have well-defined hooks like in newlib and like you suggested. But the git branch way also allows modifying code everywhere, in case the generic hooks don't fit one platform for some reason.
 I think that's really what I'm trying to work out with this 
 experiment:  How to delegate the implementation so it's obvious 
 what porters need to do without a lot of explanation, and they 
 can implement just the features they need.
Let me explain the git idea: core library; branch 'common': -------------------------------------------- object.d: class Object { } void traceLine(string msg) { static assert(false, "Not implemented"); } -------------------------------------------- branch avr -------------------------------------------- object.d: class Object { } version(none) // Not implemented for AVRs right now { void traceLine(string msg) { static assert(false, "Not implemented"); } } -------------------------------------------- branch nintendo-ds -------------------------------------------- object.d: class Object { //Lots of memory, let's add some useful methods void toString(scope delegate(const(char)[] buf)) { ... } } void traceLine(string msg) { asm (print to port xyz...) } -------------------------------------------- Porters simply grep for '"Not implemented"' and implement the function or comment/remove it. If we now change the interface in the common branch: void traceLine(string msg) => void traceLine(string msg, int level = 0) Both branches will get the changes or a merge conflict by simple merging the common branch. This way we can keep a kinda common interface and have lots of customization possibilities. Note 1: The core lib might be a bad example. It should be mostly portable and have little user-facing code so keeping a common interface is not important here. It's more useful for a high-level library (e.g. keep emplace/allocator interface compatible while still allowing custom malloc or other allocator implementations) Note 2: Even a simple 'traceLine' function depends on board definitions. So a port targeting many boards could not even implement a 'traceLine' in the corlib. It would have to delegate the implementation to the hardware library* effectively tying the core library to the hardware library. I think this is sometimes unavoidable for low-level code but this should probably be a decision porters make. The 'common' corlib should not depend on other libraries. *or even user code. Note 3: It could still make sense to keep all the 'to-be-implemented' functions/hooks in a special module or package. But instead of having ports/arm/ ports/avr we could have one port/*.d with template files as described above and the actual platform specific implementations in branches.
 A radically different approach instead of using ports 
 directories is
 using git features: Have a base repository with a master branch 
 which
 only includes the interfaces, probably with static assert 
 wherever
 platform specific code is necessary. Then have AVR/STM32/LPC/...
 branches where you simply implement these functions. (We could 
 also
 have different repositories instead of branches)

 + you can modify all code
 + you already start with a common interface
 + changes are easy to compare by comparing git branches
 + changes have descriptions (in their git commits)
 + it's easy to merge further generic changes
I'll have to think about this and give it a try. I fear, however, that it might be a little too radical for what people are used to, but maybe not.
Many newlib ports are not part of official newlib but custom forks AFAIK. There's no big difference here (instead of branches we could also use different repos). By using git we make maintaining custom changes much simpler. But it is indeed a rather radical approach and there could certainly be some drawbacks as well. (Even if we define formal hooks and a port/ directory structure porters could of course still fork&modify the code instead. They could probably use the techniques described here without our explicit support. But if we decide to go this way, we can simplify _our_ code. No need to have lots of hooks or version(ARM) else version(AVR) else version() or a port/ directory structure. Every port is in its own branch)
May 07 2015
parent reply "Jens Bauer" <doctor who.no> writes:
On Thursday, 7 May 2015 at 16:12:36 UTC, Johannes Pfau wrote:
{snip}
 Let me explain the git idea:
I think it's a great idea. {snip}
 Porters simply grep for '"Not implemented"' and implement the 
 function
This sounds easy. {snip}
 (Even if we define formal hooks and a port/ directory structure 
 porters
 could of course still fork&modify the code instead. They could 
 probably
 use the techniques described here without our explicit support.
That was the only 'downside' I could think of; eg. if I wanted to develop firmware for both the LPC and STM series, switching branch for my repository all the time would not be ideal. ;) -But of course, it can be solved easily by making a repository, which contains a "read-only" directory structure of all branches (or something similar). -Such a repository could be generated automatically and tagged by a script very easily. {snip}
 Every port is in its own branch
In this case, the files must share the same parent directory, in order to be updated by a merge with master; correct ?
May 07 2015
parent Johannes Pfau <nospam example.com> writes:
Am Thu, 07 May 2015 16:29:50 +0000
schrieb "Jens Bauer" <doctor who.no>:

 
 {snip}
 Every port is in its own branch
In this case, the files must share the same parent directory, in order to be updated by a merge with master; correct ?
IIRC it's not always necessary. Git can detect if you moved files or you can tell git by using git mv. It'll then merge changes into the moved files as well*. But I wouldn't really want to rely on this 'feature'. * (in GDC it always tries to merge gcc-4.10.diff changes into gcc-4.9.diff which never works...)
May 07 2015
prev sibling next sibling parent reply "Kagamin" <spam here.lot> writes:
On Tuesday, 5 May 2015 at 02:26:28 UTC, Mike wrote:
 Porting to a New Platform
 ******************************
 The platform-agnostic code in "d" delgates implementation 
 details to the platform-specific code using `extern(C) extern 
 _d_sys_name` "system calls" (for lack of a better term).
You plan to have a sizable API without type safety? Why PAL is not good?
May 07 2015
parent reply "Mike" <none none.com> writes:
On Thursday, 7 May 2015 at 09:55:11 UTC, Kagamin wrote:
 On Tuesday, 5 May 2015 at 02:26:28 UTC, Mike wrote:
 Porting to a New Platform
 ******************************
 The platform-agnostic code in "d" delgates implementation 
 details to the platform-specific code using `extern(C) extern 
 _d_sys_name` "system calls" (for lack of a better term).
You plan to have a sizable API without type safety? Why PAL is not good?
The "ports" folder in this experiment is essentially the platform abstraction layer. However, it's resolved at link-time. What do you suggest: A reserved module/package implementing a standard interface that is imported at compile-time? I could do that. Mike
May 07 2015
next sibling parent "Kagamin" <spam here.lot> writes:
On Friday, 8 May 2015 at 02:25:48 UTC, Mike wrote:
 What do you suggest:  A reserved module/package implementing a 
 standard interface that is imported at compile-time?  I could 
 do that.
You just import the right PAL (maybe renamed private import) and everything is typechecked.
May 08 2015
prev sibling parent "Jens Bauer" <doctor who.no> writes:
On Friday, 8 May 2015 at 02:25:48 UTC, Mike wrote:
 The "ports" folder in this experiment is essentially the 
 platform abstraction layer.  However, it's resolved at 
 link-time.  What do you suggest:  A reserved module/package 
 implementing a standard interface that is imported at 
 compile-time?  I could do that.
I'd like to help out (where I can). As far as I remember, you have a STM32F4xx, correct ? If we all have the same model STM32F4xx and we all have one or more different MCUs, we could probably do some initial drafts. -Because I'm sure there are obstacles we do not think about in advance. Testing some drafts on several MCUs in the beginning, will help us avoid a lot of trouble later on. I have STM32F407, STM32F427, STM32F429 (I use these regularly). On the shelf, I have LPC812, LPC1114, LPC1342 (seems to be dead though), LPC1549 (I don't have any libraries for this one yet), LPC1751, LPC1768, LPC1788 and LPC4337. Though I have a few from Freescale, I am not able to flash-program them, as I have no driver for them in OpenOCD.
May 09 2015
prev sibling parent "Jens Bauer" <doctor who.no> writes:
On Tuesday, 5 May 2015 at 02:26:28 UTC, Mike wrote:
{snip}
 Putting it all Together
 ************************
 Users are not expected to use this code directly.  Rather, I 
 envision toolchain, silicon, and board vendors will use this 
 code as a small, but essential part of, their platform's D 
 programming package.  For example, a package for the ARM 
 Cortex-M family of MCUs might contain the following:
 * arm-none-eabi GDC cross-compiler
 * a library containing compiled code for the "d" folder in this 
 repository
 * a library containing the compiled code for the 
 "ports/arm/cortexm" folders in this repository
I'd like to insert CMSIS here. CMSIS should be shared between all Cortex-M platforms. My impression is that it's fairly easy to do; CMSIS also provides convenience functions like WFI(), WFE(), ROR(), NOP(), CLZ(), REV(), RBIT(), etc... Note: It might be a good idea to add these as well: CLO(), CTZ() and CTO()
 * cortex-m core startup files
 * the newlib C library
 * the C library bindings from Deimos
 * multilibs from the GNU toolchain.
{snip}
May 07 2015