www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - LWDR (Light Weight D Runtime) for Microcontrollers v0.2.3

reply Dylan Graham <dylan.graham2000 gmail.com> writes:
Github: https://github.com/0dyl/LWDR
DUB: https://code.dlang.org/packages/lwdr

Hi, all!
This is LWDR (Light Weight D Runtime) It is a ground-up 
implementation of a D runtime targeting the ARM Cortex-M 
microcontrollers and other microcontroller platforms with RTOSes 
(Real Time Operating Systems).

It doesn't, and possibly may not, support all D features in order 
to make it viable for the constrained environments. For example, 
all memory allocation is manually done via `new` and `delete` - 
no GC.

It works by providing a series of barebones API hooks (alloc, 
dealloc, assert, etc) (defined in `rtoslink.d`), which you must 
implement and/or point to your RTOS implementation.

It can be compiled with either GDC or LDC and it is DUB 
compatible.

It has so far been successfully run on a real STM32F407.

LWDR currently supports the following language features:
- Class allocations and deallocations (via new and delete)
- Struct heap allocations and deallocations (via new and delete)
- Invariants
- Asserts
- Contract programming
- Basic RTTI (via TypeInfo stubs)
- Interfaces
- Static Arrays
- Virtual functions and overrides
- Abstract classes
- Static classes
- Allocation and deallocation of dynamic arrays
- Concatenate an item to a dynamic array
- Concatenate two dynamic arrays together
- Dynamic array resizing

The following features are experimental:
- Exceptions and Throwables (so far are working on GDC only)

Not supported:
- Module constructors and destructors
- ModuleInfo
- There is no GC implementation
- TLS (thread local static) variables
- Delegates/closures
- Associative arrays
- Shared/synchronized
- Object hashing
- Other stuff I have forgotten :(

It is beta, so expect bugs.
May 30 2021
next sibling parent Dylan Graham <dylan.graham2000 gmail.com> writes:
On Sunday, 30 May 2021 at 14:28:25 UTC, Dylan Graham wrote:

 Github: https://github.com/0dyl/LWDR
 DUB: https://code.dlang.org/packages/lwdr


I added a Wiki tutorial on compiling with LDC and DUB (which is 
how I currently test LWDR). It's about 12:53 AM AEST, so I'm 
heading to bed. I plan on adding more usage and compilation 
tutorials in the future.

I've made a tonne of progress on this project in just a few 
hours, so I'm pretty happy, I hope you'll like it too!
May 30 2021
prev sibling next sibling parent reply Denis Feklushkin <feklushkin.denis gmail.com> writes:
Nice job!

Are you tried compile apps with Phobos?
May 30 2021
parent Dylan Graham <dylan.graham2000 gmail.com> writes:
On Sunday, 30 May 2021 at 15:07:54 UTC, Denis Feklushkin wrote:

 Nice job!

 Are you tried compile apps with Phobos?


Thank you!

No, I haven't tried any of Phobos yet. It should work, but will 
leak like a sieve.

I need to develop a solution that tracks memory allocations and 
exposes a simplified interface to the user for deletion.
May 30 2021
prev sibling next sibling parent reply Guillaume Piolat <first.last gmail.com> writes:
On Sunday, 30 May 2021 at 14:28:25 UTC, Dylan Graham wrote:

 LWDR currently supports the following language features:
 - Class allocations and deallocations (via new and delete)
 - Struct heap allocations and deallocations (via new and delete)
 - Invariants
 - Asserts
 - Contract programming
 - Basic RTTI (via TypeInfo stubs)
 - Interfaces
 - Static Arrays
 - Virtual functions and overrides
 - Abstract classes
 - Static classes
 - Allocation and deallocation of dynamic arrays
 - Concatenate an item to a dynamic array
 - Concatenate two dynamic arrays together
 - Dynamic array resizing

 The following features are experimental:
 - Exceptions and Throwables (so far are working on GDC only)



 It is beta, so expect bugs.


Nice job!
May 30 2021
parent Dylan Graham <dylan.graham2000 gmail.com> writes:
On Sunday, 30 May 2021 at 15:35:34 UTC, Guillaume Piolat wrote:

 On Sunday, 30 May 2021 at 14:28:25 UTC, Dylan Graham wrote:

 LWDR currently supports the following language features:
 - Class allocations and deallocations (via new and delete)
 - Struct heap allocations and deallocations (via new and 
 delete)
 - Invariants
 - Asserts
 - Contract programming
 - Basic RTTI (via TypeInfo stubs)
 - Interfaces
 - Static Arrays
 - Virtual functions and overrides
 - Abstract classes
 - Static classes
 - Allocation and deallocation of dynamic arrays
 - Concatenate an item to a dynamic array
 - Concatenate two dynamic arrays together
 - Dynamic array resizing

 The following features are experimental:
 - Exceptions and Throwables (so far are working on GDC only)



 It is beta, so expect bugs.


 Nice job!


Thank you all for a kind words :D
May 30 2021
prev sibling next sibling parent Paulo Pinto <pjmlp progtools.org> writes:
On Sunday, 30 May 2021 at 14:28:25 UTC, Dylan Graham wrote:

 Github: https://github.com/0dyl/LWDR
 DUB: https://code.dlang.org/packages/lwdr

 Hi, all!
 This is LWDR (Light Weight D Runtime) It is a ground-up 
 implementation of a D runtime targeting the ARM Cortex-M 
 microcontrollers and other microcontroller platforms with 
 RTOSes (Real Time Operating Systems).

 It doesn't, and possibly may not, support all D features in 
 order to make it viable for the constrained environments. For 
 example, all memory allocation is manually done via `new` and 
 `delete` - no GC.

 It works by providing a series of barebones API hooks (alloc, 
 dealloc, assert, etc) (defined in `rtoslink.d`), which you must 
 implement and/or point to your RTOS implementation.

 It can be compiled with either GDC or LDC and it is DUB 
 compatible.

 It has so far been successfully run on a real STM32F407.

 LWDR currently supports the following language features:
 - Class allocations and deallocations (via new and delete)
 - Struct heap allocations and deallocations (via new and delete)
 - Invariants
 - Asserts
 - Contract programming
 - Basic RTTI (via TypeInfo stubs)
 - Interfaces
 - Static Arrays
 - Virtual functions and overrides
 - Abstract classes
 - Static classes
 - Allocation and deallocation of dynamic arrays
 - Concatenate an item to a dynamic array
 - Concatenate two dynamic arrays together
 - Dynamic array resizing

 The following features are experimental:
 - Exceptions and Throwables (so far are working on GDC only)

 Not supported:
 - Module constructors and destructors
 - ModuleInfo
 - There is no GC implementation
 - TLS (thread local static) variables
 - Delegates/closures
 - Associative arrays
 - Shared/synchronized
 - Object hashing
 - Other stuff I have forgotten :(

 It is beta, so expect bugs.


Great work!
May 30 2021
prev sibling next sibling parent reply Dukc <ajieskola gmail.com> writes:
On Sunday, 30 May 2021 at 14:28:25 UTC, Dylan Graham wrote:

 Hi, all!
 This is LWDR (Light Weight D Runtime) It is a ground-up 
 implementation of a D runtime targeting the ARM Cortex-M 
 microcontrollers and other microcontroller platforms with 
 RTOSes (Real Time Operating Systems).


Sounds very useful! However, first thing first: What's the 
license? DUB package says it's FOSS. Great, but what kind of 
FOSS? It makes a big difference whether it's GNU or BSD, for 
instance.


 It doesn't, and possibly may not, support all D features in 
 order to make it viable for the constrained environments. For 
 example, all memory allocation is manually done via `new` and 
 `delete` - no GC.


Regarding `new` - is there a good way to iterate though chunks 
allocated with `new`? One could call an unmodified piece of D 
code that normally uses the GC, and then manually free all it's 
allocations.


 It works by providing a series of barebones API hooks (alloc, 
 dealloc, assert, etc) (defined in `rtoslink.d`), which you must 
 implement and/or point to your RTOS implementation.


Quickly looking, the implementation looks very portable, save for 
exceptions. with `rtoslink.d`, this will probably enable a lot of 
stuff on any platform without DRuntime. Not just 
microcontrollers. If I'm right, you just did a BIG service for D 
on bare-metal.


 It is beta, so expect bugs.


And open source, so the bugs can be fixed as discovered :-). 
Thanks for the warning anyway.
May 30 2021
parent reply Dylan Graham <dylan.graham2000 gmail.com> writes:
On Sunday, 30 May 2021 at 17:31:37 UTC, Dukc wrote:

 On Sunday, 30 May 2021 at 14:28:25 UTC, Dylan Graham wrote:

 Hi, all!
 This is LWDR (Light Weight D Runtime) It is a ground-up 
 implementation of a D runtime targeting the ARM Cortex-M 
 microcontrollers and other microcontroller platforms with 
 RTOSes (Real Time Operating Systems).


 Sounds very useful! However, first thing first: What's the 
 license? DUB package says it's FOSS. Great, but what kind of 
 FOSS? It makes a big difference whether it's GNU or BSD, for 
 instance.


I haven't put any thought into the license. Since LWDR is derived 
from DRuntime, I assume I'll have to use its license. If not, I'd 
like to go with something permissive like MIT.


 It doesn't, and possibly may not, support all D features in 
 order to make it viable for the constrained environments. For 
 example, all memory allocation is manually done via `new` and 
 `delete` - no GC.


 Regarding `new` - is there a good way to iterate though chunks 
 allocated with `new`? One could call an unmodified piece of D 
 code that normally uses the GC, and then manually free all it's 
 allocations.


Bingo. I was thinking of thread-local tracking of allocations. 
Something like follows:

Impl:
```D
size_t numAllocations = 0;
List!(void*) allocations;

     void* allocateMemoryForNew(int size)
     {
         numAllocations++;
         auto ptr = rtosbackend_heapalloc(size);
         allocations.add(ptr);
         return allocations;
     }
     void deallocateMemory(void* ptr)
     {
         allocations.removeLastOccurenceOf(ptr);
         numAllocations--;
         rtosbackend_heapfreealloc(ptr);
     }

     struct MemAlloc
     {
         size_t allocs;

         void free() {
             // allocs should be less than numAllocations,
             // so delete the last n items of allocations

             auto difference = numAllocations - allocs;
             foreach(i; 0 .. difference) {
                 auto ptr = allocations[allocations.length - i]; 
// get ith to last
                 deallocateMemory(ptr);
             }
         }
     }

     MemAlloc enterTrackedMemory()
     {
         return MemAlloc(.numAllocations);
     }
```

Usage:
```D
auto mem0 = enterTrackedMemory(); // mem0 says that there should 
be 0 allocations
A a = new A(); // 1 allocation
{
     auto mem1 = enterTrackedMemory(); // mem1 says that there 
should be 1 allocations total
     auto b = new A(); // 1 allocation, so total is now 2
     mem1.free; // we need to get back to 1 allocation, so delete 
b (most recent allocation)
}
mem0.free; // delete everything
```

This works with `scope(...)`.

There's some caveats with this - if a phobos function calls a 
user function, anything that user function allocates will be 
wiped, too (may or may not be desirable behaviour), unless some 
protocol for opting out of tracking is implemented.


 It works by providing a series of barebones API hooks (alloc, 
 dealloc, assert, etc) (defined in `rtoslink.d`), which you 
 must implement and/or point to your RTOS implementation.


 Quickly looking, the implementation looks very portable, save 
 for exceptions. with `rtoslink.d`, this will probably enable a 
 lot of stuff on any platform without DRuntime. Not just 
 microcontrollers. If I'm right, you just did a BIG service for 
 D on bare-metal.


Exceptions are a nightmare. It works for GDC with GCC code. My 
codebase uses GCC for its C (ST toolchain), so I need to write 
some code that can take LDC's exception handling and make it 
compatible with how GCC operates. So, it looks like there will be 
multiple exception handling implementations (LDC with GCC 
backend, LDC with clang backend, GDC with GCC backend). I wish D 
had something like Zig's error handling.

Otherwise, thank you! It was designed to be agnostic as much as 
possible. I didn't know it'd help out for more than just 
microcontrollers :)
May 30 2021
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 31/05/2021 1:05 PM, Dylan Graham wrote:

 I haven't put any thought into the license. Since LWDR is derived from 
 DRuntime, I assume I'll have to use its license. If not, I'd like to go 
 with something permissive like MIT.


Boost is permissive.
May 30 2021
parent Dylan Graham <dylan.graham2000 gmail.com> writes:
On Monday, 31 May 2021 at 01:16:46 UTC, rikki cattermole wrote:

 On 31/05/2021 1:05 PM, Dylan Graham wrote:

 I haven't put any thought into the license. Since LWDR is 
 derived from DRuntime, I assume I'll have to use its license. 
 If not, I'd like to go with something permissive like MIT.


 Boost is permissive.


Checked it out. I'm happy with it. I committed the Boost v1 
license to the repo.
May 31 2021
prev sibling parent reply Iain Buclaw <ibuclaw gdcproject.org> writes:
On Monday, 31 May 2021 at 01:05:03 UTC, Dylan Graham wrote:

 On Sunday, 30 May 2021 at 17:31:37 UTC, Dukc wrote:

 On Sunday, 30 May 2021 at 14:28:25 UTC, Dylan Graham wrote:

 It works by providing a series of barebones API hooks (alloc, 
 dealloc, assert, etc) (defined in `rtoslink.d`), which you 
 must implement and/or point to your RTOS implementation.


 Quickly looking, the implementation looks very portable, save 
 for exceptions. with `rtoslink.d`, this will probably enable a 
 lot of stuff on any platform without DRuntime. Not just 
 microcontrollers. If I'm right, you just did a BIG service for 
 D on bare-metal.


 Exceptions are a nightmare. It works for GDC with GCC code. My 
 codebase uses GCC for its C (ST toolchain), so I need to write 
 some code that can take LDC's exception handling and make it 
 compatible with how GCC operates. So, it looks like there will 
 be multiple exception handling implementations (LDC with GCC 
 backend, LDC with clang backend, GDC with GCC backend). I wish 
 D had something like Zig's error handling.

 Otherwise, thank you! It was designed to be agnostic as much as 
 possible. I didn't know it'd help out for more than just 
 microcontrollers :)


Good to see this work come to fruition.  First thing I stumbled 
across was a 
[mispelling](https://github.com/0dyl/LWDR/blob/eb5de110ba2cff4bd0e654e8a68b59fc5eb76157/so
rce/rtoslink.d#L14) of one of the RTOS hooks.

Regarding exceptions, apart from one small detail I'd have 
thought that GDC and LDC would be compatible, as both just use 
libunwind.  Said small detail are the name of the entry-points 
for the "throw" and "personality" routines.  Both of which can be 
sorted out trivially with some stubs to forward from one to the 
other.

```
extern(C) void _d_throw(Throwable o) { return _d_throw_common(o); 
}
extern(C) void _d_throw_exception(Throwable o) { return 
_d_throw_common(o); }

extern(C) void _d_throw_common(Throwable o)
{
    pragma(inline, false);
    // Implementation here: tail merging should take care of the 
entrypoints.
}
```
May 31 2021
parent Dylan Graham <dylan.graham2000 gmail.com> writes:
On Monday, 31 May 2021 at 11:16:01 UTC, Iain Buclaw wrote:

 Good to see this work come to fruition.  First thing I stumbled 
 across was a 
 [mispelling](https://github.com/0dyl/LWDR/blob/eb5de110ba2cff4bd0e654e8a68b59fc5eb76157/so
rce/rtoslink.d#L14) of one of the RTOS hooks.


I'll get on it!


 Regarding exceptions, apart from one small detail I'd have 
 thought that GDC and LDC would be compatible, as both just use 
 libunwind.  Said small detail are the name of the entry-points 
 for the "throw" and "personality" routines.  Both of which can 
 be sorted out trivially with some stubs to forward from one to 
 the other.


It was a while ago I was messing with exceptions. I can't 
remember the errors specifically. I'll give exceptions another go 
today and report my findings.
May 31 2021
prev sibling next sibling parent Dylan Graham <dylan.graham2000 gmail.com> writes:
On Sunday, 30 May 2021 at 14:28:25 UTC, Dylan Graham wrote:

 Github: https://github.com/0dyl/LWDR
 DUB: https://code.dlang.org/packages/lwdr


As for my next steps, I'm going to look at implementing TLS 
variables. It doesn't look too difficult.
May 30 2021
prev sibling parent reply Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Sunday, 30 May 2021 at 14:28:25 UTC, Dylan Graham wrote:

 Github: https://github.com/0dyl/LWDR
 DUB: https://code.dlang.org/packages/lwdr

 [...]


Well done sir!

Keep it up ☀️
May 31 2021
parent Dylan Graham <dylan.graham2000 gmail.com> writes:
On Monday, 31 May 2021 at 15:41:12 UTC, Imperatorn wrote:

 On Sunday, 30 May 2021 at 14:28:25 UTC, Dylan Graham wrote:

 Github: https://github.com/0dyl/LWDR
 DUB: https://code.dlang.org/packages/lwdr

 [...]


 Well done sir!

 Keep it up ☀️


Thank you :)
May 31 2021