www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - SVD_to_D: Generate over 100k lines of highly-optimized microcontroller

reply Mike <none none.com> writes:
https://github.com/JinShil/svd_to_d

SVD_to_D is a command-line utility that generates D code from ARM 
Cortex-M SVD files.

SVD files are XML files that describe, in great detail, the 
memory layout and characteristics of registers in an ARM Cortex-M 
microcontroller. See 
https://github.com/posborne/cmsis-svd/tree/master/data for a 
curated list of SVD files for many ARM Cortex-M microcontrollers 
from various silicon vendeors.

 From the information in an SVD file, code for accessing the 
microcontroller's memory-mapped-io registers can be automatically 
generated, and SVD_to_D does exactly that.

For a large microcontroller like the SM32F7, the amount of code 
generated can be more than 100k lines.  See an example of 
generated D code here: 
https://github.com/JinShil/svd_to_d/tree/master/examples/stm32f7x9

The code generated by SVD_to_D depends on this memory-mapped-IO 
library: https://github.com/JinShil/memory_mapped_io  That 
library uses D's CTFE and meta-programming features to generate 
highly optimized code (for both size and speed) at compile-time 
with additional features such as type safety and 
compile-time-enforced mutability.

Enjoy!

Mike
Jul 31 2017
next sibling parent Andrey <avraliov gmail.com> writes:
On Monday, 31 July 2017 at 08:51:16 UTC, Mike wrote:

 https://github.com/JinShil/svd_to_d

 SVD_to_D is a command-line utility that generates D code from 
 ARM Cortex-M SVD files.

 [...]



Thanks for your work. Not many people in D community thinking 
about embedded one
Jul 31 2017
prev sibling next sibling parent aberba <karabutawotld gmail.com> writes:
On Monday, 31 July 2017 at 08:51:16 UTC, Mike wrote:

 https://github.com/JinShil/svd_to_d

 SVD_to_D is a command-line utility that generates D code from 
 ARM Cortex-M SVD files.

 [...]


I'm currently not into embedded systems but I hope to see D used 
for smart robots and IoT someday. And your work paves the way.
Jul 31 2017
prev sibling next sibling parent Mike <none none.com> writes:
On Monday, 31 July 2017 at 08:51:16 UTC, Mike wrote:


 The code generated by SVD_to_D depends on this memory-mapped-IO 
 library: https://github.com/JinShil/memory_mapped_io  That 
 library uses D's CTFE and meta-programming features to generate 
 highly optimized code (for both size and speed) at compile-time 
 with additional features such as type safety and 
 compile-time-enforced mutability.


By the way, there are a couple of bugs that are preventing me 
from perfecting the memory-mapped-IO library.  It'd be great to 
have them addressed.

https://issues.dlang.org/show_bug.cgi?id=12496
https://issues.dlang.org/show_bug.cgi?id=17684

Thanks,
Mike
Aug 01 2017
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 7/31/17 4:51 AM, Mike wrote:

 https://github.com/JinShil/svd_to_d
 
 SVD_to_D is a command-line utility that generates D code from ARM 
 Cortex-M SVD files.
 


Mike, I have to say still your talk in 2014 was one of my favorites. One 
of the things that was so impressive to me was the way you scraped the 
PDF to generate all the registers automatically. Having worked with STM 
chips (not ARM, but really basic 8-bit versions), I can definitely 
appreciate all the work this saves.

This seems like a natural extension, so awesome!

-Steve
Aug 01 2017
parent reply Mike <none none.com> writes:
On Tuesday, 1 August 2017 at 14:52:51 UTC, Steven Schveighoffer 
wrote:


 Mike, I have to say still your talk in 2014 was one of my 
 favorites. One of the things that was so impressive to me was 
 the way you scraped the PDF to generate all the registers 
 automatically. Having worked with STM chips (not ARM, but 
 really basic 8-bit versions), I can definitely appreciate all 
 the work this saves.


I'm not sure my work is worthy of such kind words, but thank you.

The PDF screen utility is here: 
https://github.com/JinShil/stm32_datasheet_to_d.  I'm not sure if 
it still compiles and works, but I might need to update it 
someday.

It generates much better code and more complete documentation 
than svd_to_d.  Most silicon vendors don't appear to invest much 
into their SVD files, so the SVD files are often incomplete and 
lacking the documentation that makes "good code + a good IDE = a 
substitute for the datasheet".

If I were creating a professional product, I'd probably prefer 
scraping the PDF over using the SVD files, or maybe some way to 
merge the two.

Mike
Aug 01 2017
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 8/1/17 8:34 PM, Mike wrote:

 On Tuesday, 1 August 2017 at 14:52:51 UTC, Steven Schveighoffer wrote:
 

 Mike, I have to say still your talk in 2014 was one of my favorites. 
 One of the things that was so impressive to me was the way you scraped 
 the PDF to generate all the registers automatically. Having worked 
 with STM chips (not ARM, but really basic 8-bit versions), I can 
 definitely appreciate all the work this saves.


 
 I'm not sure my work is worthy of such kind words, but thank you.


Sorry, it was awesome. You should own it. I think many people were 
impressed with what you were able to achieve, especially those who have 
dealt with embedded programming.


 The PDF screen utility is here: 
 https://github.com/JinShil/stm32_datasheet_to_d.  I'm not sure if it 
 still compiles and works, but I might need to update it someday.
 
 It generates much better code and more complete documentation than 
 svd_to_d.  Most silicon vendors don't appear to invest much into their 
 SVD files, so the SVD files are often incomplete and lacking the 
 documentation that makes "good code + a good IDE = a substitute for the 
 datasheet".

 If I were creating a professional product, I'd probably prefer scraping 
 the PDF over using the SVD files, or maybe some way to merge the two.
 


Yes, if the datasheet is wrong, they probably hear about it more, and 
fix it more readily than some xml files. Much harder to fix etch than it 
is to update some software.

In addition, I bet the software engineers take both into account and 
defer to the spec, meaning the bugs in the SVD may not get reported.

I would prefer the more accurate picture if it were me. In fact, you may 
be able to give them a better tool to generate the SVD file from the pdf :)

-Steve
Aug 02 2017
parent Taylor Hillegeist <taylorh140 gmail.com> writes:
On Wednesday, 2 August 2017 at 13:31:40 UTC, Steven Schveighoffer 
wrote:

 On 8/1/17 8:34 PM, Mike wrote:

 On Tuesday, 1 August 2017 at 14:52:51 UTC, Steven 
 Schveighoffer wrote:
 

 [...]


 
 I'm not sure my work is worthy of such kind words, but thank 
 you.


 Sorry, it was awesome. You should own it. I think many people 
 were impressed with what you were able to achieve, especially 
 those who have dealt with embedded programming.


 The PDF screen utility is here: 
 https://github.com/JinShil/stm32_datasheet_to_d.  I'm not sure 
 if it still compiles and works, but I might need to update it 
 someday.
 
 It generates much better code and more complete documentation 
 than svd_to_d.  Most silicon vendors don't appear to invest 
 much into their SVD files, so the SVD files are often 
 incomplete and lacking the documentation that makes "good code 
 + a good IDE = a substitute for the datasheet".

 If I were creating a professional product, I'd probably prefer 
 scraping the PDF over using the SVD files, or maybe some way 
 to merge the two.
 


 Yes, if the datasheet is wrong, they probably hear about it 
 more, and fix it more readily than some xml files. Much harder 
 to fix etch than it is to update some software.

 In addition, I bet the software engineers take both into 
 account and defer to the spec, meaning the bugs in the SVD may 
 not get reported.

 I would prefer the more accurate picture if it were me. In 
 fact, you may be able to give them a better tool to generate 
 the SVD file from the pdf :)

 -Steve


I'm not completely sure but I would guess that the documentation 
was generated from the .svd files. If you think about it doing 
the documentation by hand would be another thing that the chip 
manufactures would have to validate against. If the documentation 
is generated from the .svd and the documentation is correct than 
the .svd file is also correct.
Aug 04 2017
prev sibling next sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Mon, 31 Jul 2017 08:51:16 +0000
schrieb Mike <none none.com>:


 https://github.com/JinShil/svd_to_d
 
 SVD_to_D is a command-line utility that generates D code from ARM 
 Cortex-M SVD files.
 
 SVD files are XML files that describe, in great detail, the 
 memory layout and characteristics of registers in an ARM Cortex-M 
 microcontroller. See 
 https://github.com/posborne/cmsis-svd/tree/master/data for a 
 curated list of SVD files for many ARM Cortex-M microcontrollers 
 from various silicon vendeors.
 
  From the information in an SVD file, code for accessing the 
 microcontroller's memory-mapped-io registers can be automatically 
 generated, and SVD_to_D does exactly that.


Nice work! SVD seems to be an ARM standard / initiative? I wish there
was something similar for MSP/AVR/PIC controllers.


-- Johannes
Aug 01 2017
parent Mike <none none.com> writes:
On Tuesday, 1 August 2017 at 19:04:44 UTC, Johannes Pfau wrote:


 SVD seems to be an ARM standard / initiative?


Yeah, ARM appears to maintain the XML schema, but the SVD files 
are usually created and distributed by the silicon vendors, in 
varying levels of quality, unfortunately.

Mike
Aug 01 2017
prev sibling next sibling parent reply Taylor Hillegeist <taylorh140 gmail.com> writes:
On Monday, 31 July 2017 at 08:51:16 UTC, Mike wrote:

 https://github.com/JinShil/svd_to_d

 SVD_to_D is a command-line utility that generates D code from 
 ARM Cortex-M SVD files.

 [...]


Reminds me of something I put together a while ago.
https://github.com/taylorh140/SVD_TO_D
But this looks much nicer, nice work!
Aug 01 2017
parent reply Mike <none none.com> writes:
On Tuesday, 1 August 2017 at 20:11:13 UTC, Taylor Hillegeist 
wrote:


 Reminds me of something I put together a while ago.
 https://github.com/taylorh140/SVD_TO_D
 But this looks much nicer, nice work!


ha ha! Even the pattern used in our code is similar.  I swear I 
never saw it before.

Looking at your code though, I probably should have used Adam's 
dom.d too; std.xml was weird to say the least.

Mike
Aug 01 2017
parent Adam D. Ruppe <destructionator gmail.com> writes:
On Wednesday, 2 August 2017 at 00:39:24 UTC, Mike wrote:

 Looking at your code though, I probably should have used Adam's 
 dom.d too; std.xml was weird to say the least.


There's a couple functions in dom.d too that might have 
simplified this:

foreach(EnumsTop; Flds.getElementsByTagName("enumeratedValues")){
	foreach(Enums; EnumsTop.getElementsByTagName("enumeratedValue")){
		foreach(child; Enums.childNodes){
			if(child.tagName=="description"){EnumV.description  = 
child.innerText();}
			if(child.tagName=="name"       ){EnumV.name         = 
child.innerText();}		
			if(child.tagName=="value"      ){EnumV.value        = 
child.innerText();}
		}


This pattern is used throughout, and it is pretty nice. Not 
saying it is wrong!

But that could prolly be written:

foreach(Enums; Flds.querySelectorAll("enumeratedValues > 
enumeratedValue")) {
     EnumV.description  = 
Enum.optionSelector("description").innerText();
     EnumV.name  = Enum.optionSelector("name").innerText();
     EnumV.value  = Enum.optionSelector("value").innerText();
}




The various *selector functions do CSS selector syntax. 
querySelector/querySelectorAll returns the first/all matching 
element, or null if none. requireSelector returns the first 
matching, or throws exception if there is none. optionSelector 
returns a wrapper object that is never null so you can call its 
methods, but they return null if it was.

Meaning you don't have to explicitly check presence, the 
.innerText just returns the null string if the element wasn't 
there.


The idea of these is to make navigating xml like this a bit more 
convenient.
Aug 01 2017
prev sibling parent reply WebFreak001 <d.forum webfreak.org> writes:
On Monday, 31 July 2017 at 08:51:16 UTC, Mike wrote:

 https://github.com/JinShil/svd_to_d

 SVD_to_D is a command-line utility that generates D code from 
 ARM Cortex-M SVD files.

 [...]


I just clicked through some random files in the example folder, 
this line seems broken: 
https://github.com/JinShil/svd_to_d/blob/master/examples/atsamd21g18a/AC.d#L13
Aug 05 2017
parent Mike <none none.com> writes:
On Saturday, 5 August 2017 at 20:08:39 UTC, WebFreak001 wrote:


 I just clicked through some random files in the example folder, 
 this line seems broken: 
 https://github.com/JinShil/svd_to_d/blob/master/examples/atsamd21g18a/AC.d#L13


Fixed. Thanks!
Aug 05 2017