www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DIP62: Volatile type qualifier for unoptimizable variables in

reply Johannes Pfau <nospam example.com> writes:
You may have seen Mike's talk about D on embedded systems and were
surprised about Iain's comment that using 'shared' as a replacement for
'volatile' is actually only possible because of a bug in gdc.

DIP62 describes how to solve this problem and make embedded programming
a first-class citizen in D:
http://wiki.dlang.org/DIP62

This is the only language change necessary to make D really perfect for
embedded programming, so it's important that we fix this.
There are some more implementation issues, but these aren't language
problems and easy to fix.

The DIP provides detailed descriptions and rationales so I won't
copy/paste this information into this post. But please:

* Read the complete DIP before commenting. I'd took a long time to
  write, so it's OK if it takes some time to read ;-) Also your question
  might already be answered.
* Although it's explained in the DIP in detail, I must mention this
  here again: We need _first-class_ support for embedded programming in
  D. Workarounds might be available (peek/poke, inline asm) but it will
  be obvious to most C programmers that C has got the better solution
  with the volatile qualifier. D enthusiasts might be happy if they can
  use workarounds to make D work on microcontrollers. But if we want to
  appeal to outsiders we can't confront them with obvious workarounds
  and hacks as the first thing they see in D. And if they think embedded
  programming is a second class citizen in D, why should they use it
  at all?
Jul 15 2014
next sibling parent reply Artur Skawina via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 07/15/14 18:25, Johannes Pfau via Digitalmars-d wrote:
   here again: We need _first-class_ support for embedded programming in
   D. Workarounds might be available (peek/poke, inline asm) but it will
   be obvious to most C programmers that C has got the better solution
   with the volatile qualifier. D enthusiasts might be happy if they can
Compiler barriers are not "workarounds". "volatile" is not a "better solution". It's used in C only because it's defined and reasonably portable; barriers are a language extension, hence dialect specific. First class support for "embedded programming" would be defining barriers, not transplanting C's volatile. You can already express all the described volatile semantics in GDC's D dialect, in a completely portable way and without using a single asm instruction. What's missing is just a defined standard interface which can be supported by all D compilers. artur
Jul 15 2014
next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Artur Skawina:

 You can already express
 all the described volatile semantics in GDC's D dialect, in a
 completely portable way and without using a single asm 
 instruction.
What's GDC syntax? Bye, bearophile
Jul 15 2014
next sibling parent Johannes Pfau <nospam example.com> writes:
Am Tue, 15 Jul 2014 17:07:26 +0000
schrieb "bearophile" <bearophileHUGS lycos.com>:

 Artur Skawina:
 
 You can already express
 all the described volatile semantics in GDC's D dialect, in a
 completely portable way and without using a single asm 
 instruction.
What's GDC syntax? Bye, bearophile
That's off-topic, but its asm("" : : : "memory");
Jul 15 2014
prev sibling parent Artur Skawina via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 07/15/14 19:07, bearophile via Digitalmars-d wrote:
 Artur Skawina:
 
 You can already express
 all the described volatile semantics in GDC's D dialect, in a
 completely portable way and without using a single asm instruction.
What's GDC syntax?
Some examples: http://forum.dlang.org/post/501A6E01.7010809 gmail.com It can all be wrapped in "saner" templates and structs (for mmapped register banks etc). Johannes specifically didn't ask for mem barriers, cpu barriers or atomicity, just for compiler optimization barriers -- these don't need 'volatile'. artur
Jul 15 2014
prev sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Tue, 15 Jul 2014 19:03:52 +0200
schrieb Artur Skawina via Digitalmars-d <digitalmars-d puremagic.com>:

 Compiler barriers are not "workarounds". "volatile" is not a "better
 solution". It's used in C only because it's defined and reasonably
 portable; barriers are a language extension, hence dialect specific.
 
 First class support for "embedded programming" would be defining
 barriers, not transplanting C's volatile. You can already express
 all the described volatile semantics in GDC's D dialect, in a
 completely portable way and without using a single asm instruction.
 What's missing is just a defined standard interface which can be
 supported by all D compilers.
 
 artur
Did you even read the section that explains why volatility is a property of the memory address and not of the access (4.2.3)? What's your response to that? Why do shared variables have the privilege to prevent accessing shared memory in inappropriate ways but it's fine to place the burden of checking that all accesses to volatile memory are backed by compiler barriers to the user? How is that 'first class' support? I intentionally didn't put any technical information into this post to make sure that people actually read the DIP and the only think you responded to were the non-technical parts of my first post...
Jul 15 2014
next sibling parent reply "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Johannes Pfau"  wrote in message news:lq3nf4$rbp$1 digitalmars.com...

 Did you even read the section that explains why volatility is a property
 of the memory address and not of the access (4.2.3)? What's your
 response to that?
This is true, and it's the ideal I guess, but I'm not sure that it makes it worth extending the type system for. My use of volatile in C consists entirely of #define SOME_REGISTER (*(volatile unsigned *)(BASE + OFFSET)) And this could easily be done in D with a wrapper around peek/poke functions. (assuming they're intrinsics and inlining/optimization works correctly) So, the complexity it adds to the type system and overloading etc is not worth it for my use cases. What are the other use cases where it pulls its weight?
Jul 15 2014
parent Johannes Pfau <nospam example.com> writes:
Am Wed, 16 Jul 2014 03:33:35 +1000
schrieb "Daniel Murphy" <yebbliesnospam gmail.com>:

 "Johannes Pfau"  wrote in message news:lq3nf4$rbp$1 digitalmars.com...
 
 Did you even read the section that explains why volatility is a
 property of the memory address and not of the access (4.2.3)?
 What's your response to that?
This is true, and it's the ideal I guess, but I'm not sure that it makes it worth extending the type system for. My use of volatile in C consists entirely of #define SOME_REGISTER (*(volatile unsigned *)(BASE + OFFSET)) And this could easily be done in D with a wrapper around peek/poke functions. (assuming they're intrinsics and inlining/optimization works correctly) So, the complexity it adds to the type system and overloading etc is not worth it for my use cases. What are the other use cases where it pulls its weight?
That's a judgement call of course and the main reason why I constantly say 'first class support'. This is the only language change we need for embedded systems, do we really want to compromise here? Especially if C has already got a better solution? There's some complexity in the compiler implementation, indeed. There's also some complexity in the compiler for -cov, for nogc and for shared. And we recently added complexity to support C++ namespaces. All of these things are completely useless on embedded systems. So I think we could add one slightly complex thing for embedded development. Also there's effectively no increase in language complexity for users, as nobody will use volatile with desktop applications. Volatile is already a reserved & deprecated keyword. We don't break any code. Code on embedded systems will usually be less complex with volatile and it'll be safer. For some examples where this matters: - Memory used in interrupt handlers with redirection - registers with the same layout/type could be referred by pointer
Jul 15 2014
prev sibling next sibling parent reply "Kagamin" <spam here.lot> writes:
On Tuesday, 15 July 2014 at 17:13:09 UTC, Johannes Pfau wrote:
 Why do shared variables have the privilege to prevent accessing 
 shared
 memory in inappropriate ways but it's fine to place the burden 
 of
 checking that all accesses to volatile memory are backed by 
 compiler
 barriers to the user? How is that 'first class' support?
Maybe it can be automated by a wrapper template? struct Volatile(T) { T val; void opAssign(T v) { volatile_store(val, v); } } extern(C) Volatile!byte led; foreach(i;0..10) { led=0; led=1; //blink }
Jul 15 2014
parent reply Johannes Pfau <nospam example.com> writes:
Am Tue, 15 Jul 2014 17:42:47 +0000
schrieb "Kagamin" <spam here.lot>:

 On Tuesday, 15 July 2014 at 17:13:09 UTC, Johannes Pfau wrote:
 Why do shared variables have the privilege to prevent accessing 
 shared
 memory in inappropriate ways but it's fine to place the burden 
 of
 checking that all accesses to volatile memory are backed by 
 compiler
 barriers to the user? How is that 'first class' support?
Maybe it can be automated by a wrapper template? struct Volatile(T) { T val; void opAssign(T v) { volatile_store(val, v); } } extern(C) Volatile!byte led; foreach(i;0..10) { led=0; led=1; //blink }
Then we have a templated struct. Which generates TypeInfo for every instance. Which might generate an Initializer for every instance and it might generate extended debug info for every instance. These are exactly the kind of workarounds I don't want. Sure we can make it work: noTypeInfo noInit noDebug (struct volatile). But then I don't see why we don't just use the proposal in this DIP. It's less complicated, has got fewer corner case, ... And that's why I say first class support: We don't have shared!(T) we have shared T. And volatile should get it's own qualifier as well. Also some things will just not work with Volatile!T, for example volatile/nonvolatile member function overloading.
Jul 15 2014
next sibling parent "Kagamin" <spam here.lot> writes:
On Tuesday, 15 July 2014 at 17:50:13 UTC, Johannes Pfau wrote:
 And that's why I say first class support: We don't have 
 shared!(T) we
 have shared T. And volatile should get it's own qualifier as 
 well.

 Also some things will just not work with Volatile!T, for example
 volatile/nonvolatile member function overloading.
Volatile has probably much smaller scope than shared. Shared memory is used more often and in more non-trivial ways, it's not unthinkable for even complex data structures to be shared or abstracted with an interface, so it makes sense that interfaces should support shared methods. Volatile is usually a small range of bytes (a communication device), which should be read once or written once. D doesn't have a way of saving on typeinfos, so volatile is not alone in this respect. I'd prefer to see it applicable to anything (or rather everything).
Jul 15 2014
prev sibling parent reply "Meta" <jared771 gmail.com> writes:
On Tuesday, 15 July 2014 at 17:50:13 UTC, Johannes Pfau wrote:
 Then we have a templated struct. Which generates TypeInfo for 
 every
 instance. Which might generate an Initializer for every 
 instance and
 it might generate extended debug info for every instance.

 These are exactly the kind of workarounds I don't want. Sure we 
 can
 make it work:  noTypeInfo  noInit  noDebug (struct volatile). 
 But then
 I don't see why we don't just use the proposal in this DIP. 
 It's less
 complicated, has got fewer corner case, ...

 And that's why I say first class support: We don't have 
 shared!(T) we
 have shared T. And volatile should get it's own qualifier as 
 well.

 Also some things will just not work with Volatile!T, for example
 volatile/nonvolatile member function overloading.
Functions can take Volatile!T, of course, and you can always have "set/volatileSet". You might even be able to use opDispatch to make it less DRY.
Jul 15 2014
parent reply Johannes Pfau <nospam example.com> writes:
Am Tue, 15 Jul 2014 18:13:26 +0000
schrieb "Meta" <jared771 gmail.com>:

 Also some things will just not work with Volatile!T, for example
 volatile/nonvolatile member function overloading.
Functions can take Volatile!T, of course, and you can always have "set/volatileSet". You might even be able to use opDispatch to make it less DRY.
No, this doesn't work with Volatile!: struct A { private int _val; int read(){ return val+val }; int read() volatile { auto tmp = val; return tmp*tmp; }; } opDispatch sounds _awesome_ for systems with a few KB of SRAM. People here always have nice ideas but will you implement it and tune it until it has as little overhead as the C version?
Jul 15 2014
parent reply "Meta" <jared771 gmail.com> writes:
On Tuesday, 15 July 2014 at 18:30:11 UTC, Johannes Pfau wrote:
 No, this doesn't work with Volatile!:

 struct A
 {
     private int _val;
     int read(){ return val+val };
     int read(bool volatile)() { auto tmp = val; return tmp*tmp; 
 };
 }
struct A { private int _val; int read(bool volatile)() { static if (volatile) { auto tmp = val; return tmp*tmp; } else { return val+val; } }; } And you are no worse off than before. This template can only generate 2 possible functions, and the instantiations are also cached so as far as I know, there's no code bloat.
 opDispatch sounds _awesome_ for systems with a few KB of SRAM. 
 People here always have nice ideas but will you implement it 
 and tune it until it has as little overhead as the C version?
That's not something I'm currently interested in, but it's something to look into if someone wants to use it in embedded programming.
Jul 15 2014
parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 15 July 2014 19:38, Meta via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Tuesday, 15 July 2014 at 18:30:11 UTC, Johannes Pfau wrote:
 No, this doesn't work with Volatile!:

 struct A
 {
     private int _val;
     int read(){ return val+val };
     int read(bool volatile)() { auto tmp = val; return tmp*tmp; };
 }
struct A { private int _val; int read(bool volatile)() { static if (volatile) { auto tmp = val; return tmp*tmp; } else { return val+val; } }; } And you are no worse off than before. This template can only generate 2 possible functions, and the instantiations are also cached so as far as I know, there's no code bloat.
Worse, there's no code because the compiler optimises everything away!
Jul 15 2014
parent "Meta" <jared771 gmail.com> writes:
On Tuesday, 15 July 2014 at 19:23:36 UTC, Iain Buclaw via 
Digitalmars-d wrote:
 Worse, there's no code because the compiler optimises 
 everything away!
I was assuming you'd do something in the static if (volatile) section to stop it from doing that. I know very little when it comes to these things.
Jul 15 2014
prev sibling parent reply Artur Skawina via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 07/15/14 19:11, Johannes Pfau via Digitalmars-d wrote:
 Am Tue, 15 Jul 2014 19:03:52 +0200
 schrieb Artur Skawina via Digitalmars-d <digitalmars-d puremagic.com>:
 
 Compiler barriers are not "workarounds". "volatile" is not a "better
 solution". It's used in C only because it's defined and reasonably
 portable; barriers are a language extension, hence dialect specific.

 First class support for "embedded programming" would be defining
 barriers, not transplanting C's volatile. You can already express
 all the described volatile semantics in GDC's D dialect, in a
 completely portable way and without using a single asm instruction.
 What's missing is just a defined standard interface which can be
 supported by all D compilers.
Did you even read the section that explains why volatility is a property of the memory address and not of the access (4.2.3)? What's your response to that?
You're assuming that the raw memory needs to be exposed. It doesn't. IOW, instead of: struct HWRegs { uint a; uint b; /*...*/ } auto ptr = cast(HWReg*)0xc0000000; do: struct HWRegs { uint a; uint b; /*...*/ } auto ptr = cast(Volatile!HwRegs*)0xc0000000; and the Volatile template will magically add all the access barriers.
 Why do shared variables have the privilege to prevent accessing shared
 memory in inappropriate ways but it's fine to place the burden of
 checking that all accesses to volatile memory are backed by compiler
 barriers to the user? How is that 'first class' support?
See above; just don't expose mmapped regs as normal memory.
 I intentionally didn't put any technical information into this post to
 make sure that people actually read the DIP and the only thing you
 responded to were the non-technical parts of my first post...
Because the technical details don't matter when the point is that 'volatile' is not necessary. I did read your DIP, when you announced it originally and also the gdc ml discussion that followed. I have not seen any good argument for introducing a volatile type qualifier. Could you show one (concrete) example where using 'volatile' is better than my approach? artur
Jul 15 2014
next sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Tue, 15 Jul 2014 19:44:51 +0200
schrieb Artur Skawina via Digitalmars-d <digitalmars-d puremagic.com>:

 On 07/15/14 19:11, Johannes Pfau via Digitalmars-d wrote:
 Am Tue, 15 Jul 2014 19:03:52 +0200
 schrieb Artur Skawina via Digitalmars-d
 <digitalmars-d puremagic.com>:
 
 Compiler barriers are not "workarounds". "volatile" is not a
 "better solution". It's used in C only because it's defined and
 reasonably portable; barriers are a language extension, hence
 dialect specific.

 First class support for "embedded programming" would be defining
 barriers, not transplanting C's volatile. You can already express
 all the described volatile semantics in GDC's D dialect, in a
 completely portable way and without using a single asm instruction.
 What's missing is just a defined standard interface which can be
 supported by all D compilers.
Did you even read the section that explains why volatility is a property of the memory address and not of the access (4.2.3)? What's your response to that?
You're assuming that the raw memory needs to be exposed. It doesn't. IOW, instead of: struct HWRegs { uint a; uint b; /*...*/ } auto ptr = cast(HWReg*)0xc0000000; do: struct HWRegs { uint a; uint b; /*...*/ } auto ptr = cast(Volatile!HwRegs*)0xc0000000;
You didn't say anything about a template in your last response. See my reply to Kagamin: - Template bloat: TypeInfo, Initializer, DebugInfo - Requires inlining to produce decent results - Requires force inlining: You wouldn't want any method implementations of Volatile to actually end up in object files.
 
 and the Volatile template will magically add all the access barriers.
 
 Why do shared variables have the privilege to prevent accessing
 shared memory in inappropriate ways but it's fine to place the
 burden of checking that all accesses to volatile memory are backed
 by compiler barriers to the user? How is that 'first class' support?
See above; just don't expose mmapped regs as normal memory.
This is not only for mmaped regs. It's also for memory used in interrupt handlers with indirection.
 I have
 not seen any good argument for introducing a volatile type qualifier.
 Could you show one (concrete) example where using 'volatile' is better
 than my approach?
 
For one thing we should not dictate usage patterns (like don't expose raw memory) on users. Then volatile is very simple for every compiler to implement and optimize, your idea requires multiple pragmas/attributes to make sure there's no bloat and inlining always works. (For example, in gcc even a force inline attribute does not inline if -finline is not used. I don't consider that acceptable for this kind of code. So we might have to special case Volatile!T even more in the compiler, to really always inline) Otherwise Volatile!T is OK, but it's a new, fancy idea and might need lots of work, whereas a volatile qualifer could be implemented easily and we know right now for sure that it'll work 100% as expected.
Jul 15 2014
parent Artur Skawina via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 07/15/14 20:20, Johannes Pfau via Digitalmars-d wrote:
 (For example, in gcc even a force inline attribute does not inline if
 -finline is not used. I don't consider that acceptable for this kind of
 code. So we might have to special case Volatile!T even more in the
 compiler, to really always inline)
Not inlining in unoptimized (-O0) builds wouldn't be a problem, it could even be useful sometimes (setting breakpoints for specific hw accesses etc). But is that actually the case? I practically never use -O0, so you just made me check - and indeed the GDC here *always* inlines always_inline functions, at "-O0", even when explicitly given '-fno-inline'... Anyway, the language *relies* on automatic inlining happening, this case isn't special.
 Otherwise Volatile!T is OK, but it's a new, fancy idea and might need
 lots of work, whereas a volatile qualifer could be implemented easily
 and we know right now for sure that it'll work 100% as expected.
There's nothing new about it - it's how it has been done in gcc-ese, for many, many years; it's just combining it with templates, which aren't exactly novel either.
 Also there's effectively no increase in language complexity for users,
 as nobody will use volatile with desktop applications.
Actually, there are quite a few "normal" uses. The most obvious example probably being communication with signal handlers. Then there are various kinds of inter-thread/shared-mem communication schemes. Yes, D has many problems, some of which you have mentioned in this thread. [1] But introducing additional complexity to a language just to avoid other problematic areas is not a good idea; that approach does not scale. The result are many half-finished features, every one with some kind of problem. Could a 'volatile' type qualifier be made to work, and are there technical problems with the DIP? Yes, and No. I'm not ignoring the details; I probably agree with most, if not all, of them. I'm saying the DIP is a good answer, to the wrong question. artur [1] Which I didn't respond to, to avoid derailing the thread. ;)
Jul 15 2014
prev sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Tue, 15 Jul 2014 19:44:51 +0200
schrieb Artur Skawina via Digitalmars-d <digitalmars-d puremagic.com>:

 it originally and also the gdc ml discussion that followed. I have
 not seen any good argument for introducing a volatile type qualifier.
 Could you show one (concrete) example where using 'volatile' is better
 than my approach?
 
 artur
I experimented a little more with Volatile!T and I don't think you can make this example work: struct Timer { uint control; uint data; } enum timerA = (Volatile!Timer)* = cast()0xDEADBEAF; timerA.control |= 0b1;
Jul 16 2014
next sibling parent "Dominikus Dittes Scherkl" writes:
On Wednesday, 16 July 2014 at 13:04:37 UTC, Johannes Pfau wrote:
 I experimented a little more with Volatile!T and I don't think 
 you can make this example work:

 struct Timer
 {
     uint control;
     uint data;
 }

 enum timerA = (Volatile!Timer)* = cast()0xDEADBEAF;

 timerA.control |= 0b1;
How about struct VolatileTimer { Volatile!uint control; Volatile!uint data; } enum timerA = cast(VolatileTimer*)0xDEADBEAF; timerA.control |= 0b1;
Jul 16 2014
prev sibling next sibling parent reply "Kagamin" <spam here.lot> writes:
On Wednesday, 16 July 2014 at 13:04:37 UTC, Johannes Pfau wrote:
 I experimented a little more with Volatile!T and I don't think 
 you can
 make this example work:

 struct Timer
 {
     uint control;
     uint data;
 }

 enum timerA = (Volatile!Timer)* = cast()0xDEADBEAF;

 timerA.control |= 0b1;
The actual code is a little different: http://forum.dlang.org/thread/lq3kq8$pa3$1 digitalmars.com?page=5#post-fkgjcnqyoqltftbfmwqd:40forum.dlang.org
Jul 16 2014
parent "Kagamin" <spam here.lot> writes:
On Wednesday, 16 July 2014 at 13:51:49 UTC, Kagamin wrote:
 On Wednesday, 16 July 2014 at 13:04:37 UTC, Johannes Pfau wrote:
 I experimented a little more with Volatile!T and I don't think 
 you can
 make this example work:

 struct Timer
 {
    uint control;
    uint data;
 }

 enum timerA = (Volatile!Timer)* = cast()0xDEADBEAF;

 timerA.control |= 0b1;
The actual code is a little different: http://forum.dlang.org/thread/lq3kq8$pa3$1 digitalmars.com?page=5#post-fkgjcnqyoqltftbfmwqd:40forum.dlang.org
I think, the idea behind such declarations is to not let user to set volatile qualifiers. What if he forgets?
Jul 16 2014
prev sibling parent reply Artur Skawina via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 07/16/14 15:02, Johannes Pfau via Digitalmars-d wrote:
 Am Tue, 15 Jul 2014 19:44:51 +0200
 schrieb Artur Skawina via Digitalmars-d <digitalmars-d puremagic.com>:
 
 Could you show one (concrete) example where using 'volatile' is better
 than my approach?
I experimented a little more with Volatile!T and I don't think you can make this example work: struct Timer { uint control; uint data; } enum timerA = (Volatile!Timer)* = cast()0xDEADBEAF; timerA.control |= 0b1;
Thank you very much for showing actual code. There are many (wrong) assumptions being made in this thread (and the one on the gdc list). Claims like "A is required" or "B is impossible", where A and B are not actually defined and only vaguely described, do not usually lead to a productive dialogue. Concrete examples and code make it possible for the discussion to be about facts and not perceptions. struct Timer { uint control; uint data; } enum timerA = cast(Volatile!(Timer)*)0xDEADBEAF; //enum timerA = volatile_(cast(Timer*)0xDEADBEAF); // works too. void main() { timerA.control |= 0b1; } struct Volatile(PT) { PT raw; auto opDispatch(string M)() property { return volatile_(mixin(q{&raw.}~M)); } auto opOpAssign(string OP, B)(B b) { auto a = load(*raw); auto r = mixin("a "~OP~" b"); store(*raw, r); return this; } static: T load(T)(ref T v) { asm { "" : "+m" v; } T res = v; asm { "" : "+g" res; } return res; } T store(T)(ref T v, const T a) { asm { "" : : "m" v; } v = a; asm { "" : "+m" v; } return a; } } auto volatile_(A)(A a) { return Volatile!A(a); } // Just for IFTI. when compiled w/ "gdc -O1" results in 0000000000402949 <_Dmain>: 402949: b8 af be ad de mov $0xdeadbeaf,%eax 40294e: 8b 10 mov (%rax),%edx 402950: 83 ca 01 or $0x1,%edx 402953: 89 10 mov %edx,(%rax) 402955: b8 00 00 00 00 mov $0x0,%eax 40295a: c3 retq which is the exact op sequence you'd expect. Doing it as just one R-M-W op would be possible too, but that might not be supported by all HW/platforms. And, yes, gdc inlines it even at -O0, if inline is used. But at -O0 the generated code is /already/ huge and inefficient, hence not really usable in constrained mem environments. Obviously, this is just a proof of concept and the minimal implementation that can only handle your example. But it's enough to disprove the it-cannot-be-made-to-work-without-c-volatile claim. Are there any other examples of things that cannot be efficiently handled w/o a built-in volatile type qualifier? artur
Jul 16 2014
parent reply Johannes Pfau <nospam example.com> writes:
Am Wed, 16 Jul 2014 21:59:57 +0200
schrieb Artur Skawina via Digitalmars-d <digitalmars-d puremagic.com>:

 On 07/16/14 15:02, Johannes Pfau via Digitalmars-d wrote:
 Am Tue, 15 Jul 2014 19:44:51 +0200
 schrieb Artur Skawina via Digitalmars-d
 <digitalmars-d puremagic.com>:
 
 Could you show one (concrete) example where using 'volatile' is
 better than my approach?
I experimented a little more with Volatile!T and I don't think you can make this example work: struct Timer { uint control; uint data; } enum timerA = (Volatile!Timer)* = cast()0xDEADBEAF; timerA.control |= 0b1;
Thank you very much for showing actual code. There are many (wrong) assumptions being made in this thread (and the one on the gdc list). Claims like "A is required" or "B is impossible", where A and B are not actually defined and only vaguely described, do not usually lead to a productive dialogue. Concrete examples and code make it possible for the discussion to be about facts and not perceptions. struct Timer { uint control; uint data; } enum timerA = cast(Volatile!(Timer)*)0xDEADBEAF; //enum timerA = volatile_(cast(Timer*)0xDEADBEAF); // works too. void main() { timerA.control |= 0b1; } struct Volatile(PT) { PT raw; auto opDispatch(string M)() property { return volatile_(mixin(q{&raw.}~M)); } auto opOpAssign(string OP, B)(B b) { auto a = load(*raw); auto r = mixin("a "~OP~" b"); store(*raw, r); return this; } static: T load(T)(ref T v) { asm { "" : "+m" v; } T res = v; asm { "" : "+g" res; } return res; } T store(T)(ref T v, const T a) { asm { "" : : "m" v; } v = a; asm { "" : "+m" v; } return a; } } auto volatile_(A)(A a) { return Volatile!A(a); } // Just for IFTI. when compiled w/ "gdc -O1" results in 0000000000402949 <_Dmain>: 402949: b8 af be ad de mov $0xdeadbeaf,%eax 40294e: 8b 10 mov (%rax),%edx 402950: 83 ca 01 or $0x1,%edx 402953: 89 10 mov %edx,(%rax) 402955: b8 00 00 00 00 mov $0x0,%eax 40295a: c3 retq which is the exact op sequence you'd expect. Doing it as just one R-M-W op would be possible too, but that might not be supported by all HW/platforms. And, yes, gdc inlines it even at -O0, if inline is used. But at -O0 the generated code is /already/ huge and inefficient, hence not really usable in constrained mem environments. Obviously, this is just a proof of concept and the minimal implementation that can only handle your example. But it's enough to disprove the it-cannot-be-made-to-work-without-c-volatile claim. Are there any other examples of things that cannot be efficiently handled w/o a built-in volatile type qualifier? artur
Well I guess this is already decided anyway. I think it's kinda ridiculous that D embedded code will only be usable with strong optimization flags, but whatever. Maybe it works better if the usage is restricted to basic types only.
Jul 16 2014
parent Artur Skawina via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 07/16/14 22:09, Johannes Pfau via Digitalmars-d wrote:
 Well I guess this is already decided anyway.
Well, peek/poke is not the right solution either...
 I think it's kinda ridiculous that D embedded code will only be usable
 with strong optimization flags, but whatever. Maybe it works better
Why would it be unusable without "strong optimization flags"? At -O0 the generated code will be so slow and bloated that it will be unusable in a constrained embedded environment. Yes, the implicitly generated asserts are a problem - but _this is not a problem specific to volatile_. Ditto for RTTI, unwinding tables, dead code etc. Those issues needs to be dealt with anyway. artur
Jul 16 2014
prev sibling next sibling parent Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 15 July 2014 17:25, Johannes Pfau via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 You may have seen Mike's talk about D on embedded systems and were
 surprised about Iain's comment that using 'shared' as a replacement for
 'volatile' is actually only possible because of a bug in gdc.

 DIP62 describes how to solve this problem and make embedded programming
 a first-class citizen in D:
 http://wiki.dlang.org/DIP62

 This is the only language change necessary to make D really perfect for
 embedded programming, so it's important that we fix this.
 There are some more implementation issues, but these aren't language
 problems and easy to fix.

 The DIP provides detailed descriptions and rationales so I won't
 copy/paste this information into this post. But please:

 * Read the complete DIP before commenting. I'd took a long time to
   write, so it's OK if it takes some time to read ;-) Also your question
   might already be answered.
 * Although it's explained in the DIP in detail, I must mention this
   here again: We need _first-class_ support for embedded programming in
   D. Workarounds might be available (peek/poke, inline asm) but it will
   be obvious to most C programmers that C has got the better solution
   with the volatile qualifier. D enthusiasts might be happy if they can
   use workarounds to make D work on microcontrollers. But if we want to
   appeal to outsiders we can't confront them with obvious workarounds
   and hacks as the first thing they see in D. And if they think embedded
   programming is a second class citizen in D, why should they use it
   at all?
More simply put. A volatile keyword would be strictly for hardware access without locks. We have 'shared' for inter-thread communication, and for ensuring that there is an ordered relationship between threads when accessing the same data location at the same time. Regards Iain
Jul 15 2014
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/15/2014 9:25 AM, Johannes Pfau wrote:
 DIP62 describes how to solve this problem and make embedded programming
 a first-class citizen in D:
 http://wiki.dlang.org/DIP62
This is a particularly well-written DIP, thank you and the other contributors to it.
 * Although it's explained in the DIP in detail, I must mention this
    here again: We need _first-class_ support for embedded programming in
    D. Workarounds might be available (peek/poke, inline asm) but it will
    be obvious to most C programmers that C has got the better solution
    with the volatile qualifier. D enthusiasts might be happy if they can
    use workarounds to make D work on microcontrollers. But if we want to
    appeal to outsiders we can't confront them with obvious workarounds
    and hacks as the first thing they see in D. And if they think embedded
    programming is a second class citizen in D, why should they use it
    at all?
1. Volatile has caused a major increase in the complexity of the C++ type system - a complexity far out of proportion to the value it provides. It would have a similar complex and pervasive effect on the D type system. 2. You mention peek/poke here, but not in the DIP. [Background: I've designed and built embedded systems, and written the software for them.] There aren't that many memory-mapped I/O registers. They just aren't worth bending the entire language around them. Peek/poke work just fine: 1. they can be implemented as compiler intrinsics so there is no efficiency cost for using them 2. they make it clear in user code when memory-mapped I/O is accessed 3. if you really hate peek/poke calls appearing in the code, you can use UFCS to make them look like variables 4. they are simple to explain and understand, they do not introduce a numbing complexity to the type system 5. peek/poke will not add to the confusion about the difference between volatile and shared. No sane person would be tempted to use peek/poke to implement concurrency. The semantics of volatile in other languages won't confuse things for D peek/poke. The one thing peek/poke doesn't offer is transitivity. C/C++ don't offer volatile transitivity either. I'm not at all sure that anyone builds a data structure in memory-mapped registers, so I'm not convinced this is a need.
Jul 15 2014
next sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Tue, 15 Jul 2014 12:48:21 -0700
schrieb Walter Bright <newshound2 digitalmars.com>:

 On 7/15/2014 9:25 AM, Johannes Pfau wrote:
 DIP62 describes how to solve this problem and make embedded
 programming a first-class citizen in D:
 http://wiki.dlang.org/DIP62
This is a particularly well-written DIP, thank you and the other contributors to it.
 * Although it's explained in the DIP in detail, I must mention this
    here again: We need _first-class_ support for embedded
 programming in D. Workarounds might be available (peek/poke, inline
 asm) but it will be obvious to most C programmers that C has got
 the better solution with the volatile qualifier. D enthusiasts
 might be happy if they can use workarounds to make D work on
 microcontrollers. But if we want to appeal to outsiders we can't
 confront them with obvious workarounds and hacks as the first thing
 they see in D. And if they think embedded programming is a second
 class citizen in D, why should they use it at all?
1. Volatile has caused a major increase in the complexity of the C++ type system - a complexity far out of proportion to the value it provides. It would have a similar complex and pervasive effect on the D type system.
Well as described in the DIP it works just like shared from an implementation point of view, so I doesn't add much complexity in the compiler / type system.
 
 2. You mention peek/poke here, but not in the DIP. [Background: I've
 designed and built embedded systems, and written the software for
 them.] There aren't that many memory-mapped I/O registers. They just
 aren't worth bending the entire language around them. Peek/poke work
 just fine:
Indeed there's nothing about peek/poke in DIP62, I thought I added that. The main point is that we can not leak pointers to volatile memory to users and pretend it's normal memory. This will lead to many problems: ------------------------------------------------------------- //In runtime: enum int* SOME_REG = 0xFFFF; //In user code peek(SOME_REG); poke(SOME_REG); *SOME_REG = 1; //Oops, forgot poke! This is not acceptable ------------------------------------------------------------- This is like saying we don't need the shared qualifier as people should use atomicOp to access the data. This is actually the crucial point about this DIP. If we can make this work without a type qualifier than that's fine as well. The only other solution to this is a wrapper as described by Artur Skawina. (Can basically wrap around peek/poke): http://dpaste.dzfl.pl/6f8ca6d7e7b8 But the problem there is that it produces quite some overhead right now. We'd need to enforce at least these points: * property value must be inlined, there should not be a 'normal' function at all * value must always be treated as in release mode. For example DMD adds an assert(this) into that function which is not acceptable * There should be no TypeInfo for Volatile!T * There should be no initializer data for Volatile!T * WE should be careful with debug info for Volatile!T * We should not emit postblits or similar stuff In the end the struct basically has to 'vanish' from the object file and ' property value' needs to be substituted into the correct places.
 
 1. they can be implemented as compiler intrinsics so there is no
 efficiency cost for using them
 
 2. they make it clear in user code when memory-mapped I/O is accessed
Way more dangerous and important is if you forget peek/poke.
 
 3. if you really hate peek/poke calls appearing in the code, you can
 use UFCS to make them look like variables
But you cant do REGISTER.peek() |= 0b1;
 
 4. they are simple to explain and understand, they do not introduce a
 numbing complexity to the type system
 
Sorry, but I don't buy any of these complexity arguments. A few months ago you suggested namespaces for D - in addition to modules - which would have lead to a complexity disaster. This just shows the priorities of the project leads: Desktop apps matter, we add nogc, c++ namespaces, shared, immutable, const, -cov, ... but for embedded systems we won't even add one qualifier.
 5. peek/poke will not add to the confusion about the difference
 between volatile and shared. No sane person would be tempted to use
 peek/poke to implement concurrency. The semantics of volatile in
 other languages won't confuse things for D peek/poke.
actually peek/poke are not that different from atomicOp from a syntax perspective ;-) You'll still have to educate people about the difference.
 
 The one thing peek/poke doesn't offer is transitivity. C/C++ don't
 offer volatile transitivity either. I'm not at all sure that anyone
 builds a data structure in memory-mapped registers, so I'm not
 convinced this is a need.
Well such data structures are sometimes used in interrupts, but not that often.
Jul 15 2014
next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Johannes Pfau:

 Well as described in the DIP it works just like shared from an
 implementation point of view, so I doesn't add much complexity 
 in the compiler / type system.
I am reading blogs about compiler bugs, and I see that the implementation of volatile is the buggiest part of GCC/Clang (and the Intel and Microsoft compilers). Despite numerous bug fixes, it's still a stubbornly buggy part. So it can't be as simple to implement correctly as you say. volatile fights against the optimization stages all the time. And I recognize that Walter has a significant experience on this topic, perhaps higher than yours.
 This just shows the priorities of the project leads:
 Desktop apps matter, we add  nogc, c++ namespaces, shared, 
 immutable,
 const, -cov, ...
 but for embedded systems we won't even add one qualifier.
Having priorities is not something to be ashamed of. D has a GC, works only on 32+ bits, and several of its features are not meant for embedded systems. Even if you can restrict it for such restricted memory and CPU usages, D is not designed primarily for them. Bye, bearophile
Jul 15 2014
next sibling parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 15 July 2014 22:07, bearophile via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 Johannes Pfau:


 Well as described in the DIP it works just like shared from an
 implementation point of view, so I doesn't add much complexity in the
 compiler / type system.
I am reading blogs about compiler bugs, and I see that the implementation of volatile is the buggiest part of GCC/Clang (and the Intel and Microsoft compilers). Despite numerous bug fixes, it's still a stubbornly buggy part. So it can't be as simple to implement correctly as you say. volatile fights against the optimization stages all the time. And I recognize that Walter has a significant experience on this topic, perhaps higher than yours.
The use of volatile can be buggy in C because there is no other safe way to do inter-thread communication.
 This just shows the priorities of the project leads:
 Desktop apps matter, we add  nogc, c++ namespaces, shared, immutable,
 const, -cov, ...
 but for embedded systems we won't even add one qualifier.
Having priorities is not something to be ashamed of. D has a GC, works only on 32+ bits, and several of its features are not meant for embedded systems. Even if you can restrict it for such restricted memory and CPU usages, D is not designed primarily for them.
So your saying that embedded systems don't matter? Shame on you. :o)
Jul 15 2014
parent "bearophile" <bearophileHUGS lycos.com> writes:
Iain Buclaw:

 The use of volatile can be buggy in C because there is no other 
 safe way to do inter-thread communication.
C11 offers means much better than volatile for multi thread programming: http://en.wikipedia.org/wiki/C11_%28C_standard_revision%29#Changes_from_C99
 So your saying that embedded systems don't matter?
I was explaining that D design has priorities different from the ones for embedded systems. And I'd like D to be fitter for low level coding, two hours ago I have written this: http://forum.dlang.org/post/ftvdiqvjqgmsardoyksp forum.dlang.org Bye, bearophile
Jul 15 2014
prev sibling next sibling parent Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 15 July 2014 22:17, Iain Buclaw <ibuclaw gdcproject.org> wrote:
 On 15 July 2014 22:07, bearophile via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 Johannes Pfau:


 Well as described in the DIP it works just like shared from an
 implementation point of view, so I doesn't add much complexity in the
 compiler / type system.
I am reading blogs about compiler bugs, and I see that the implementation of volatile is the buggiest part of GCC/Clang (and the Intel and Microsoft compilers). Despite numerous bug fixes, it's still a stubbornly buggy part. So it can't be as simple to implement correctly as you say. volatile fights against the optimization stages all the time. And I recognize that Walter has a significant experience on this topic, perhaps higher than yours.
The use of volatile can be buggy in C because there is no other safe way to do inter-thread communication.
I say safe in the lightest possible terms. Volatile is in no way safe.
Jul 15 2014
prev sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Tue, 15 Jul 2014 21:07:21 +0000
schrieb "bearophile" <bearophileHUGS lycos.com>:

 Johannes Pfau:
 
 Well as described in the DIP it works just like shared from an
 implementation point of view, so I doesn't add much complexity 
 in the compiler / type system.
I am reading blogs about compiler bugs, and I see that the implementation of volatile is the buggiest part of GCC/Clang (and the Intel and Microsoft compilers). Despite numerous bug fixes, it's still a stubbornly buggy part. So it can't be as simple to implement correctly as you say. volatile fights against the optimization stages all the time. And I recognize that Walter has a significant experience on this topic, perhaps higher than yours.
And how do you think peek/poke are easier to implement in this regard? You still have to be very careful with inlining, instruction scheduling, etc. The main benefit of a volatile type qualifier is that it's similar to the C volatile qualifier, so all these bugs have already been fixed. GDC/LDC could probably just map to the volatile C qualifier. If we invent some homebrew solution we can start at zero with bug fixing. Also what does buggy mean if C doesn't have a clear specification about volatile?
 
 This just shows the priorities of the project leads:
 Desktop apps matter, we add  nogc, c++ namespaces, shared, 
 immutable,
 const, -cov, ...
 but for embedded systems we won't even add one qualifier.
Having priorities is not something to be ashamed of. D has a GC, works only on 32+ bits, and several of its features are not meant for embedded systems. Even if you can restrict it for such restricted memory and CPU usages, D is not designed primarily for them.
Well as long as those priorities are clearly communicated. If you tell me 'we don't care about embedded programming' then I'll shut up and move back to C. But 'D is a systems programming language for low level tasks' and 'we don't want to introduce a type qualifier for low level programming, but nogc is just fine' don't go together. This leaves all contributors and devs hoping to see D on embedded systems in uncertainty.
Jul 15 2014
next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Johannes Pfau:

 And how do you think peek/poke are easier to implement in this 
 regard?
Let's ask Walter.
 Well as long as those priorities are clearly communicated.
Most of the design of D is not geared toward embedded computing. It could be adapted and improved for those purposes, but surely they aren't its main target. Bye, bearophile
Jul 15 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 7/15/2014 2:28 PM, bearophile wrote:
 Johannes Pfau:

 And how do you think peek/poke are easier to implement in this regard?
Let's ask Walter.
peek/poke should be implemented as compiler intrinsics that are then mapped onto whatever the backend does best for 'volatile' accesses.
Jul 15 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/15/2014 2:17 PM, Johannes Pfau wrote:
 Well as long as those priorities are clearly communicated. If you tell
 me 'we don't care about embedded programming' then I'll shut up and move
 back to C.
Rest assured we definitely do care about embedded systems programming.
 But 'D is a systems programming language for low level tasks' and 'we
 don't want to introduce a type qualifier for low level programming, but
  nogc is just fine' don't go together. This leaves all contributors and
 devs hoping to see D on embedded systems in uncertainty.
It isn't about whether we want to introduce a type qualifier for low level programming or not. It is about what is the best solution for MMIO for low level programming. Note that D already supports inp() and outp() as compiler intrinsics, adding peek() and poke() will complement them nicely: http://dlang.org/phobos/core_bitop.html#.inp
Jul 15 2014
next sibling parent reply "Mike" <none none.com> writes:
On Tuesday, 15 July 2014 at 21:57:56 UTC, Walter Bright wrote:

 Rest assured we definitely do care about embedded systems 
 programming.
That's most encouraging to read! D has a lot of potential for embedded programming, we just need to be thrown a few bones.
 It isn't about whether we want to introduce a type qualifier 
 for low level programming or not. It is about what is the best 
 solution for MMIO for low level programming.
Exactly, and I think a type qualifier is the right tool for the job. I was all in favor of peek/poke after our brief conversation at DConf, but DIP62 sold me on a type qualifier... see below.
 Note that D already supports inp() and outp() as compiler 
 intrinsics, adding peek() and poke() will complement them 
 nicely:
OT: Isn't inp/outp and Intel-only thing, though? DIP62 addressed why peek/poke are not the right tool for the job in the "why a type qualifier" section. In summary, one would never want to access volatile memory with non-volatile semantics, and a type qualifier is the only proposal I've seen that enforces that: peek/poke intrinsics do not. The Volatile!(T) workaround is proof that peek/poke is not the right tool. The Volatile!(T) workaround is a jerry-rig on top of peek/poke to provide what a 'volatile' type qualifier would provide, and is necessary because the language lacks the proper tool. From what I've read in this thread, I suspect you favor peek/poke due to the complexity it introduces into the implementation. If a 'volatile' type qualifier was a trivial implementation, and did not introduce complexity into implementation, would your position be different? I and the others in favor of DIP62 will likely concede the following: * Workarounds exist * The workarounds are not overly burdensome * 'volatile' as defined by DIP62 would not be a pervasive feature of the language with high leverage. It is only used in the low level hardware abstractions (i.e. systems programming) ...but that's beside the point. I remember watching a talk you gave where you compared code with an airplane, and how when you get it right, it just looks like it wants to fly. I don't know how many people got that, but I sure did. peek/poke delegates responsibility to the programmer, as at every usage s/he must make sure they employ it properly to beat their code into flying. Volatile!(T) is an inelegant lever to make peek/poke more less prone to disobedience. A type qualifier is an elegant tool for modeling one's hardware because at every usage, it already knows how, and wants to fly. Mike
Jul 15 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 7/15/2014 6:03 PM, Mike wrote:
 OT:  Isn't inp/outp and Intel-only thing, though?
Of course. But it's still used on embedded systems.
 DIP62 addressed why peek/poke are not the right tool for the job in the "why a
 type qualifier" section.  In summary, one would never want to access volatile
 memory with non-volatile semantics, and a type qualifier is the only proposal
 I've seen that enforces that:  peek/poke intrinsics do not.
See my reply to Steve Sobel.
  From what I've read in this thread, I suspect you favor peek/poke due to the
 complexity it introduces into the implementation. If a 'volatile' type
qualifier
 was a trivial implementation, and did not introduce complexity into
 implementation, would your position be different?
Unfortunately, it isn't trivial, so your point is moot. Even if it were, "trivial" is not a good reason to put a feature in a language. The language must not become a kitchen sink grab-bag of trivia. Go has demonstrated there is considerable value in having a small set of orthogonal features instead.
 I remember watching a talk you gave where you compared code with an airplane,
 and how when you get it right, it just looks like it wants to fly.  I don't
know
 how many people got that, but I sure did.
You're the only one that likes my airplane metaphors :-)
 A type qualifier is an elegant tool for modeling one's hardware because at
every
 usage, it already knows how, and wants to fly.
Type qualifiers are not elegant. Type qualifiers are fairly burdensome to the user.
Jul 15 2014
prev sibling parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 15 July 2014 22:57, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 7/15/2014 2:17 PM, Johannes Pfau wrote:
 Well as long as those priorities are clearly communicated. If you tell
 me 'we don't care about embedded programming' then I'll shut up and move
 back to C.
Rest assured we definitely do care about embedded systems programming.
 But 'D is a systems programming language for low level tasks' and 'we
 don't want to introduce a type qualifier for low level programming, but
  nogc is just fine' don't go together. This leaves all contributors and
 devs hoping to see D on embedded systems in uncertainty.
It isn't about whether we want to introduce a type qualifier for low level programming or not. It is about what is the best solution for MMIO for low level programming. Note that D already supports inp() and outp() as compiler intrinsics, adding peek() and poke() will complement them nicely: http://dlang.org/phobos/core_bitop.html#.inp
s/D/DMD/ GDC doesn't support it as compiler intrinsics. Also, isn't inp/outp all x86-specific? I've never come across a similar thing in other architectures (at least, of the same name)
Jul 15 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/15/2014 11:23 PM, Iain Buclaw via Digitalmars-d wrote:
 GDC doesn't support it as compiler intrinsics.
I know.
 Also, isn't inp/outp all x86-specific?
Yes. This is not an argument that x86 compilers shouldn't support them. After all, D is supposed to be a systems programming language.
 I've never come across a similar thing in other architectures (at
 least, of the same name)
I have. Special I/O ports and associated special instructions used to be commonplace in microprocessors. Remember, D is supposed to be a systems programming language, not a language restricted to the least common denominator of every CPU now and in the future. One needs access to the system parts of the CPU.
Jul 16 2014
parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 16 July 2014 08:16, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 7/15/2014 11:23 PM, Iain Buclaw via Digitalmars-d wrote:
 GDC doesn't support it as compiler intrinsics.
I know.
 Also, isn't inp/outp all x86-specific?
Yes. This is not an argument that x86 compilers shouldn't support them. After all, D is supposed to be a systems programming language.
It certainly isn't! And it is never my intention to prevent optimisations on one platform if another doesn't benefit. I think my argument comes down to this, consider the following compiler intrisinsic. --- ubyte inp(uint port_address); It should instead be written as. --- ubyte inp()(uint port_address) { version(ARM) { asm { /* ... */ } } else static assert(false, "Not Implemented"); } What's going on here? 1) If the compiler recognises the function as an intrinsic, it discards the body and does whatever it knows to be correct. 2) If its not an intrinsic, it falls back to the library implementation. 3) So not to hurt targets that do not support inp/outp, it only errors at compile-time if *actually* used. As an alternative, instead of a template, we could use core.internal to do the same thing in this way. core/internal/bitops.d --- module core.bitops; version(ARM) { ubyte inp(uint port_address) { /* ... */ } } core/bitops.d --- module core.bitops; ubyte inp(uint port_address); But then it falls down to a linker error if 1) The compiler didn't expand inp() in the backend 2) core.internal.bitops doesn't implement inp() Which would you prefer? Regards Iain.
Jul 16 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 7/16/2014 1:35 AM, Iain Buclaw via Digitalmars-d wrote:
 Which would you prefer?
The simplest, which is just to use a version declaration to select whatever works for any particular compiler/CPU.
Jul 16 2014
prev sibling next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Johannes Pfau:

 //In runtime:
 enum int* SOME_REG = 0xFFFF;

 //In user code
 peek(SOME_REG);
 poke(SOME_REG);
 *SOME_REG = 1; //Oops, forgot poke! This is not acceptable
Perhaps this syntax: volatile enum int* SOME_REG = 0xFFFF; Could turn this in a syntax error (only peek/poke are allowed to write and read from this address): *SOME_REG = 1; Bye, bearophile
Jul 15 2014
parent reply Johannes Pfau <nospam example.com> writes:
Am Tue, 15 Jul 2014 21:16:20 +0000
schrieb "bearophile" <bearophileHUGS lycos.com>:

 Johannes Pfau:
 
 //In runtime:
 enum int* SOME_REG = 0xFFFF;

 //In user code
 peek(SOME_REG);
 poke(SOME_REG);
 *SOME_REG = 1; //Oops, forgot poke! This is not acceptable
Perhaps this syntax: volatile enum int* SOME_REG = 0xFFFF; Could turn this in a syntax error (only peek/poke are allowed to write and read from this address): *SOME_REG = 1; Bye, bearophile
You just reinvented this DIP ;-) Once you add a volatile type qualifier, there's no need to use peek/poke. The compiler can then easily detect reads/writes to these variables and do the right thing. The DIP may seem complicated cause it considers overloading, transitivity and similar stuff. In practice almost nobody will use these features. But if I had not specified them Walters first question would have been "How does it interact with overloading" ;-)
Jul 15 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 7/15/2014 2:22 PM, Johannes Pfau wrote:
 The DIP may seem complicated cause it considers overloading,
 transitivity and similar stuff. In practice almost nobody will use
 these features. But if I had not specified them Walters first question
 would have been "How does it interact with overloading" ;-)
You're right that almost nobody would use those features. But they'd have to be defined, documented, implemented and tested, like they are in C++. It negatively impacts users in that D becomes a larger, more complex language with next to no corresponding benefit. (Just grep for 'shared' in the D source code and see how pervasive it is.) This is why I propose that peek/poke have a much better cost/benefit ratio.
Jul 15 2014
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/15/2014 1:43 PM, Johannes Pfau wrote:
 1. Volatile has caused a major increase in the complexity of the C++
 type system
 - a complexity far out of proportion to the value it provides. It
 would have a similar complex and pervasive effect on the D type
 system.
Well as described in the DIP it works just like shared from an implementation point of view, so I doesn't add much complexity in the compiler / type system.
It's not so easy. All the interactions between volatile and other modifiers have to be carefully designed, documented, and implemented.
 Indeed there's nothing about peek/poke in DIP62, I thought I added that.
 The main point is that we can not leak pointers to volatile memory to
 users and pretend it's normal memory. This will lead to many problems:
I have a hard time seeing that MMIO is used so much in an embedded system that this will be a significant problem. And if it is, you can build a wrapper type and control access.
 This is like saying we don't need the shared qualifier as people
 should use atomicOp to access the data.
Shared use, however, is pervasive in concurrent code.
 But the problem there is that it produces quite some overhead right
 now.
Are you sure? The compiler is pretty good at inlining trivial functions.
 3. if you really hate peek/poke calls appearing in the code, you can
 use UFCS to make them look like variables
But you cant do REGISTER.peek() |= 0b1;
As you mentioned in the DIP, read-modify-write operations should be split up, as their semantics are murky.
 4. they are simple to explain and understand, they do not introduce a
 numbing complexity to the type system
Sorry, but I don't buy any of these complexity arguments.
No need to be sorry. Your opinion is not invalid.
 A few months
 ago you suggested namespaces for D - in addition to modules - which
 would have lead to a complexity disaster.
C++ namespaces are now implemented in D, and will they lead to complexity disaster? Certainly not in the implementation. For users? I don't think so, but we'll see.
 This just shows the priorities of the project leads:
 Desktop apps matter, we add  nogc, c++ namespaces, shared, immutable,
 const, -cov, ...
 but for embedded systems we won't even add one qualifier.
I don't think this is the right argument. It isn't about embedded systems should be acknowledged with a qualifier. I see it as about how often does MMIO logic appear in an actual embedded program? In the embedded systems I've written, it only appears in a handful of places. It is not pervasive. BTW, immutable data in D is designed to be placed in ROM for embedded systems. So there's already a qualifier :-)
 5. peek/poke will not add to the confusion about the difference
 between volatile and shared. No sane person would be tempted to use
 peek/poke to implement concurrency. The semantics of volatile in
 other languages won't confuse things for D peek/poke.
actually peek/poke are not that different from atomicOp from a syntax perspective ;-)
That's correct.
 You'll still have to educate people about the difference.
Peek/poke has a 40 year history of being used for MMIO, and I've never heard of it being used for concurrency. I don't think there's endemic confusion, quite unlike volatile.
Jul 15 2014
next sibling parent reply "Steve Sobel" <s.sobellian gmail.com> writes:
On Tuesday, 15 July 2014 at 21:28:24 UTC, Walter Bright wrote:
 I don't think this is the right argument. It isn't about 
 embedded systems should be acknowledged with a qualifier. I see 
 it as about how often does MMIO logic appear in an actual 
 embedded program? In the embedded systems I've written, it only 
 appears in a handful of places. It is not pervasive.

 BTW, immutable data in D is designed to be placed in ROM for 
 embedded systems. So there's already a qualifier :-)
I just wrote some embedded code for a Bluetooth LE microcontroller. The MCU in question has no less than 25 separate peripheral systems, each with its own set of registers, which use MMIO for configuration and operation. Just one of the header files provided by the chip vendor for this SoC defines 524 volatile variables. I'm sure that this complexity is barely anything compared to, say, the SoCs found in smart phones. Of course, one confines MMIO accesses to one layer of the program. Even so, that layer can be very broad given the sheer amount of stuff it has to touch - potentially many thousands of lines of code. With the code leverage that D has, I would not be surprised if this made up 10% or more of an embedded program's size. Also, there are many ways to misconfigure these peripherals and produce an effect physically similar to a compiler-eliminated access. A single omission of peek() or poke() could take hours to track down if one is unfortunate. If one is having trouble, for example, with the bluetooth radio, it could be due to: - An illegal configuration amongst the many registers touching BTLE - A legal configuration which is not accepted by the other device (e.g. causes a timeout) - The chip is damaged - The other device is damaged - The other device has a bug - The compiler eliminated access to the BTLE control registers Finally, the compiler could have eliminated access to other registers that had a side-effect on the BTLE code (for example, eliminating configuration of the memory protection unit). These would probably be incredibly pernicious because the true error is not in the code that has broken, and could feasibly be in any of a large number of systems. I certainly would not have a lot of fun scanning thousands of lines of code for a mistaken omission of peek() or poke(). So in my opinion, and without prescribing the exact solution, volatility is an important enough concept in embedded to merit a standard solution that can be mechanically verified.
Jul 15 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/15/2014 4:52 PM, Steve Sobel wrote:
 So in my opinion, and without prescribing the exact solution,
 volatility is an important enough concept in embedded to merit a
 standard solution that can be mechanically verified.
There is another way. The peek() and poke() functions are primitive. You can wrap a "pointer to volatile" in its own type, and in that type, control access to the pointer so it cannot be used outside of peek/poke. For example, off the top of my head: struct VolatilePointerToUint { private: size_t ptr; public: this(size_t ptr) { this.ptr = ptr; } uint read() { return peek(cast(uint*)ptr); } void write(uint value) { poke(cast(uint*)ptr, value); } } You'd probably wish to flesh this out a bit more, but it's the general idea. It's a zero cost abstraction. D has very capable abilities to create types that are restricted versions of other types - this should be explored and exhausted before considering language extensions.
Jul 15 2014
parent "bearophile" <bearophileHUGS lycos.com> writes:
Walter Bright:

 struct VolatilePointerToUint {
   private:
     size_t ptr;
   public:
     this(size_t ptr) { this.ptr = ptr; }
     uint read() { return peek(cast(uint*)ptr); }
     void write(uint value) { poke(cast(uint*)ptr, value); }
 }

 You'd probably wish to flesh this out a bit more, but it's the 
 general idea.
To be added to Phobos?
 It's a zero cost abstraction. D has very capable abilities to 
 create types that are restricted versions of other types - this 
 should be explored and exhausted before considering language 
 extensions.
See also the fantom suggested by Kagamin. Bye, bearophile
Jul 16 2014
prev sibling next sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Tue, 15 Jul 2014 14:28:23 -0700
schrieb Walter Bright <newshound2 digitalmars.com>:

 On 7/15/2014 1:43 PM, Johannes Pfau wrote:
 But the problem there is that it produces quite some overhead right
 now.
Are you sure? The compiler is pretty good at inlining trivial functions.
Yes, I actually tried tested that simple implementation with gdc. With aggressive optimizations the code inlines just fine and it'll generate the same asm as an access to a volatile variable in C. But it's not space-effecient: Even if it inlines, the function is still in the object file as a distinct function. There's TypeInfo and there's the assert(this), there's an initializer symbol and I guess there could be copy constructors as well. I expect many bug reports/fixes till we get a Volatile!T wrapper to be really space-efficient. I think if we decide that Volatile!T is the solution I'd probably hack an attribute TypeWrapper for gdc which simply recognized Volatile!T and does the right thing. The 'standard D' option requires forceinline noTypeinfo noInitializer noDebug and lots of boilerplate (all opAssign methods have to be implemented as we don't rewrite RMW operations into property get/set which also affects alias this to a property).
 
 3. if you really hate peek/poke calls appearing in the code, you
 can use UFCS to make them look like variables
But you cant do REGISTER.peek() |= 0b1;
As you mentioned in the DIP, read-modify-write operations should be split up, as their semantics are murky.
But not necessarily in user code. REGISTER |= is quite common in C and it's perfectly safe. If people will have to split this up into peek |= poke that's a step backwards from C.
 A few months
 ago you suggested namespaces for D - in addition to modules - which
 would have lead to a complexity disaster.
C++ namespaces are now implemented in D, and will they lead to complexity disaster? Certainly not in the implementation. For users? I don't think so, but we'll see.
You were initially in favor of an option which would have allowed to use namespaces in D code as well (namespace A {}). The current implementation is mostly limited to C++ so that's better, but my point was that nobody cared about complexity back then.
 
 This just shows the priorities of the project leads:
 Desktop apps matter, we add  nogc, c++ namespaces, shared,
 immutable, const, -cov, ...
 but for embedded systems we won't even add one qualifier.
I don't think this is the right argument. It isn't about embedded systems should be acknowledged with a qualifier. I see it as about how often does MMIO logic appear in an actual embedded program? In the embedded systems I've written, it only appears in a handful of places. It is not pervasive.
But it's not only about frequency, it's also about convenience and safety. How often is opDispatch actually used? The synchronized statement? Nevertheless they are there, because someone thought these use cases are important enough. The perceived importance of a special qualifier for MMIO code is highly subjective, but I think that most embedded programmers would say it's necessary. In Andrei's Leverage talk he said (5:57): "Consider this: Convenience, modeling power, efficiency: [...] You gonna have to have a large language. This is a fact of life. [...]". We have weak pure/ strong pure & immutable for functional programming, A complete set of OO primitives, builtin unit tests, code coverage, ... I think it's a correct argument to say that you think embedded system programming is not important enough for the additional complexity introduced by a new qualifier.
Jul 15 2014
next sibling parent reply "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Johannes Pfau"  wrote in message news:lq51u4$21qb$1 digitalmars.com...

 But it's not only about frequency, it's also about convenience and
 safety. How often is opDispatch actually used? The synchronized
 statement? Nevertheless they are there, because someone thought these
 use cases are important enough.
None of those affect the type system, this does. If it could be added as a storage class or even a new statement (!) it would be less intrusive.
 The perceived importance of a special qualifier for MMIO code is highly
 subjective, but I think that most embedded programmers would say it's
 necessary. In Andrei's Leverage talk he said (5:57): "Consider this:
 Convenience, modeling power, efficiency: [...] You gonna have to have a
 large language. This is a fact of life. [...]". We have
 weak pure/ strong pure & immutable for functional programming, A
 complete set of OO primitives, builtin unit tests, code coverage, ... I
 think it's a correct argument to say that you think embedded system
 programming is not important enough for the additional complexity
 introduced by a new qualifier.
The problem is not 'this feature is too complex', the problems is that it's more complex than necessary.
Jul 15 2014
parent reply Johannes Pfau <nospam example.com> writes:
Am Wed, 16 Jul 2014 15:59:46 +1000
schrieb "Daniel Murphy" <yebbliesnospam gmail.com>:

 The problem is not 'this feature is too complex', the problems is
 that it's more complex than necessary. 
So what's necessary in this specific case?
Jul 16 2014
parent reply "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Johannes Pfau"  wrote in message news:lq5pv1$2nfb$1 digitalmars.com...

 The problem is not 'this feature is too complex', the problems is
 that it's more complex than necessary.
So what's necessary in this specific case?
Use intrinsics with a nice template wrapper. They get inlined, the unused function bodies get stripped, and the debug info/typeinfo is just as big a problem as it is for every other function in your program. Intrinsics get us away from our current 'there is no portable way to do this' situation. They are _trivial_ to implement for any of the compilers. I have complete faith that it is possible to make a performant wrapper for both the single-register read/write and the struct pattern you posted. And if I'm wrong - we can add a volatile type modifier later! I just don't see volatile pulling it's weight.
Jul 16 2014
next sibling parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 16 July 2014 15:14, Daniel Murphy via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 "Johannes Pfau"  wrote in message news:lq5pv1$2nfb$1 digitalmars.com...


 The problem is not 'this feature is too complex', the problems is
 that it's more complex than necessary.
So what's necessary in this specific case?
Use intrinsics with a nice template wrapper. They get inlined, the unused function bodies get stripped, and the debug info/typeinfo is just as big a problem as it is for every other function in your program. Intrinsics get us away from our current 'there is no portable way to do this' situation.
No they don't. Intrinsics make things worse.
 They are _trivial_ to implement for any of the compilers.
No they aren't, unless you are talking specifically about volatile., in which case, that depends...
 I have complete faith that it is possible to make a performant wrapper for
 both the single-register read/write and the struct pattern you posted.  And
 if I'm wrong - we can add a volatile type modifier later!

 I just don't see volatile pulling it's weight.
Hardware access is supposed to be correct, not performant. Regards Iain
Jul 16 2014
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/16/2014 7:31 AM, Iain Buclaw via Digitalmars-d wrote:
 They are _trivial_ to implement for any of the compilers.
No they aren't, unless you are talking specifically about volatile., in which case, that depends...
If the compiler back end doesn't support volatile, then it's a problem regardless of how the front end handles it, so the point is moot. In any case, the compiler front end can always implement an intrinsic by just calling a function.
Jul 16 2014
next sibling parent Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 16 July 2014 19:58, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 7/16/2014 7:31 AM, Iain Buclaw via Digitalmars-d wrote:
 They are _trivial_ to implement for any of the compilers.
No they aren't, unless you are talking specifically about volatile., in which case, that depends...
If the compiler back end doesn't support volatile, then it's a problem regardless of how the front end handles it, so the point is moot. In any case, the compiler front end can always implement an intrinsic by just calling a function.
I was thinking: http://bugzilla.gdcproject.org/show_bug.cgi?id=84
Jul 16 2014
prev sibling parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 16 July 2014 20:03, Iain Buclaw <ibuclaw gdcproject.org> wrote:
 On 16 July 2014 19:58, Walter Bright via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On 7/16/2014 7:31 AM, Iain Buclaw via Digitalmars-d wrote:
 They are _trivial_ to implement for any of the compilers.
No they aren't, unless you are talking specifically about volatile., in which case, that depends...
If the compiler back end doesn't support volatile, then it's a problem regardless of how the front end handles it, so the point is moot. In any case, the compiler front end can always implement an intrinsic by just calling a function.
I was thinking: http://bugzilla.gdcproject.org/show_bug.cgi?id=84
By the way, it's 'so-called' bugs like this that make we want to remove any notion of volatile from shared. Regards Iain
Jul 16 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 7/16/2014 12:10 PM, Iain Buclaw via Digitalmars-d wrote:
 By the way, it's 'so-called' bugs like this that make we want to
 remove any notion of volatile from shared.
Right.
Jul 16 2014
prev sibling parent "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Iain Buclaw via Digitalmars-d"  wrote in message 
news:mailman.4214.1405521071.2907.digitalmars-d puremagic.com...

 No they don't.  Intrinsics make things worse.
? At worst they're useless.
 They are _trivial_ to implement for any of the compilers.
No they aren't, unless you are talking specifically about volatile., in which case, that depends...
Yes I'm talking about the peek/poke intrinsics. As all D backends are C backends, and C supports these semantics already...
 Hardware access is supposed to be correct, not performant.
Ideally both!
Jul 17 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/16/2014 7:14 AM, Daniel Murphy wrote:
 Intrinsics get us away from our current 'there is no portable way to do this'
 situation.  They are _trivial_ to implement for any of the compilers. I have
 complete faith that it is possible to make a performant wrapper for both the
 single-register read/write and the struct pattern you posted.  And if I'm wrong
 - we can add a volatile type modifier later!
I'll add that if it turns out we can't do a wrapper because of some limitations/bugs in D's expressiveness, that is the problem with D we need to fix, not adding another type modifier. It all comes down to leverage. Making user defined types more capable has far, far more productive leverage than welding specifics into the compiler.
Jul 16 2014
next sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Wed, 16 Jul 2014 11:49:21 -0700
schrieb Walter Bright <newshound2 digitalmars.com>:

 On 7/16/2014 7:14 AM, Daniel Murphy wrote:
 Intrinsics get us away from our current 'there is no portable way
 to do this' situation.  They are _trivial_ to implement for any of
 the compilers. I have complete faith that it is possible to make a
 performant wrapper for both the single-register read/write and the
 struct pattern you posted.  And if I'm wrong
 - we can add a volatile type modifier later!
I'll add that if it turns out we can't do a wrapper because of some limitations/bugs in D's expressiveness, that is the problem with D we need to fix, not adding another type modifier. It all comes down to leverage. Making user defined types more capable has far, far more productive leverage than welding specifics into the compiler.
I'll take this as you pre-approve all the mentioned extensions? * way to disable typeinfo for struct * way to disable initializer * force inlining * (way to omit copy constructor function / force inline)
Jul 16 2014
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/16/2014 1:12 PM, Johannes Pfau wrote:
 I'll take this as you pre-approve all the mentioned extensions?

 * way to disable typeinfo for struct
Worth investigating. It'd probably be much more effective to work the linker angle of removing items for which no references exist. Martin was working on that, but his latest attempt had to be reverted, as it needed a new approach.
 * way to disable initializer
T t = void; // no initialization
 * force inlining
http://wiki.dlang.org/DIP56 (Note the author!)
 * (way to omit copy constructor function / force inline)
I don't know what that means
Jul 16 2014
parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 17 July 2014 07:09, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 7/16/2014 1:12 PM, Johannes Pfau wrote:
 I'll take this as you pre-approve all the mentioned extensions?

 * way to disable typeinfo for struct
Worth investigating. It'd probably be much more effective to work the linker angle of removing items for which no references exist. Martin was working on that, but his latest attempt had to be reverted, as it needed a new approach.
 * way to disable initializer
T t = void; // no initialization
 * force inlining
http://wiki.dlang.org/DIP56 (Note the author!)
For the time being, GDC has attribute("forceinline") https://github.com/D-Programming-GDC/GDC/blob/master/libphobos/libdruntime/gcc/attribute.d
 * (way to omit copy constructor function / force inline)
I don't know what that means
disable this(this); ? I'll let Johannes clarify. Iain
Jul 17 2014
parent reply Johannes Pfau <nospam example.com> writes:
Am Thu, 17 Jul 2014 08:05:33 +0100
schrieb Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com>:

 On 17 July 2014 07:09, Walter Bright via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On 7/16/2014 1:12 PM, Johannes Pfau wrote:
 I'll take this as you pre-approve all the mentioned extensions?

 * way to disable typeinfo for struct
Worth investigating. It'd probably be much more effective to work the linker angle of removing items for which no references exist. Martin was working on that, but his latest attempt had to be reverted, as it needed a new approach.
 * way to disable initializer
T t = void; // no initialization
I'm referring to the __initZ symbol/data, it gets emitted even if all fields of a struct are =void initialized.
 * force inlining
http://wiki.dlang.org/DIP56 (Note the author!)
For the time being, GDC has attribute("forceinline") https://github.com/D-Programming-GDC/GDC/blob/master/libphobos/libdruntime/gcc/attribute.d
Yes. We'll have to take a look if we can prevent the function from appearing in the object file (with force inline, there's no need to keep the function, it must always be inlined anyway)
 
 * (way to omit copy constructor function / force inline)
I don't know what that means
disable this(this); ? I'll let Johannes clarify.
I was referring to the implicitly generated __cpctor not the postblit. As it's implicitly generated you can't put force-inline on it. This applies to all compiler generated functions though, __fieldPostBlit, __fieldDtor autogenerated opAssign and probably some more. However, simple (POD?) types are probably not affected as those don't get a __cpctor IIRC?
Jul 17 2014
parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 17 July 2014 21:01, Johannes Pfau via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 Am Thu, 17 Jul 2014 08:05:33 +0100
 schrieb Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com>:

 On 17 July 2014 07:09, Walter Bright via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On 7/16/2014 1:12 PM, Johannes Pfau wrote:
 I'll take this as you pre-approve all the mentioned extensions?

 * way to disable typeinfo for struct
Worth investigating. It'd probably be much more effective to work the linker angle of removing items for which no references exist. Martin was working on that, but his latest attempt had to be reverted, as it needed a new approach.
 * way to disable initializer
T t = void; // no initialization
I'm referring to the __initZ symbol/data, it gets emitted even if all fields of a struct are =void initialized.
 * force inlining
http://wiki.dlang.org/DIP56 (Note the author!)
For the time being, GDC has attribute("forceinline") https://github.com/D-Programming-GDC/GDC/blob/master/libphobos/libdruntime/gcc/attribute.d
Yes. We'll have to take a look if we can prevent the function from appearing in the object file (with force inline, there's no need to keep the function, it must always be inlined anyway)
Just mark them as extern. More important, we need a way to build the bodies of functions in other modules for cross-module inlining (needed for forceinline to work correctly without getting 'function body is not available' errors).
Jul 17 2014
parent "David Nadlinger" <code klickverbot.at> writes:
On Thursday, 17 July 2014 at 20:42:16 UTC, Iain Buclaw via 
Digitalmars-d wrote:
 Just mark them as extern.  More important, we need a way to 
 build the
 bodies of functions in other modules for cross-module inlining 
 (needed
 for  forceinline to work correctly without getting 'function 
 body is
 not available' errors).
Jernej Krempuš put something together for LDC a while back: https://github.com/jerro/ldc/compare/inlining. Unfortunately, it seems like he never got around to submitting the symbol naming fixes to the upstream frontend. Cheers, David
Jul 17 2014
prev sibling next sibling parent "Kagamin" <spam here.lot> writes:
On Wednesday, 16 July 2014 at 20:11:22 UTC, Johannes Pfau wrote:
 I think it's kinda ridiculous that D embedded code will only be 
 usable
 with strong optimization flags, but whatever.
Do you have numbers? Transitive volatility implies you planned to do very complex things in embedded. If optimizations are so critical, how do you know if complex code can work without them due to its sheer complexity even if typeinfos were removed somehow? On Wednesday, 16 July 2014 at 20:14:38 UTC, Johannes Pfau wrote:
 I'll take this as you pre-approve all the mentioned extensions?

 * way to disable typeinfo for struct
BTW, typeinfos are needed for basic types too and not everything is inline - e.g. array comparison.
 * way to disable initializer
 * force inlining
 * (way to omit copy constructor function / force inline)
We have bugzilla for issues and you can refer to them (with appropriate severity) to describe a level of support for embedded (and baremetal). Would be better with numbers to justify severity.
Jul 17 2014
prev sibling parent "Kagamin" <spam here.lot> writes:
On Wednesday, 16 July 2014 at 20:14:38 UTC, Johannes Pfau wrote:
 * way to disable typeinfo for struct
 * way to disable initializer
 * force inlining
 * (way to omit copy constructor function / force inline)
BTW, can't lto already remove this unused data? LDC inlines simple initializers already, so they are likely to end up unused (same for functions which where inlined).
Jul 17 2014
prev sibling parent reply Byron Heads <byron.heads gmail.com> writes:
On Wed, 16 Jul 2014 11:49:21 -0700, Walter Bright wrote:

 
 I'll add that if it turns out we can't do a wrapper because of some
 limitations/bugs in D's expressiveness, that is the problem with D we
 need to fix, not adding another type modifier.
 
 It all comes down to leverage. Making user defined types more capable
 has far,
 far more productive leverage than welding specifics into the compiler.
A Volatile(T) in stdlib seems to make a lot of sense, I am also wondering if the implementation of shared should be done the same way (if it does already then ignore), just use the compile to enforce shared and have it rewrite it as Shared(T).
Jul 17 2014
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 07/17/2014 04:01 PM, Byron Heads wrote:
 A Volatile(T) in stdlib seems to make a lot of sense, I am also wondering
 if the implementation of shared should be done the same way (if it does
 already then ignore), just use the compile to enforce shared and have it
 rewrite it as Shared(T).
 ...
I've grepped the DMD source and currently 'shared' is a _bit flag_ set or not set on some uint member field of the 'Type' class. I assume this is also the main reason why adding new type qualifiers is a really unpleasant undertaking: this is a highly non-modular design.
Jul 18 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 7/18/2014 10:53 AM, Timon Gehr wrote:
 I've grepped the DMD source and currently 'shared' is a _bit flag_ set or not
 set on some uint member field of the 'Type' class. I assume this is also the
 main reason why adding new type qualifiers is a really unpleasant undertaking:
 this is a highly non-modular design.
I also recommend that you grep the C++ Standard for 'volatile'. I counted 137 occurrences by page 300 of 1353 pages, at that point I gave up. And those occurrences were not clustered, they were sprinkled everywhere. A C++ compiler is obliged to carefully account for each and every semantic issue in the Standard. Just writing test cases for all those mentions is a major undertaking.
Jul 18 2014
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/15/2014 10:15 PM, Johannes Pfau wrote:
 Yes, I actually tried tested that simple implementation with gdc. With
 aggressive optimizations the code inlines just fine and it'll generate
 the same asm as an access to a volatile variable in C. But it's not
 space-effecient: Even if it inlines, the function is still in the
 object file as a distinct function. There's TypeInfo and there's the
 assert(this), there's an initializer symbol and I guess there could be
 copy constructors as well. I expect many bug reports/fixes till we get
 a Volatile!T wrapper to be really space-efficient.
D is intended to have a robust enough feature set in order to implement types as library types rather than builtin, and have them operate as good as a builtin type would. These are more general issues, and are not specific to volatile.
 3. if you really hate peek/poke calls appearing in the code, you
 can use UFCS to make them look like variables
But you cant do REGISTER.peek() |= 0b1;
As you mentioned in the DIP, read-modify-write operations should be split up, as their semantics are murky.
But not necessarily in user code. REGISTER |= is quite common in C and it's perfectly safe.
The point is it is not perfectly safe. There is no definition, and certainly no portable definition, as to how many read/write cycles such an operation entails. That's what I meant, and DIP62 says pretty much said the same thing. Please allow me to quote: "CISC machines may allow other operations to memory than simple load/store operations. The question is then if a read-modify-write operation like 'x++' should translate to 'r = load x; r++; x = store r' or to a single add instruction 'add x;'. This DIP does not dictate any specific behaviour, but this is recommended: If it is known that instructions operating on memory do not work for memory mapped IO as expected, compilers should default to generating a load/modify/store sequence. Otherwise compilers should generate the same code sequence as for regular variables. It is recommended to provide a compiler switch on CISC architectures to allow the user to choose between these two different behaviours."
 If people will have to split this up into peek |=
 poke that's a step backwards from C.
Support for op= is fairly simple for VolatilePointerToUint to implement, and it can implement them in terms of peek/poke for precise and portable semantics.
 A few months
 ago you suggested namespaces for D - in addition to modules - which
 would have lead to a complexity disaster.
C++ namespaces are now implemented in D, and will they lead to complexity disaster? Certainly not in the implementation. For users? I don't think so, but we'll see.
You were initially in favor of an option which would have allowed to use namespaces in D code as well (namespace A {}). The current implementation is mostly limited to C++ so that's better, but my point was that nobody cared about complexity back then.
I objected to several proposals for namespace support that I regarded as significantly more complex.
 This just shows the priorities of the project leads:
 Desktop apps matter, we add  nogc, c++ namespaces, shared,
 immutable, const, -cov, ...
 but for embedded systems we won't even add one qualifier.
I don't think this is the right argument. It isn't about embedded systems should be acknowledged with a qualifier. I see it as about how often does MMIO logic appear in an actual embedded program? In the embedded systems I've written, it only appears in a handful of places. It is not pervasive.
But it's not only about frequency, it's also about convenience and safety. How often is opDispatch actually used? The synchronized statement? Nevertheless they are there, because someone thought these use cases are important enough.
The synchronized statement was something I thought would be widely used. It turns out to be largely a failure. opDispatch is actually pretty simple, and I believe it is an "enabling" feature that hasn't, but will, come into its own. Nevertheless, we should not follow the trap of "we made a mistake on A, so that justifies making another mistake B". Nor should we say "feature X has reasons A,B,C,D, therefore reason D is justification enough for other things." Changes to D must stand/fall on their own merits.
 The perceived importance of a special qualifier for MMIO code is highly
 subjective,
Of course. There is no doubt that both yours and my opinions here are subjective.
 but I think that most embedded programmers would say it's
 necessary.
I do believe that a wrapper type around peek/poke satisfies essentially all your necessary requirements, except for transitivity.
 In Andrei's Leverage talk he said (5:57): "Consider this:
 Convenience, modeling power, efficiency: [...] You gonna have to have a
 large language. This is a fact of life. [...]". We have
 weak pure/ strong pure & immutable for functional programming, A
 complete set of OO primitives, builtin unit tests, code coverage, ... I
 think it's a correct argument to say that you think embedded system
 programming is not important enough for the additional complexity
 introduced by a new qualifier.
That is not what I'm saying at all. Please do not conflate disagreement about the best way to achieve a goal with disagreement about the goal. I understand that you and others put a lot of work into DIP62. As I said before, it's a very nice example of a well-done DIP.
Jul 15 2014
parent reply Johannes Pfau <nospam example.com> writes:
Am Tue, 15 Jul 2014 23:38:21 -0700
schrieb Walter Bright <newshound2 digitalmars.com>:

 On 7/15/2014 10:15 PM, Johannes Pfau wrote:
 Yes, I actually tried tested that simple implementation with gdc.
 With aggressive optimizations the code inlines just fine and it'll
 generate the same asm as an access to a volatile variable in C. But
 it's not space-effecient: Even if it inlines, the function is still
 in the object file as a distinct function. There's TypeInfo and
 there's the assert(this), there's an initializer symbol and I guess
 there could be copy constructors as well. I expect many bug
 reports/fixes till we get a Volatile!T wrapper to be really
 space-efficient.
D is intended to have a robust enough feature set in order to implement types as library types rather than builtin, and have them operate as good as a builtin type would. These are more general issues, and are not specific to volatile.
And how do you implement this with a wrapper? This is common in C/C++: struct Timer { uint control; uint data; } enum timerA = (Volatile!Timer)* = cast()oxDEADBEAF; timerA.control |= 0b1; Your peek/poke intrinsics don't even consider this example. If you extend peek/poke to T, then it'll always load the complete type, whereas the C code loads only the field. And you can't even do anything about it, cause peek!T explicitly tells the compiler to load the complete type and prevent optimization of this load... And it seems timerA.control |= 0b1; produces absolute nonsense. It compiles, but * The codegen is horrible. In C this compiles to 3 instructions, in D 30-40 instructions * This doesn't do what it's supposed to do at all. For example it calls the getter twice and never calls the setter. The more I think about it the more corner cases appear. After all implementing this in the library is much more complex than a type qualifier. Some of the complexity is already implemented (alias this, ...) so there's less changes in DMD. But any bug in alias this, templates, etc will immediately leak into Volatile!T.
 
 3. if you really hate peek/poke calls appearing in the code, you
 can use UFCS to make them look like variables
But you cant do REGISTER.peek() |= 0b1;
As you mentioned in the DIP, read-modify-write operations should be split up, as their semantics are murky.
But not necessarily in user code. REGISTER |= is quite common in C and it's perfectly safe.
The point is it is not perfectly safe. There is no definition, and certainly no portable definition, as to how many read/write cycles such an operation entails. That's what I meant, and DIP62 says pretty much said the same thing. Please allow me to quote:
In cases where you use |= the exact number of accesses usually does not matter. What matters is that there's at least one access and that it's not reordered with other reads/writes to volatile. For example |= is used on control registers. But I don't care if I read/set the bit twice or once, I just doesn't matter.
 
 "CISC machines may allow other operations to memory than simple
 load/store operations. The question is then if a read-modify-write
 operation like 'x++' should translate to 'r = load x; r++; x = store
 r' or to a single add instruction 'add x;'. This DIP does not dictate
 any specific behaviour, but this is recommended: If it is known that
 instructions operating on memory do not work for memory mapped IO as
 expected, compilers should default to generating a load/modify/store
 sequence. Otherwise compilers should generate the same code sequence
 as for regular variables. It is recommended to provide a compiler
 switch on CISC architectures to allow the user to choose between
 these two different behaviours."
And what does peek/poke guarantee in that case? Everything you could specify for peek/poke can be specified for volatile variables as well.
 
 If people will have to split this up into peek |=
 poke that's a step backwards from C.
Support for op= is fairly simple for VolatilePointerToUint to implement, and it can implement them in terms of peek/poke for precise and portable semantics.
See above
 I do believe that a wrapper type around peek/poke satisfies
 essentially all your necessary requirements, except for transitivity.
See above. One important point is that this must be as effective as c+volatile (in terms of code size and execution cycles). This is an important aspect for embedded programming and it'll take ages and many language changes to make a type wrapper as efficient. If it's even possible at all.
 I think it's a correct argument to say that you think
 embedded system programming is not important enough for the
 additional complexity introduced by a new qualifier.
That is not what I'm saying at all. Please do not conflate disagreement about the best way to achieve a goal with disagreement about the goal.
So you do think peek/poke is a better solution than DIP62? Then you have to explain this. All arguments so far have been it's too complex in the compiler implementation, but never that peek/poke is a better way to achieve that goal of accessing volatile memory. In the end it's a judgement call between complexity and more convenience for embedded programming and it's absolutely your decision. But as long as I'm not convinced that peek/poke is a _better_ solution I'll think that embedded programming is not important enough for you to implement the _best_ solution, as it causes some implementation complexity. And this in the end means it's of lower priority to you than keeping some complexity out of the compiler.
 I understand that you and others put a lot of work into DIP62. As I
 said before, it's a very nice example of a well-done DIP.
 
That's not the point. If I were convinced there's a better solution, I'd be glad to accept it. But I'm not convinced at all, the only argument against DIP62 so far was complexity.
Jul 16 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 7/16/2014 5:59 AM, Johannes Pfau wrote:
 And how do you implement this with a wrapper? This is common in C/C++:

 struct Timer
 {
      uint control;
      uint data;
 }

 enum timerA = (Volatile!Timer)* = cast()oxDEADBEAF;

 timerA.control |= 0b1;
auto timerA = VolatileStruct!Timer(0xDEADBEAF); timerA.control |= 1; i.e. with introspection, the template can automatically generate the getters/setters and operator overloads.
 The more I think about it the more corner cases appear. After all
 implementing this in the library is much more complex than a type
 qualifier. Some of the complexity is already implemented (alias
 this, ...) so there's less changes in DMD. But any bug in alias this,
 templates, etc will immediately leak into Volatile!T.
It's true that VolatileStruct will be more complex than the average template. But it's quite doable, and the user won't see that complexity.
 The point is it is not perfectly safe. There is no definition, and
 certainly no portable definition, as to how many read/write cycles
 such an operation entails. That's what I meant, and DIP62 says pretty
 much said the same thing. Please allow me to quote:
In cases where you use |= the exact number of accesses usually does not matter. What matters is that there's at least one access and that it's not reordered with other reads/writes to volatile.
I understand, but the pedantic in me notes the word "usually".
 For example |= is used on control registers. But I don't care if I
 read/set the bit twice or once, I just doesn't matter.
I understand that it usually does not matter. But sometimes it does. It should be under user control rather than "whatever the compiler does" which can change with optimization settings. C compiler semantics for volatile are "implementation defined", which pretty much means "random".
 And what does peek/poke guarantee in that case?
It guarantees semantics for (A op= B) to be (A = A op B).
 One important point is that this must be as effective as
 c+volatile (in terms of code size and execution cycles). This is an
 important aspect for embedded programming
I fully agree.
 and it'll take ages and many
 language changes to make a type wrapper as efficient.
This is unduly pessimistic.
 If it's even possible at all.
I see no reason to quit before we start.
 That is not what I'm saying at all. Please do not conflate
 disagreement about the best way to achieve a goal with disagreement
 about the goal.
So you do think peek/poke is a better solution than DIP62? Then you have to explain this. All arguments so far have been it's too complex in the compiler implementation, but never that peek/poke is a better way to achieve that goal of accessing volatile memory. In the end it's a judgement call between complexity and more convenience for embedded programming and it's absolutely your decision. But as long as I'm not convinced that peek/poke is a _better_ solution I'll think that embedded programming is not important enough for you to implement the _best_ solution, as it causes some implementation complexity. And this in the end means it's of lower priority to you than keeping some complexity out of the compiler.
I think it unfair to impute motives when I've specifically said otherwise.
 I understand that you and others put a lot of work into DIP62. As I
 said before, it's a very nice example of a well-done DIP.
That's not the point.
I think it is fair to acknowledge the good work you've done.
 If I were convinced there's a better solution,
 I'd be glad to accept it. But I'm not convinced at all, the only
 argument against DIP62 so far was complexity.
Complexity is not a minor issue. Complexity means time and bugs. I am all too familiar with how volatile in C++ has been costly in a lot of effort, bugs, specification, and code that had no use for volatile, yet had to account for it. BTW, testing of "volatile" behavior in the compiler is problematic as there just doesn't seem to be a reasonable way to do it. This engenders a regular risk of compiler breakage, and shows up to the end user as erratic behavior and lots of cursing at the compiler vendor.
Jul 16 2014
prev sibling next sibling parent reply "Kagamin" <spam here.lot> writes:
On Wednesday, 16 July 2014 at 05:17:56 UTC, Johannes Pfau wrote:
 I think if we decide that Volatile!T is the solution I'd 
 probably hack
 an attribute  TypeWrapper for gdc which simply recognized 
 Volatile!T
 and does the right thing.
I'd recommend fantom name, because not only type wrappers can benefit from these optimizations. Should be also applicable to functions and maybe modules.
Jul 16 2014
parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Kagamin:

 I'd recommend  fantom name, because not only type wrappers can 
 benefit from these optimizations. Should be also applicable to 
 functions and maybe modules.
This sounds like something fit for D too, not just gdc. You can use it for std.typecons.Typedef too. Bye, bearophile
Jul 16 2014
parent "Kagamin" <spam here.lot> writes:
Then we can have phantom functions for porting C macros instead 
of templates :) (oh it's spelled with 'ph').
Jul 16 2014
prev sibling parent "Kagamin" <spam here.lot> writes:
On Wednesday, 16 July 2014 at 05:17:56 UTC, Johannes Pfau wrote:
 There's TypeInfo and there's the assert(this)
The optimizer should be able to eliminate this assert after inlining as it knows it's not null. It should be looked into, why the optimizer didn't remove it.
Jul 16 2014
prev sibling next sibling parent "ponce" <contact gam3sfrommars.fr> writes:
On Tuesday, 15 July 2014 at 21:28:24 UTC, Walter Bright wrote:
 Peek/poke has a 40 year history of being used for MMIO, and 
 I've never heard of it being used for concurrency. I don't 
 think there's endemic confusion, quite unlike volatile.
There is a huge confusion about volatile in the native space. No one really knows what it does and when it should be used, because it's kind of rare. I've seen pervasive volatile use make some GPU kernels 100x slower. Programming by coincidence, sure, but _explicit_ barriers, being read/write/optimizer barriers, I find them much better. volatile is more like denial, and importing the history behind the keyword would negative PR. Pascal had no volatile, and using &= from a memory-mapped location doesn't exactly scream "performance".
Jul 16 2014
prev sibling parent reply "Sean Kelly" <sean invisibleduck.org> writes:
We had the volatile statement as a compiler barrier in D1. Why 
not basically that instead of a type qualifier?  We pretty much 
need it back for atomics anyway.
Jul 17 2014
next sibling parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 17 July 2014 17:16, Sean Kelly via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 We had the volatile statement as a compiler barrier in D1. Why not basically
 that instead of a type qualifier?  We pretty much need it back for atomics
 anyway.
I don't recall volatile statements ever working properly, or at least working in an agreed way.
Jul 17 2014
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 7/17/14, 12:15 PM, Iain Buclaw via Digitalmars-d wrote:
 On 17 July 2014 17:16, Sean Kelly via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 We had the volatile statement as a compiler barrier in D1. Why not basically
 that instead of a type qualifier?  We pretty much need it back for atomics
 anyway.
I don't recall volatile statements ever working properly, or at least working in an agreed way.
And that goes for volatile in C and C++ as well. -- Andrei
Jul 17 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/17/2014 9:16 AM, Sean Kelly wrote:
 We had the volatile statement as a compiler barrier in D1. Why not basically
 that instead of a type qualifier?  We pretty much need it back for atomics
anyway.
Volatile and atomic semantics are very different, are historically conflated and confused, and I think it's well worth it to use completely distinct mechanisms for both.
Jul 17 2014
parent "Sean Kelly" <sean invisibleduck.org> writes:
On Thursday, 17 July 2014 at 19:31:25 UTC, Walter Bright wrote:
 On 7/17/2014 9:16 AM, Sean Kelly wrote:
 We had the volatile statement as a compiler barrier in D1. Why 
 not basically
 that instead of a type qualifier?  We pretty much need it back 
 for atomics anyway.
Volatile and atomic semantics are very different, are historically conflated and confused, and I think it's well worth it to use completely distinct mechanisms for both.
Fair enough. My point was simply that volatile as defined for D1 seems potentially useful. I certainly liked it for concurrent programming, even if the guarantees it provided were only ever on paper and not actually implemented.
Jul 17 2014
prev sibling parent "safety0ff" <safety0ff.dev gmail.com> writes:
On Tuesday, 15 July 2014 at 20:45:20 UTC, Johannes Pfau wrote:
 Am Tue, 15 Jul 2014 12:48:21 -0700
 schrieb Walter Bright <newshound2 digitalmars.com>:
 
 3. if you really hate peek/poke calls appearing in the code, 
 you can
 use UFCS to make them look like variables
But you cant do REGISTER.peek() |= 0b1;
You can create a function which returns a struct that wraps your desired read-modify-write semantics, e.g.: auto myvolatile(size_t* addr) { static struct VolatileWrapper { size_t* addr; // read-modify-write auto opOpAssign(string op)(size_t rhs) { /*...*/ } // write auto opAssign(size_t rhs) { addr.poke(rhs); return this; } // read auto rval() { return addr.peek(); } alias rval this; } return VolatileWrapper(addr); } REGISTER.myvolatile() |= 0b1; Taking it further, I think you could implement something that looked like this: enum : VolatileWrapper { TIMER0 = VolatileWrapper(REGISTER), // etc } Don't get me wrong, I'm in favor of volatile, it's just that that counter-argument isn't extremely convincing.
Jul 15 2014
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
I've been jawboning about peek/poke for years - finally decided to implement it.

   https://issues.dlang.org/show_bug.cgi?id=13138
Jul 15 2014
next sibling parent reply Brad Roberts via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 7/15/14, 6:25 PM, Walter Bright via Digitalmars-d wrote:
 I've been jawboning about peek/poke for years - finally decided to implement
it.

    https://issues.dlang.org/show_bug.cgi?id=13138
I don't have any skin in this particular discussion, but it's worth pointing out here that while it's cute that builtin's can be added, when they're not part of the language definition, they're not really part of a portable solution.
Jul 15 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/15/2014 9:29 PM, Brad Roberts via Digitalmars-d wrote:
 I don't have any skin in this particular discussion, but it's worth pointing
out
 here that while it's cute that builtin's can be added, when they're not part of
 the language definition, they're not really part of a portable solution.
They're as portable as we want them to be.
Jul 15 2014
parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 16 July 2014 07:12, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 7/15/2014 9:29 PM, Brad Roberts via Digitalmars-d wrote:
 I don't have any skin in this particular discussion, but it's worth
 pointing out
 here that while it's cute that builtin's can be added, when they're not
 part of
 the language definition, they're not really part of a portable solution.
They're as portable as we want them to be.
I read that as, not portable at all then. :o)
Jul 15 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/15/2014 11:32 PM, Iain Buclaw via Digitalmars-d wrote:
 On 16 July 2014 07:12, Walter Bright via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On 7/15/2014 9:29 PM, Brad Roberts via Digitalmars-d wrote:
 I don't have any skin in this particular discussion, but it's worth
 pointing out
 here that while it's cute that builtin's can be added, when they're not
 part of
 the language definition, they're not really part of a portable solution.
They're as portable as we want them to be.
I read that as, not portable at all then. :o)
C'mon, guys. It's our spec and we can make it work.
Jul 16 2014
parent Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 16 July 2014 08:08, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 7/15/2014 11:32 PM, Iain Buclaw via Digitalmars-d wrote:
 On 16 July 2014 07:12, Walter Bright via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On 7/15/2014 9:29 PM, Brad Roberts via Digitalmars-d wrote:
 I don't have any skin in this particular discussion, but it's worth
 pointing out
 here that while it's cute that builtin's can be added, when they're not
 part of
 the language definition, they're not really part of a portable solution.
 They're as portable as we want them to be.
I read that as, not portable at all then. :o)
C'mon, guys. It's our spec and we can make it work.
And we are trying to make it work. I actually have quite a bit to be planned out after removing the 'volatility' from shared.
From the following spec:
''' Unlike the shared attribute, __gshared provides no safe-guards against data races or other multi-threaded synchronization issues. It is the responsibility of the programmer to ensure that access to variables marked __gshared is synchronized correctly. ''' GDC will likely in future emit atomic loads, stores, and possibly fences around the use of shared data. As there is no specific meaning to what 'safe-guards' come with shared. I interpret that to keep in line with a happens-before style relationship with sequential reads/writes to shared data. Regards Iain
Jul 16 2014
prev sibling parent "Mike" <none none.com> writes:
On Wednesday, 16 July 2014 at 01:25:05 UTC, Walter Bright wrote:
 I've been jawboning about peek/poke for years - finally decided 
 to implement it.

   https://issues.dlang.org/show_bug.cgi?id=13138
If this is indeed the last straw, please mark DIP20 as accepted/implemented and DIP62 as rejected. I have neither the authority nor the stomach for it. Mike DIP20: http://wiki.dlang.org/DIP20 DIP62: http://wiki.dlang.org/DIP62
Jul 15 2014
prev sibling parent reply "Mike" <none none.com> writes:
On Tuesday, 15 July 2014 at 19:48:23 UTC, Walter Bright wrote:

 The one thing peek/poke doesn't offer is transitivity. C/C++ 
 don't offer volatile transitivity either. I'm not at all sure 
 that anyone builds a data structure in memory-mapped registers, 
 so I'm not convinced this is a need.
Here's an example from the STM32F4 ARM Cortex-M peripheral library: #define __IO volatile /** * brief DMA Controller */ typedef struct { __IO uint32_t CR; /*!< DMA stream x configuration register */ __IO uint32_t NDTR; /*!< DMA stream x number of data register */ __IO uint32_t PAR; /*!< DMA stream x peripheral address register */ __IO uint32_t M0AR; /*!< DMA stream x memory 0 address register */ __IO uint32_t M1AR; /*!< DMA stream x memory 1 address register */ __IO uint32_t FCR; /*!< DMA stream x FIFO control register */ } DMA_Stream_TypeDef; typedef struct { __IO uint32_t LISR; /*!< DMA low interrupt status register, Address offset: 0x00 */ __IO uint32_t HISR; /*!< DMA high interrupt status register, Address offset: 0x04 */ __IO uint32_t LIFCR; /*!< DMA low interrupt flag clear register, Address offset: 0x08 */ __IO uint32_t HIFCR; /*!< DMA high interrupt flag clear register, Address offset: 0x0C */ } DMA_TypeDef; There are several hundreds lines of this stuff in the library. It is used to model register banks. Each register bank roughly corresponds to one of the embedded peripherals. Mike
Jul 15 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 7/15/2014 9:30 PM, Mike wrote:
 On Tuesday, 15 July 2014 at 19:48:23 UTC, Walter Bright wrote:

 The one thing peek/poke doesn't offer is transitivity. C/C++ don't offer
 volatile transitivity either. I'm not at all sure that anyone builds a data
 structure in memory-mapped registers, so I'm not convinced this is a need.
Here's an example from the STM32F4 ARM Cortex-M peripheral library: #define __IO volatile /** * brief DMA Controller */ typedef struct { __IO uint32_t CR; /*!< DMA stream x configuration register */ __IO uint32_t NDTR; /*!< DMA stream x number of data register */ __IO uint32_t PAR; /*!< DMA stream x peripheral address register */ __IO uint32_t M0AR; /*!< DMA stream x memory 0 address register */ __IO uint32_t M1AR; /*!< DMA stream x memory 1 address register */ __IO uint32_t FCR; /*!< DMA stream x FIFO control register */ } DMA_Stream_TypeDef; typedef struct { __IO uint32_t LISR; /*!< DMA low interrupt status register, Address offset: 0x00 */ __IO uint32_t HISR; /*!< DMA high interrupt status register, Address offset: 0x04 */ __IO uint32_t LIFCR; /*!< DMA low interrupt flag clear register, Address offset: 0x08 */ __IO uint32_t HIFCR; /*!< DMA high interrupt flag clear register, Address offset: 0x0C */ } DMA_TypeDef; There are several hundreds lines of this stuff in the library. It is used to model register banks. Each register bank roughly corresponds to one of the embedded peripherals.
There are no pointers to pointers there, so there is no transitivity issue. A single volatile reference to the whole struct will suffice.
Jul 15 2014
prev sibling next sibling parent "Mike" <none none.com> writes:
On Tuesday, 15 July 2014 at 16:27:53 UTC, Johannes Pfau wrote:
 You may have seen Mike's talk about D on embedded systems and 
 were
 surprised about Iain's comment that using 'shared' as a 
 replacement for
 'volatile' is actually only possible because of a bug in gdc.

 DIP62 describes how to solve this problem and make embedded 
 programming
 a first-class citizen in D:
 http://wiki.dlang.org/DIP62
Well, this was posted just as I went to sleep for the night (I'm in Asia), so I wasn't able to get my word in until the end. Sorry! DIP62 is the exemplary DIP: well researched and thoroughly specified. I was actually in favor of peek/poke intrinsics until the DIP62 and its citations took me to school. I still believe that the type qualifier is the right tool for the job, because despite having peek/poke intrinsics, users will still navigate towards making a volatile type to wrap peek/poke, as has already been proposed [1] Volatile!(T), [2] struct VolatilePointerToUint, etc... It actually justifies DIP62, IMO. After reading through this thread, the only argument against DIP62 that really made much sense was the cost/benefit tradeoff in the implementation. I can't speak to the complexity of the implementation, but I do concede that the use of volatile will only be used in the lowest layer of hardware abstraction (although it will be used quite often there), and the workarounds are not burdensome. The type qualifier would avoid such nonsense. Anyway, I can see where this is going[3], so I guess I'll have to patch my airplane[4] with a template to make it fly :( [1] http://forum.dlang.org/post/eznqslwpnxfnwlgzlsll forum.dlang.org [2] http://forum.dlang.org/post/lq4hqi$1j77$1 digitalmars.com [3] https://github.com/D-Programming-Language/dmd/pull/3773 [4] http://www.youtube.com/watch?feature=player_detailpage&v=WKRRgcEk0wg#t=65 Mike
Jul 15 2014
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 7/15/14, 9:25 AM, Johannes Pfau wrote:
 DIP62 describes how to solve this problem and make embedded programming
 a first-class citizen in D:
 http://wiki.dlang.org/DIP62
[snip] The good: This is a very crisply-written and comprehensive proposal. It could use a little work (e.g. define the typing relationships among volatile, other qualifiers, and unqualified if any) but completeness is outstanding. The bad: I think we should reject this proposal. Moreover, the comprehensiveness of the DIP gives me confidence to assess that no amount of work on the proposal could take it to acceptance. The ugly: There's none! Actually there's some nice stuff: we could use a bunch of this proposal to specify alternative approaches such as peek/poke. ============= My rationale: * I reject he premise ("The volatile qualifier is often used for embedded programming in C/C++[1]".) At most I agree with "often" as in "you'll see at least one use of volatile in some, maybe many, embedded programs" but not "volatile is present many times in a typical embedded application". Hardware/firmware providers often focus on giving access to low-level hardware facilities by using classic encapsulation and exposing APIs, as opposed to just publishing the raw addresses of various ports. Furthermore, the link given as a justification (http://www.barrgroup.com/Embedded-Systems/How-To/C-Volatile-Keyword) is not support for the statement in text, but instead a mere description of volatile, which is also technically incorrect (viz. the use of volatile with threads). * "volatile" has been one of the most awkward parts of the C and C++ specifications. If we're to learn of anything about it, it's we must stay away from it. It is extremely heavy for what it provides. * Currently C++ defines observable behavior as "reads and writes of volatile variables and sequences of calls to certain library functions". There is only a small burden associated with sequences of function calls, but the volatile parts are complicating the specification enormously. We'd be better off specifying sequences of calls to functions as defined behavior. * Adding qualifiers is heavy for the language definition and implementation. Probably they're the most impactful artifact to add. * Transitivity is only partially justified. For immutable and shared data, transitivity is motivated by code access patterns that chase pointers, but for volatile data there's no hardware reality reflected by transitivity: MMIO only pertains simple address ranges, not node-based volatile structures. The justification in section "Transitivity" is valid but puts undue emphasis on obscure interrupt service routines; even C doesn't go that far (instead it just limits the functions such routines may call). * The DIP is correct in mentioning that "volatility" is a property of the memory location so it should apply to the respective variable's type, as opposed to just letting the variable have a regular type and then unsafely relying on calls to e.g. peek and poke. But here's a practicality argument that's been already mentioned - such volatile data is rare and classic encapsulation techniques can be applied to it. * The scalability aspect should be emphasized: a large system may have more need to synchronize threads or to characterize parts of it as functional (pure + immutable). In contrast, a large embedded system does not manifest a proportionally larger presence of volatile data. ============== I think an approach based on functions peek/poke is a lot more promising. D programs must define sequences of std calls anyway, otherwise even the simplest programs that use writeln("What's your name?") followed by a readln() are incorrect. So in a way peek/poke come for "free". Clearly an approach that adds a qualifier would have superior abstraction capabilities, but my experience with C++ (both specification and use) has been that the weight to power ratio of volatile is enormous. Andrei
Jul 17 2014
next sibling parent reply "John Colvin" <john.loughran.colvin gmail.com> writes:
On Thursday, 17 July 2014 at 15:58:05 UTC, Andrei Alexandrescu 
wrote:
 I think an approach based on functions peek/poke is a lot more 
 promising. D programs must define sequences of std calls 
 anyway, otherwise even the simplest programs that use 
 writeln("What's your name?") followed by a readln() are 
 incorrect. So in a way peek/poke come for "free".
Could you expand on this "sequences of calls"? What exactly do you mean by defining it?
Jul 17 2014
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 7/17/14, 9:06 AM, John Colvin wrote:
 On Thursday, 17 July 2014 at 15:58:05 UTC, Andrei Alexandrescu wrote:
 I think an approach based on functions peek/poke is a lot more
 promising. D programs must define sequences of std calls anyway,
 otherwise even the simplest programs that use writeln("What's your
 name?") followed by a readln() are incorrect. So in a way peek/poke
 come for "free".
Could you expand on this "sequences of calls"? What exactly do you mean by defining it?
For example if you run this C program: printf("ur name: "); fflush(stdout); scanf("%s", &name); printf("Hello, %s!", name); it's guaranteed the order of calls is preserved. Andrei
Jul 17 2014
parent reply Johannes Pfau <nospam example.com> writes:
Am Thu, 17 Jul 2014 11:43:04 -0700
schrieb Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>:

 On 7/17/14, 9:06 AM, John Colvin wrote:
 On Thursday, 17 July 2014 at 15:58:05 UTC, Andrei Alexandrescu
 wrote:
 I think an approach based on functions peek/poke is a lot more
 promising. D programs must define sequences of std calls anyway,
 otherwise even the simplest programs that use writeln("What's your
 name?") followed by a readln() are incorrect. So in a way peek/poke
 come for "free".
Could you expand on this "sequences of calls"? What exactly do you mean by defining it?
For example if you run this C program: printf("ur name: "); fflush(stdout); scanf("%s", &name); printf("Hello, %s!", name); it's guaranteed the order of calls is preserved.
Function calls are actually compiler barriers in every C (like) compiler, I guess that's even in the standard. Relying on this too much can be dangerous with inlining though. Back to topic: Thanks for your response, so let's go with peek/poke / volatileLoad/Store then.
Jul 17 2014
parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Thursday, 17 July 2014 at 19:56:23 UTC, Johannes Pfau wrote:
 Am Thu, 17 Jul 2014 11:43:04 -0700
 schrieb Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>:

 On 7/17/14, 9:06 AM, John Colvin wrote:
 On Thursday, 17 July 2014 at 15:58:05 UTC, Andrei 
 Alexandrescu
 wrote:
 I think an approach based on functions peek/poke is a lot 
 more
 promising. D programs must define sequences of std calls 
 anyway,
 otherwise even the simplest programs that use 
 writeln("What's your
 name?") followed by a readln() are incorrect. So in a way 
 peek/poke
 come for "free".
Could you expand on this "sequences of calls"? What exactly do you mean by defining it?
For example if you run this C program: printf("ur name: "); fflush(stdout); scanf("%s", &name); printf("Hello, %s!", name); it's guaranteed the order of calls is preserved.
only because rearranging those cannot be proved to create the same observable semantics (in this case of course it can be actively disproved). That doesn't apply to all sequences of function calls, thankfully.
 Function calls are actually compiler barriers in every C (like)
 compiler, I guess that's even in the standard. Relying on this 
 too
 much can be dangerous with inlining though.
As I understand it (I'm not a compiler writer or a C standard expert): *Opaque* function calls are a compiler memory barrier in C and that's only by necessity (that opaque function could contain a memory barrier, amongst other problems). If the compiler can guarantee to maintain the same visible semantics of defined behaviour it can rearrange function calls just as freely as anything else.
Jul 17 2014
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 07/17/2014 05:58 PM, Andrei Alexandrescu wrote:
 * "volatile" has been one of the most awkward parts of the C and C++
 specifications. If we're to learn of anything about it, it's we must
 stay away from it. It is extremely heavy for what it provides.
 ...

 ==============

 I think an approach based on functions peek/poke is a lot more promising.
I am not saying that we should add volatile as a type qualifier, but it should be noted that programs with volatile type qualifiers can be lowered to programs using peek/poke and possibly other API calls with limited effort. (it is especially limited since the effort could be mostly shared with 'shared', both in specification and implementation, as it is another type qualifier that (will) (presumably) influence(s) the behaviour of loads and stores.)
Jul 18 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/18/2014 11:06 AM, Timon Gehr wrote:
 On 07/17/2014 05:58 PM, Andrei Alexandrescu wrote:
 * "volatile" has been one of the most awkward parts of the C and C++
 specifications. If we're to learn of anything about it, it's we must
 stay away from it. It is extremely heavy for what it provides.
 ...

 ==============

 I think an approach based on functions peek/poke is a lot more promising.
I am not saying that we should add volatile as a type qualifier, but it should be noted that programs with volatile type qualifiers can be lowered to programs using peek/poke and possibly other API calls with limited effort. (it is especially limited since the effort could be mostly shared with 'shared', both in specification and implementation, as it is another type qualifier that (will) (presumably) influence(s) the behaviour of loads and stores.)
That doesn't change the heavy effort in the semantic front end and language specification to support volatile. Take the C++ Standard and grep for 'volatile' and 'cv-qualifier'.
Jul 18 2014
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 07/18/2014 08:37 PM, Walter Bright wrote:
 On 7/18/2014 11:06 AM, Timon Gehr wrote:
 On 07/17/2014 05:58 PM, Andrei Alexandrescu wrote:
 * "volatile" has been one of the most awkward parts of the C and C++
 specifications. If we're to learn of anything about it, it's we must
 stay away from it. It is extremely heavy for what it provides.
 ...

 ==============

 I think an approach based on functions peek/poke is a lot more
 promising.
I am not saying that we should add volatile as a type qualifier, but it should be noted that programs with volatile type qualifiers can be lowered to programs using peek/poke and possibly other API calls with limited effort. (it is especially limited since the effort could be mostly shared with 'shared', both in specification and implementation, as it is another type qualifier that (will) (presumably) influence(s) the behaviour of loads and stores.)
That doesn't change the heavy effort in the semantic front end
What does this mean in detail? Is it just about the unfortunate AST representation chosen or are there more sources of heavy effort in the front end?
 and language specification to support volatile.
 ...
The language specification would just state what the rules for lowering are. In any case, I think the other points that Andrei raised are convincing enough.
Jul 18 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/18/2014 4:58 PM, Timon Gehr wrote:
 On 07/18/2014 08:37 PM, Walter Bright wrote:
 That doesn't change the heavy effort in the semantic front end
What does this mean in detail? Is it just about the unfortunate AST representation chosen or are there more sources of heavy effort in the front end?
Again, take a look at the hundreds of references to volatile all through the C++ specification for some idea.
Jul 18 2014
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 07/19/2014 02:07 AM, Walter Bright wrote:
 On 7/18/2014 4:58 PM, Timon Gehr wrote:
 On 07/18/2014 08:37 PM, Walter Bright wrote:
 That doesn't change the heavy effort in the semantic front end
What does this mean in detail? Is it just about the unfortunate AST representation chosen or are there more sources of heavy effort in the front end?
Again, take a look at the hundreds of references to volatile all through the C++ specification for some idea.
I am not suggesting to do what C++ does.
Jul 18 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 7/18/2014 6:16 PM, Timon Gehr wrote:
 On 07/19/2014 02:07 AM, Walter Bright wrote:
 On 7/18/2014 4:58 PM, Timon Gehr wrote:
 On 07/18/2014 08:37 PM, Walter Bright wrote:
 That doesn't change the heavy effort in the semantic front end
What does this mean in detail? Is it just about the unfortunate AST representation chosen or are there more sources of heavy effort in the front end?
Again, take a look at the hundreds of references to volatile all through the C++ specification for some idea.
I am not suggesting to do what C++ does.
That is what DIP62 suggests - the only difference being transitivity.
Jul 18 2014