digitalmars.D - DIP62: Volatile type qualifier for unoptimizable variables in
- Johannes Pfau (25/25) Jul 15 2014 You may have seen Mike's talk about D on embedded systems and were
- Artur Skawina via Digitalmars-d (11/15) Jul 15 2014 Compiler barriers are not "workarounds". "volatile" is not a "better
- bearophile (4/8) Jul 15 2014 What's GDC syntax?
- Johannes Pfau (3/14) Jul 15 2014 That's off-topic, but its asm("" : : : "memory");
- Artur Skawina via Digitalmars-d (7/14) Jul 15 2014 Some examples: http://forum.dlang.org/post/501A6E01.7010809@gmail.com
- Johannes Pfau (12/24) Jul 15 2014 Did you even read the section that explains why volatility is a property
- Daniel Murphy (11/14) Jul 15 2014 This is true, and it's the ideal I guess, but I'm not sure that it makes...
- Johannes Pfau (20/41) Jul 15 2014 That's a judgement call of course and the main reason why I constantly
- Kagamin (16/23) Jul 15 2014 Maybe it can be automated by a wrapper template?
- Johannes Pfau (13/40) Jul 15 2014 Then we have a templated struct. Which generates TypeInfo for every
- Kagamin (11/17) Jul 15 2014 Volatile has probably much smaller scope than shared. Shared
- Meta (4/22) Jul 15 2014 Functions can take Volatile!T, of course, and you can always have
- Johannes Pfau (12/19) Jul 15 2014 No, this doesn't work with Volatile!:
- Meta (22/33) Jul 15 2014 struct A
- Iain Buclaw via Digitalmars-d (3/31) Jul 15 2014 Worse, there's no code because the compiler optimises everything away!
- Meta (5/7) Jul 15 2014 I was assuming you'd do something in the static if (volatile)
- Artur Skawina via Digitalmars-d (17/41) Jul 15 2014 You're assuming that the raw memory needs to be exposed. It doesn't.
- Johannes Pfau (22/68) Jul 15 2014 You didn't say anything about a template in your last response. See my
- Artur Skawina via Digitalmars-d (24/33) Jul 15 2014 Not inlining in unoptimized (-O0) builds wouldn't be a problem, it could
- Johannes Pfau (11/17) Jul 16 2014 I experimented a little more with Volatile!T and I don't think you can
- Dominikus Dittes Scherkl (9/18) Jul 16 2014 How about
- Kagamin (3/13) Jul 16 2014 The actual code is a little different:
- Kagamin (3/19) Jul 16 2014 I think, the idea behind such declarations is to not let user to
- Artur Skawina via Digitalmars-d (61/79) Jul 16 2014 Thank you very much for showing actual code. There are many (wrong)
- Johannes Pfau (6/102) Jul 16 2014 Well I guess this is already decided anyway.
- Artur Skawina via Digitalmars-d (9/12) Jul 16 2014 Why would it be unusable without "strong optimization flags"? At -O0 the
- Iain Buclaw via Digitalmars-d (9/34) Jul 15 2014 More simply put. A volatile keyword would be strictly for hardware
- Walter Bright (23/36) Jul 15 2014 1. Volatile has caused a major increase in the complexity of the C++ typ...
- Johannes Pfau (49/101) Jul 15 2014 Well as described in the DIP it works just like shared from an
- bearophile (15/23) Jul 15 2014 I am reading blogs about compiler bugs, and I see that the
- Iain Buclaw via Digitalmars-d (5/23) Jul 15 2014 The use of volatile can be buggy in C because there is no other safe
- bearophile (11/14) Jul 15 2014 C11 offers means much better than volatile for multi thread
- Iain Buclaw via Digitalmars-d (2/21) Jul 15 2014 I say safe in the lightest possible terms. Volatile is in no way @safe.
- Johannes Pfau (19/45) Jul 15 2014 And how do you think peek/poke are easier to implement in this regard?
- bearophile (7/10) Jul 15 2014 Most of the design of D is not geared toward embedded computing.
- Walter Bright (3/6) Jul 15 2014 peek/poke should be implemented as compiler intrinsics that are then map...
- Walter Bright (8/15) Jul 15 2014 It isn't about whether we want to introduce a type qualifier for low lev...
- Mike (42/50) Jul 15 2014 That's most encouraging to read! D has a lot of potential for
- Walter Bright (9/23) Jul 15 2014 See my reply to Steve Sobel.
- Iain Buclaw via Digitalmars-d (7/23) Jul 15 2014 s/D/DMD/
- Walter Bright (9/13) Jul 16 2014 Yes. This is not an argument that x86 compilers shouldn't support them. ...
- Iain Buclaw via Digitalmars-d (47/54) Jul 16 2014 It certainly isn't! And it is never my intention to prevent
- Walter Bright (3/4) Jul 16 2014 The simplest, which is just to use a version declaration to select whate...
- bearophile (8/14) Jul 15 2014 Perhaps this syntax:
- Johannes Pfau (10/31) Jul 15 2014 You just reinvented this DIP ;-)
- Walter Bright (7/11) Jul 15 2014 You're right that almost nobody would use those features. But they'd hav...
- Walter Bright (24/59) Jul 15 2014 It's not so easy. All the interactions between volatile and other modifi...
- Steve Sobel (38/45) Jul 15 2014 I just wrote some embedded code for a Bluetooth LE
- Walter Bright (17/20) Jul 15 2014 There is another way. The peek() and poke() functions are primitive. You...
- bearophile (5/19) Jul 16 2014 See also the @fantom suggested by Kagamin.
- Johannes Pfau (38/71) Jul 15 2014 Yes, I actually tried tested that simple implementation with gdc. With
- Daniel Murphy (5/19) Jul 15 2014 None of those affect the type system, this does. If it could be added a...
- Johannes Pfau (3/5) Jul 16 2014 So what's necessary in this specific case?
- Daniel Murphy (10/13) Jul 16 2014 Use intrinsics with a nice template wrapper. They get inlined, the unus...
- Iain Buclaw via Digitalmars-d (8/23) Jul 16 2014 No they don't. Intrinsics make things worse.
- Walter Bright (5/8) Jul 16 2014 If the compiler back end doesn't support volatile, then it's a problem
- Iain Buclaw via Digitalmars-d (4/15) Jul 16 2014 I was thinking:
- Iain Buclaw via Digitalmars-d (5/25) Jul 16 2014 By the way, it's 'so-called' bugs like this that make we want to
- Walter Bright (2/4) Jul 16 2014 Right.
- Daniel Murphy (6/11) Jul 17 2014 ? At worst they're useless.
- Walter Bright (6/11) Jul 16 2014 I'll add that if it turns out we can't do a wrapper because of some
- Johannes Pfau (7/23) Jul 16 2014 I'll take this as you pre-approve all the mentioned extensions?
- Walter Bright (7/12) Jul 16 2014 Worth investigating. It'd probably be much more effective to work the li...
- Iain Buclaw via Digitalmars-d (7/22) Jul 17 2014 For the time being, GDC has @attribute("forceinline")
- Johannes Pfau (14/56) Jul 17 2014 I'm referring to the __initZ symbol/data, it gets emitted even if all
- Iain Buclaw via Digitalmars-d (6/45) Jul 17 2014 Just mark them as extern. More important, we need a way to build the
- David Nadlinger (8/15) Jul 17 2014 Jernej Krempuš put something together for LDC a while back:
- Kagamin (12/20) Jul 17 2014 Do you have numbers? Transitive volatility implies you planned to
- Kagamin (4/8) Jul 17 2014 BTW, can't lto already remove this unused data? LDC inlines
- Byron Heads (5/13) Jul 17 2014 A Volatile(T) in stdlib seems to make a lot of sense, I am also wonderin...
- Timon Gehr (5/10) Jul 18 2014 I've grepped the DMD source and currently 'shared' is a _bit flag_ set
- Walter Bright (7/11) Jul 18 2014 I also recommend that you grep the C++ Standard for 'volatile'.
- Walter Bright (36/89) Jul 15 2014 D is intended to have a robust enough feature set in order to implement ...
- Johannes Pfau (53/113) Jul 16 2014 And how do you implement this with a wrapper? This is common in C/C++:
- Walter Bright (25/75) Jul 16 2014 auto timerA = VolatileStruct!Timer(0xDEADBEAF);
- Kagamin (4/9) Jul 16 2014 I'd recommend @fantom name, because not only type wrappers can
- bearophile (5/8) Jul 16 2014 This sounds like something fit for D too, not just gdc. You can
- Kagamin (2/2) Jul 16 2014 Then we can have phantom functions for porting C macros instead
- Kagamin (4/5) Jul 16 2014 The optimizer should be able to eliminate this assert after
- ponce (11/14) Jul 16 2014 There is a huge confusion about volatile in the native space. No
- Sean Kelly (3/3) Jul 17 2014 We had the volatile statement as a compiler barrier in D1. Why
- Iain Buclaw via Digitalmars-d (4/7) Jul 17 2014 I don't recall volatile statements ever working properly, or at least
- Andrei Alexandrescu (2/9) Jul 17 2014 And that goes for volatile in C and C++ as well. -- Andrei
- Walter Bright (4/6) Jul 17 2014 Volatile and atomic semantics are very different, are historically confl...
- Sean Kelly (5/13) Jul 17 2014 Fair enough. My point was simply that volatile as defined for D1
- safety0ff (28/35) Jul 15 2014 You can create a function which returns a struct that wraps your
- Walter Bright (2/2) Jul 15 2014 I've been jawboning about peek/poke for years - finally decided to imple...
- Brad Roberts via Digitalmars-d (4/6) Jul 15 2014 I don't have any skin in this particular discussion, but it's worth poin...
- Walter Bright (2/5) Jul 15 2014 They're as portable as we want them to be.
- Iain Buclaw via Digitalmars-d (3/11) Jul 15 2014 I read that as, not portable at all then. :o)
- Walter Bright (2/13) Jul 16 2014 C'mon, guys. It's our spec and we can make it work.
- Iain Buclaw via Digitalmars-d (18/36) Jul 16 2014 And we are trying to make it work.
- Mike (7/10) Jul 15 2014 If this is indeed the last straw, please mark DIP20 as
- Mike (37/41) Jul 15 2014 Here's an example from the STM32F4 ARM Cortex-M peripheral
- Walter Bright (3/35) Jul 15 2014 There are no pointers to pointers there, so there is no transitivity iss...
- Mike (28/37) Jul 15 2014 Well, this was posted just as I went to sleep for the night (I'm
- Andrei Alexandrescu (66/69) Jul 17 2014 [snip]
- John Colvin (4/9) Jul 17 2014 Could you expand on this "sequences of calls"? What exactly do
- Andrei Alexandrescu (8/16) Jul 17 2014 For example if you run this C program:
- Johannes Pfau (7/27) Jul 17 2014 Function calls are actually compiler barriers in every C (like)
- John Colvin (13/45) Jul 17 2014 only because rearranging those cannot be proved to create the
- Timon Gehr (8/14) Jul 18 2014 I am not saying that we should add volatile as a type qualifier, but it
- Walter Bright (4/20) Jul 18 2014 That doesn't change the heavy effort in the semantic front end and langu...
- Timon Gehr (7/33) Jul 18 2014 What does this mean in detail? Is it just about the unfortunate AST
- Walter Bright (3/7) Jul 18 2014 Again, take a look at the hundreds of references to volatile all through...
- Timon Gehr (2/11) Jul 18 2014 I am not suggesting to do what C++ does.
- Walter Bright (2/15) Jul 18 2014 That is what DIP62 suggests - the only difference being transitivity.
You may have seen Mike's talk about D on embedded systems and were surprised about Iain's comment that using 'shared' as a replacement for 'volatile' is actually only possible because of a bug in gdc. DIP62 describes how to solve this problem and make embedded programming a first-class citizen in D: http://wiki.dlang.org/DIP62 This is the only language change necessary to make D really perfect for embedded programming, so it's important that we fix this. There are some more implementation issues, but these aren't language problems and easy to fix. The DIP provides detailed descriptions and rationales so I won't copy/paste this information into this post. But please: * Read the complete DIP before commenting. I'd took a long time to write, so it's OK if it takes some time to read ;-) Also your question might already be answered. * Although it's explained in the DIP in detail, I must mention this here again: We need _first-class_ support for embedded programming in D. Workarounds might be available (peek/poke, inline asm) but it will be obvious to most C programmers that C has got the better solution with the volatile qualifier. D enthusiasts might be happy if they can use workarounds to make D work on microcontrollers. But if we want to appeal to outsiders we can't confront them with obvious workarounds and hacks as the first thing they see in D. And if they think embedded programming is a second class citizen in D, why should they use it at all?
Jul 15 2014
On 07/15/14 18:25, Johannes Pfau via Digitalmars-d wrote:here again: We need _first-class_ support for embedded programming in D. Workarounds might be available (peek/poke, inline asm) but it will be obvious to most C programmers that C has got the better solution with the volatile qualifier. D enthusiasts might be happy if they canCompiler barriers are not "workarounds". "volatile" is not a "better solution". It's used in C only because it's defined and reasonably portable; barriers are a language extension, hence dialect specific. First class support for "embedded programming" would be defining barriers, not transplanting C's volatile. You can already express all the described volatile semantics in GDC's D dialect, in a completely portable way and without using a single asm instruction. What's missing is just a defined standard interface which can be supported by all D compilers. artur
Jul 15 2014
Artur Skawina:You can already express all the described volatile semantics in GDC's D dialect, in a completely portable way and without using a single asm instruction.What's GDC syntax? Bye, bearophile
Jul 15 2014
Am Tue, 15 Jul 2014 17:07:26 +0000 schrieb "bearophile" <bearophileHUGS lycos.com>:Artur Skawina:That's off-topic, but its asm("" : : : "memory");You can already express all the described volatile semantics in GDC's D dialect, in a completely portable way and without using a single asm instruction.What's GDC syntax? Bye, bearophile
Jul 15 2014
On 07/15/14 19:07, bearophile via Digitalmars-d wrote:Artur Skawina:Some examples: http://forum.dlang.org/post/501A6E01.7010809 gmail.com It can all be wrapped in "saner" templates and structs (for mmapped register banks etc). Johannes specifically didn't ask for mem barriers, cpu barriers or atomicity, just for compiler optimization barriers -- these don't need 'volatile'. arturYou can already express all the described volatile semantics in GDC's D dialect, in a completely portable way and without using a single asm instruction.What's GDC syntax?
Jul 15 2014
Am Tue, 15 Jul 2014 19:03:52 +0200 schrieb Artur Skawina via Digitalmars-d <digitalmars-d puremagic.com>:Compiler barriers are not "workarounds". "volatile" is not a "better solution". It's used in C only because it's defined and reasonably portable; barriers are a language extension, hence dialect specific. First class support for "embedded programming" would be defining barriers, not transplanting C's volatile. You can already express all the described volatile semantics in GDC's D dialect, in a completely portable way and without using a single asm instruction. What's missing is just a defined standard interface which can be supported by all D compilers. arturDid you even read the section that explains why volatility is a property of the memory address and not of the access (4.2.3)? What's your response to that? Why do shared variables have the privilege to prevent accessing shared memory in inappropriate ways but it's fine to place the burden of checking that all accesses to volatile memory are backed by compiler barriers to the user? How is that 'first class' support? I intentionally didn't put any technical information into this post to make sure that people actually read the DIP and the only think you responded to were the non-technical parts of my first post...
Jul 15 2014
"Johannes Pfau" wrote in message news:lq3nf4$rbp$1 digitalmars.com...Did you even read the section that explains why volatility is a property of the memory address and not of the access (4.2.3)? What's your response to that?This is true, and it's the ideal I guess, but I'm not sure that it makes it worth extending the type system for. My use of volatile in C consists entirely of #define SOME_REGISTER (*(volatile unsigned *)(BASE + OFFSET)) And this could easily be done in D with a wrapper around peek/poke functions. (assuming they're intrinsics and inlining/optimization works correctly) So, the complexity it adds to the type system and overloading etc is not worth it for my use cases. What are the other use cases where it pulls its weight?
Jul 15 2014
Am Wed, 16 Jul 2014 03:33:35 +1000 schrieb "Daniel Murphy" <yebbliesnospam gmail.com>:"Johannes Pfau" wrote in message news:lq3nf4$rbp$1 digitalmars.com...That's a judgement call of course and the main reason why I constantly say 'first class support'. This is the only language change we need for embedded systems, do we really want to compromise here? Especially if C has already got a better solution? There's some complexity in the compiler implementation, indeed. There's also some complexity in the compiler for -cov, for nogc and for shared. And we recently added complexity to support C++ namespaces. All of these things are completely useless on embedded systems. So I think we could add one slightly complex thing for embedded development. Also there's effectively no increase in language complexity for users, as nobody will use volatile with desktop applications. Volatile is already a reserved & deprecated keyword. We don't break any code. Code on embedded systems will usually be less complex with volatile and it'll be safer. For some examples where this matters: - Memory used in interrupt handlers with redirection - registers with the same layout/type could be referred by pointerDid you even read the section that explains why volatility is a property of the memory address and not of the access (4.2.3)? What's your response to that?This is true, and it's the ideal I guess, but I'm not sure that it makes it worth extending the type system for. My use of volatile in C consists entirely of #define SOME_REGISTER (*(volatile unsigned *)(BASE + OFFSET)) And this could easily be done in D with a wrapper around peek/poke functions. (assuming they're intrinsics and inlining/optimization works correctly) So, the complexity it adds to the type system and overloading etc is not worth it for my use cases. What are the other use cases where it pulls its weight?
Jul 15 2014
On Tuesday, 15 July 2014 at 17:13:09 UTC, Johannes Pfau wrote:Why do shared variables have the privilege to prevent accessing shared memory in inappropriate ways but it's fine to place the burden of checking that all accesses to volatile memory are backed by compiler barriers to the user? How is that 'first class' support?Maybe it can be automated by a wrapper template? struct Volatile(T) { T val; void opAssign(T v) { volatile_store(val, v); } } extern(C) Volatile!byte led; foreach(i;0..10) { led=0; led=1; //blink }
Jul 15 2014
Am Tue, 15 Jul 2014 17:42:47 +0000 schrieb "Kagamin" <spam here.lot>:On Tuesday, 15 July 2014 at 17:13:09 UTC, Johannes Pfau wrote:Then we have a templated struct. Which generates TypeInfo for every instance. Which might generate an Initializer for every instance and it might generate extended debug info for every instance. These are exactly the kind of workarounds I don't want. Sure we can make it work: noTypeInfo noInit noDebug (struct volatile). But then I don't see why we don't just use the proposal in this DIP. It's less complicated, has got fewer corner case, ... And that's why I say first class support: We don't have shared!(T) we have shared T. And volatile should get it's own qualifier as well. Also some things will just not work with Volatile!T, for example volatile/nonvolatile member function overloading.Why do shared variables have the privilege to prevent accessing shared memory in inappropriate ways but it's fine to place the burden of checking that all accesses to volatile memory are backed by compiler barriers to the user? How is that 'first class' support?Maybe it can be automated by a wrapper template? struct Volatile(T) { T val; void opAssign(T v) { volatile_store(val, v); } } extern(C) Volatile!byte led; foreach(i;0..10) { led=0; led=1; //blink }
Jul 15 2014
On Tuesday, 15 July 2014 at 17:50:13 UTC, Johannes Pfau wrote:And that's why I say first class support: We don't have shared!(T) we have shared T. And volatile should get it's own qualifier as well. Also some things will just not work with Volatile!T, for example volatile/nonvolatile member function overloading.Volatile has probably much smaller scope than shared. Shared memory is used more often and in more non-trivial ways, it's not unthinkable for even complex data structures to be shared or abstracted with an interface, so it makes sense that interfaces should support shared methods. Volatile is usually a small range of bytes (a communication device), which should be read once or written once. D doesn't have a way of saving on typeinfos, so volatile is not alone in this respect. I'd prefer to see it applicable to anything (or rather everything).
Jul 15 2014
On Tuesday, 15 July 2014 at 17:50:13 UTC, Johannes Pfau wrote:Then we have a templated struct. Which generates TypeInfo for every instance. Which might generate an Initializer for every instance and it might generate extended debug info for every instance. These are exactly the kind of workarounds I don't want. Sure we can make it work: noTypeInfo noInit noDebug (struct volatile). But then I don't see why we don't just use the proposal in this DIP. It's less complicated, has got fewer corner case, ... And that's why I say first class support: We don't have shared!(T) we have shared T. And volatile should get it's own qualifier as well. Also some things will just not work with Volatile!T, for example volatile/nonvolatile member function overloading.Functions can take Volatile!T, of course, and you can always have "set/volatileSet". You might even be able to use opDispatch to make it less DRY.
Jul 15 2014
Am Tue, 15 Jul 2014 18:13:26 +0000 schrieb "Meta" <jared771 gmail.com>:No, this doesn't work with Volatile!: struct A { private int _val; int read(){ return val+val }; int read() volatile { auto tmp = val; return tmp*tmp; }; } opDispatch sounds _awesome_ for systems with a few KB of SRAM. People here always have nice ideas but will you implement it and tune it until it has as little overhead as the C version?Also some things will just not work with Volatile!T, for example volatile/nonvolatile member function overloading.Functions can take Volatile!T, of course, and you can always have "set/volatileSet". You might even be able to use opDispatch to make it less DRY.
Jul 15 2014
On Tuesday, 15 July 2014 at 18:30:11 UTC, Johannes Pfau wrote:No, this doesn't work with Volatile!: struct A { private int _val; int read(){ return val+val }; int read(bool volatile)() { auto tmp = val; return tmp*tmp; }; }struct A { private int _val; int read(bool volatile)() { static if (volatile) { auto tmp = val; return tmp*tmp; } else { return val+val; } }; } And you are no worse off than before. This template can only generate 2 possible functions, and the instantiations are also cached so as far as I know, there's no code bloat.opDispatch sounds _awesome_ for systems with a few KB of SRAM. People here always have nice ideas but will you implement it and tune it until it has as little overhead as the C version?That's not something I'm currently interested in, but it's something to look into if someone wants to use it in embedded programming.
Jul 15 2014
On 15 July 2014 19:38, Meta via Digitalmars-d <digitalmars-d puremagic.com> wrote:On Tuesday, 15 July 2014 at 18:30:11 UTC, Johannes Pfau wrote:Worse, there's no code because the compiler optimises everything away!No, this doesn't work with Volatile!: struct A { private int _val; int read(){ return val+val }; int read(bool volatile)() { auto tmp = val; return tmp*tmp; }; }struct A { private int _val; int read(bool volatile)() { static if (volatile) { auto tmp = val; return tmp*tmp; } else { return val+val; } }; } And you are no worse off than before. This template can only generate 2 possible functions, and the instantiations are also cached so as far as I know, there's no code bloat.
Jul 15 2014
On Tuesday, 15 July 2014 at 19:23:36 UTC, Iain Buclaw via Digitalmars-d wrote:Worse, there's no code because the compiler optimises everything away!I was assuming you'd do something in the static if (volatile) section to stop it from doing that. I know very little when it comes to these things.
Jul 15 2014
On 07/15/14 19:11, Johannes Pfau via Digitalmars-d wrote:Am Tue, 15 Jul 2014 19:03:52 +0200 schrieb Artur Skawina via Digitalmars-d <digitalmars-d puremagic.com>:You're assuming that the raw memory needs to be exposed. It doesn't. IOW, instead of: struct HWRegs { uint a; uint b; /*...*/ } auto ptr = cast(HWReg*)0xc0000000; do: struct HWRegs { uint a; uint b; /*...*/ } auto ptr = cast(Volatile!HwRegs*)0xc0000000; and the Volatile template will magically add all the access barriers.Compiler barriers are not "workarounds". "volatile" is not a "better solution". It's used in C only because it's defined and reasonably portable; barriers are a language extension, hence dialect specific. First class support for "embedded programming" would be defining barriers, not transplanting C's volatile. You can already express all the described volatile semantics in GDC's D dialect, in a completely portable way and without using a single asm instruction. What's missing is just a defined standard interface which can be supported by all D compilers.Did you even read the section that explains why volatility is a property of the memory address and not of the access (4.2.3)? What's your response to that?Why do shared variables have the privilege to prevent accessing shared memory in inappropriate ways but it's fine to place the burden of checking that all accesses to volatile memory are backed by compiler barriers to the user? How is that 'first class' support?See above; just don't expose mmapped regs as normal memory.I intentionally didn't put any technical information into this post to make sure that people actually read the DIP and the only thing you responded to were the non-technical parts of my first post...Because the technical details don't matter when the point is that 'volatile' is not necessary. I did read your DIP, when you announced it originally and also the gdc ml discussion that followed. I have not seen any good argument for introducing a volatile type qualifier. Could you show one (concrete) example where using 'volatile' is better than my approach? artur
Jul 15 2014
Am Tue, 15 Jul 2014 19:44:51 +0200 schrieb Artur Skawina via Digitalmars-d <digitalmars-d puremagic.com>:On 07/15/14 19:11, Johannes Pfau via Digitalmars-d wrote:You didn't say anything about a template in your last response. See my reply to Kagamin: - Template bloat: TypeInfo, Initializer, DebugInfo - Requires inlining to produce decent results - Requires force inlining: You wouldn't want any method implementations of Volatile to actually end up in object files.Am Tue, 15 Jul 2014 19:03:52 +0200 schrieb Artur Skawina via Digitalmars-d <digitalmars-d puremagic.com>:You're assuming that the raw memory needs to be exposed. It doesn't. IOW, instead of: struct HWRegs { uint a; uint b; /*...*/ } auto ptr = cast(HWReg*)0xc0000000; do: struct HWRegs { uint a; uint b; /*...*/ } auto ptr = cast(Volatile!HwRegs*)0xc0000000;Compiler barriers are not "workarounds". "volatile" is not a "better solution". It's used in C only because it's defined and reasonably portable; barriers are a language extension, hence dialect specific. First class support for "embedded programming" would be defining barriers, not transplanting C's volatile. You can already express all the described volatile semantics in GDC's D dialect, in a completely portable way and without using a single asm instruction. What's missing is just a defined standard interface which can be supported by all D compilers.Did you even read the section that explains why volatility is a property of the memory address and not of the access (4.2.3)? What's your response to that?and the Volatile template will magically add all the access barriers.This is not only for mmaped regs. It's also for memory used in interrupt handlers with indirection.Why do shared variables have the privilege to prevent accessing shared memory in inappropriate ways but it's fine to place the burden of checking that all accesses to volatile memory are backed by compiler barriers to the user? How is that 'first class' support?See above; just don't expose mmapped regs as normal memory.I have not seen any good argument for introducing a volatile type qualifier. Could you show one (concrete) example where using 'volatile' is better than my approach?For one thing we should not dictate usage patterns (like don't expose raw memory) on users. Then volatile is very simple for every compiler to implement and optimize, your idea requires multiple pragmas/attributes to make sure there's no bloat and inlining always works. (For example, in gcc even a force inline attribute does not inline if -finline is not used. I don't consider that acceptable for this kind of code. So we might have to special case Volatile!T even more in the compiler, to really always inline) Otherwise Volatile!T is OK, but it's a new, fancy idea and might need lots of work, whereas a volatile qualifer could be implemented easily and we know right now for sure that it'll work 100% as expected.
Jul 15 2014
On 07/15/14 20:20, Johannes Pfau via Digitalmars-d wrote:(For example, in gcc even a force inline attribute does not inline if -finline is not used. I don't consider that acceptable for this kind of code. So we might have to special case Volatile!T even more in the compiler, to really always inline)Not inlining in unoptimized (-O0) builds wouldn't be a problem, it could even be useful sometimes (setting breakpoints for specific hw accesses etc). But is that actually the case? I practically never use -O0, so you just made me check - and indeed the GDC here *always* inlines always_inline functions, at "-O0", even when explicitly given '-fno-inline'... Anyway, the language *relies* on automatic inlining happening, this case isn't special.Otherwise Volatile!T is OK, but it's a new, fancy idea and might need lots of work, whereas a volatile qualifer could be implemented easily and we know right now for sure that it'll work 100% as expected.There's nothing new about it - it's how it has been done in gcc-ese, for many, many years; it's just combining it with templates, which aren't exactly novel either.Also there's effectively no increase in language complexity for users, as nobody will use volatile with desktop applications.Actually, there are quite a few "normal" uses. The most obvious example probably being communication with signal handlers. Then there are various kinds of inter-thread/shared-mem communication schemes. Yes, D has many problems, some of which you have mentioned in this thread. [1] But introducing additional complexity to a language just to avoid other problematic areas is not a good idea; that approach does not scale. The result are many half-finished features, every one with some kind of problem. Could a 'volatile' type qualifier be made to work, and are there technical problems with the DIP? Yes, and No. I'm not ignoring the details; I probably agree with most, if not all, of them. I'm saying the DIP is a good answer, to the wrong question. artur [1] Which I didn't respond to, to avoid derailing the thread. ;)
Jul 15 2014
Am Tue, 15 Jul 2014 19:44:51 +0200 schrieb Artur Skawina via Digitalmars-d <digitalmars-d puremagic.com>:it originally and also the gdc ml discussion that followed. I have not seen any good argument for introducing a volatile type qualifier. Could you show one (concrete) example where using 'volatile' is better than my approach? arturI experimented a little more with Volatile!T and I don't think you can make this example work: struct Timer { uint control; uint data; } enum timerA = (Volatile!Timer)* = cast()0xDEADBEAF; timerA.control |= 0b1;
Jul 16 2014
On Wednesday, 16 July 2014 at 13:04:37 UTC, Johannes Pfau wrote:I experimented a little more with Volatile!T and I don't think you can make this example work: struct Timer { uint control; uint data; } enum timerA = (Volatile!Timer)* = cast()0xDEADBEAF; timerA.control |= 0b1;How about struct VolatileTimer { Volatile!uint control; Volatile!uint data; } enum timerA = cast(VolatileTimer*)0xDEADBEAF; timerA.control |= 0b1;
Jul 16 2014
On Wednesday, 16 July 2014 at 13:04:37 UTC, Johannes Pfau wrote:I experimented a little more with Volatile!T and I don't think you can make this example work: struct Timer { uint control; uint data; } enum timerA = (Volatile!Timer)* = cast()0xDEADBEAF; timerA.control |= 0b1;The actual code is a little different: http://forum.dlang.org/thread/lq3kq8$pa3$1 digitalmars.com?page=5#post-fkgjcnqyoqltftbfmwqd:40forum.dlang.org
Jul 16 2014
On Wednesday, 16 July 2014 at 13:51:49 UTC, Kagamin wrote:On Wednesday, 16 July 2014 at 13:04:37 UTC, Johannes Pfau wrote:I think, the idea behind such declarations is to not let user to set volatile qualifiers. What if he forgets?I experimented a little more with Volatile!T and I don't think you can make this example work: struct Timer { uint control; uint data; } enum timerA = (Volatile!Timer)* = cast()0xDEADBEAF; timerA.control |= 0b1;The actual code is a little different: http://forum.dlang.org/thread/lq3kq8$pa3$1 digitalmars.com?page=5#post-fkgjcnqyoqltftbfmwqd:40forum.dlang.org
Jul 16 2014
On 07/16/14 15:02, Johannes Pfau via Digitalmars-d wrote:Am Tue, 15 Jul 2014 19:44:51 +0200 schrieb Artur Skawina via Digitalmars-d <digitalmars-d puremagic.com>:Thank you very much for showing actual code. There are many (wrong) assumptions being made in this thread (and the one on the gdc list). Claims like "A is required" or "B is impossible", where A and B are not actually defined and only vaguely described, do not usually lead to a productive dialogue. Concrete examples and code make it possible for the discussion to be about facts and not perceptions. struct Timer { uint control; uint data; } enum timerA = cast(Volatile!(Timer)*)0xDEADBEAF; //enum timerA = volatile_(cast(Timer*)0xDEADBEAF); // works too. void main() { timerA.control |= 0b1; } struct Volatile(PT) { PT raw; auto opDispatch(string M)() property { return volatile_(mixin(q{&raw.}~M)); } auto opOpAssign(string OP, B)(B b) { auto a = load(*raw); auto r = mixin("a "~OP~" b"); store(*raw, r); return this; } static: T load(T)(ref T v) { asm { "" : "+m" v; } T res = v; asm { "" : "+g" res; } return res; } T store(T)(ref T v, const T a) { asm { "" : : "m" v; } v = a; asm { "" : "+m" v; } return a; } } auto volatile_(A)(A a) { return Volatile!A(a); } // Just for IFTI. when compiled w/ "gdc -O1" results in 0000000000402949 <_Dmain>: 402949: b8 af be ad de mov $0xdeadbeaf,%eax 40294e: 8b 10 mov (%rax),%edx 402950: 83 ca 01 or $0x1,%edx 402953: 89 10 mov %edx,(%rax) 402955: b8 00 00 00 00 mov $0x0,%eax 40295a: c3 retq which is the exact op sequence you'd expect. Doing it as just one R-M-W op would be possible too, but that might not be supported by all HW/platforms. And, yes, gdc inlines it even at -O0, if inline is used. But at -O0 the generated code is /already/ huge and inefficient, hence not really usable in constrained mem environments. Obviously, this is just a proof of concept and the minimal implementation that can only handle your example. But it's enough to disprove the it-cannot-be-made-to-work-without-c-volatile claim. Are there any other examples of things that cannot be efficiently handled w/o a built-in volatile type qualifier? arturCould you show one (concrete) example where using 'volatile' is better than my approach?I experimented a little more with Volatile!T and I don't think you can make this example work: struct Timer { uint control; uint data; } enum timerA = (Volatile!Timer)* = cast()0xDEADBEAF; timerA.control |= 0b1;
Jul 16 2014
Am Wed, 16 Jul 2014 21:59:57 +0200 schrieb Artur Skawina via Digitalmars-d <digitalmars-d puremagic.com>:On 07/16/14 15:02, Johannes Pfau via Digitalmars-d wrote:Well I guess this is already decided anyway. I think it's kinda ridiculous that D embedded code will only be usable with strong optimization flags, but whatever. Maybe it works better if the usage is restricted to basic types only.Am Tue, 15 Jul 2014 19:44:51 +0200 schrieb Artur Skawina via Digitalmars-d <digitalmars-d puremagic.com>:Thank you very much for showing actual code. There are many (wrong) assumptions being made in this thread (and the one on the gdc list). Claims like "A is required" or "B is impossible", where A and B are not actually defined and only vaguely described, do not usually lead to a productive dialogue. Concrete examples and code make it possible for the discussion to be about facts and not perceptions. struct Timer { uint control; uint data; } enum timerA = cast(Volatile!(Timer)*)0xDEADBEAF; //enum timerA = volatile_(cast(Timer*)0xDEADBEAF); // works too. void main() { timerA.control |= 0b1; } struct Volatile(PT) { PT raw; auto opDispatch(string M)() property { return volatile_(mixin(q{&raw.}~M)); } auto opOpAssign(string OP, B)(B b) { auto a = load(*raw); auto r = mixin("a "~OP~" b"); store(*raw, r); return this; } static: T load(T)(ref T v) { asm { "" : "+m" v; } T res = v; asm { "" : "+g" res; } return res; } T store(T)(ref T v, const T a) { asm { "" : : "m" v; } v = a; asm { "" : "+m" v; } return a; } } auto volatile_(A)(A a) { return Volatile!A(a); } // Just for IFTI. when compiled w/ "gdc -O1" results in 0000000000402949 <_Dmain>: 402949: b8 af be ad de mov $0xdeadbeaf,%eax 40294e: 8b 10 mov (%rax),%edx 402950: 83 ca 01 or $0x1,%edx 402953: 89 10 mov %edx,(%rax) 402955: b8 00 00 00 00 mov $0x0,%eax 40295a: c3 retq which is the exact op sequence you'd expect. Doing it as just one R-M-W op would be possible too, but that might not be supported by all HW/platforms. And, yes, gdc inlines it even at -O0, if inline is used. But at -O0 the generated code is /already/ huge and inefficient, hence not really usable in constrained mem environments. Obviously, this is just a proof of concept and the minimal implementation that can only handle your example. But it's enough to disprove the it-cannot-be-made-to-work-without-c-volatile claim. Are there any other examples of things that cannot be efficiently handled w/o a built-in volatile type qualifier? arturCould you show one (concrete) example where using 'volatile' is better than my approach?I experimented a little more with Volatile!T and I don't think you can make this example work: struct Timer { uint control; uint data; } enum timerA = (Volatile!Timer)* = cast()0xDEADBEAF; timerA.control |= 0b1;
Jul 16 2014
On 07/16/14 22:09, Johannes Pfau via Digitalmars-d wrote:Well I guess this is already decided anyway.Well, peek/poke is not the right solution either...I think it's kinda ridiculous that D embedded code will only be usable with strong optimization flags, but whatever. Maybe it works betterWhy would it be unusable without "strong optimization flags"? At -O0 the generated code will be so slow and bloated that it will be unusable in a constrained embedded environment. Yes, the implicitly generated asserts are a problem - but _this is not a problem specific to volatile_. Ditto for RTTI, unwinding tables, dead code etc. Those issues needs to be dealt with anyway. artur
Jul 16 2014
On 15 July 2014 17:25, Johannes Pfau via Digitalmars-d <digitalmars-d puremagic.com> wrote:You may have seen Mike's talk about D on embedded systems and were surprised about Iain's comment that using 'shared' as a replacement for 'volatile' is actually only possible because of a bug in gdc. DIP62 describes how to solve this problem and make embedded programming a first-class citizen in D: http://wiki.dlang.org/DIP62 This is the only language change necessary to make D really perfect for embedded programming, so it's important that we fix this. There are some more implementation issues, but these aren't language problems and easy to fix. The DIP provides detailed descriptions and rationales so I won't copy/paste this information into this post. But please: * Read the complete DIP before commenting. I'd took a long time to write, so it's OK if it takes some time to read ;-) Also your question might already be answered. * Although it's explained in the DIP in detail, I must mention this here again: We need _first-class_ support for embedded programming in D. Workarounds might be available (peek/poke, inline asm) but it will be obvious to most C programmers that C has got the better solution with the volatile qualifier. D enthusiasts might be happy if they can use workarounds to make D work on microcontrollers. But if we want to appeal to outsiders we can't confront them with obvious workarounds and hacks as the first thing they see in D. And if they think embedded programming is a second class citizen in D, why should they use it at all?More simply put. A volatile keyword would be strictly for hardware access without locks. We have 'shared' for inter-thread communication, and for ensuring that there is an ordered relationship between threads when accessing the same data location at the same time. Regards Iain
Jul 15 2014
On 7/15/2014 9:25 AM, Johannes Pfau wrote:DIP62 describes how to solve this problem and make embedded programming a first-class citizen in D: http://wiki.dlang.org/DIP62This is a particularly well-written DIP, thank you and the other contributors to it.* Although it's explained in the DIP in detail, I must mention this here again: We need _first-class_ support for embedded programming in D. Workarounds might be available (peek/poke, inline asm) but it will be obvious to most C programmers that C has got the better solution with the volatile qualifier. D enthusiasts might be happy if they can use workarounds to make D work on microcontrollers. But if we want to appeal to outsiders we can't confront them with obvious workarounds and hacks as the first thing they see in D. And if they think embedded programming is a second class citizen in D, why should they use it at all?1. Volatile has caused a major increase in the complexity of the C++ type system - a complexity far out of proportion to the value it provides. It would have a similar complex and pervasive effect on the D type system. 2. You mention peek/poke here, but not in the DIP. [Background: I've designed and built embedded systems, and written the software for them.] There aren't that many memory-mapped I/O registers. They just aren't worth bending the entire language around them. Peek/poke work just fine: 1. they can be implemented as compiler intrinsics so there is no efficiency cost for using them 2. they make it clear in user code when memory-mapped I/O is accessed 3. if you really hate peek/poke calls appearing in the code, you can use UFCS to make them look like variables 4. they are simple to explain and understand, they do not introduce a numbing complexity to the type system 5. peek/poke will not add to the confusion about the difference between volatile and shared. No sane person would be tempted to use peek/poke to implement concurrency. The semantics of volatile in other languages won't confuse things for D peek/poke. The one thing peek/poke doesn't offer is transitivity. C/C++ don't offer volatile transitivity either. I'm not at all sure that anyone builds a data structure in memory-mapped registers, so I'm not convinced this is a need.
Jul 15 2014
Am Tue, 15 Jul 2014 12:48:21 -0700 schrieb Walter Bright <newshound2 digitalmars.com>:On 7/15/2014 9:25 AM, Johannes Pfau wrote:Well as described in the DIP it works just like shared from an implementation point of view, so I doesn't add much complexity in the compiler / type system.DIP62 describes how to solve this problem and make embedded programming a first-class citizen in D: http://wiki.dlang.org/DIP62This is a particularly well-written DIP, thank you and the other contributors to it.* Although it's explained in the DIP in detail, I must mention this here again: We need _first-class_ support for embedded programming in D. Workarounds might be available (peek/poke, inline asm) but it will be obvious to most C programmers that C has got the better solution with the volatile qualifier. D enthusiasts might be happy if they can use workarounds to make D work on microcontrollers. But if we want to appeal to outsiders we can't confront them with obvious workarounds and hacks as the first thing they see in D. And if they think embedded programming is a second class citizen in D, why should they use it at all?1. Volatile has caused a major increase in the complexity of the C++ type system - a complexity far out of proportion to the value it provides. It would have a similar complex and pervasive effect on the D type system.2. You mention peek/poke here, but not in the DIP. [Background: I've designed and built embedded systems, and written the software for them.] There aren't that many memory-mapped I/O registers. They just aren't worth bending the entire language around them. Peek/poke work just fine:Indeed there's nothing about peek/poke in DIP62, I thought I added that. The main point is that we can not leak pointers to volatile memory to users and pretend it's normal memory. This will lead to many problems: ------------------------------------------------------------- //In runtime: enum int* SOME_REG = 0xFFFF; //In user code peek(SOME_REG); poke(SOME_REG); *SOME_REG = 1; //Oops, forgot poke! This is not acceptable ------------------------------------------------------------- This is like saying we don't need the shared qualifier as people should use atomicOp to access the data. This is actually the crucial point about this DIP. If we can make this work without a type qualifier than that's fine as well. The only other solution to this is a wrapper as described by Artur Skawina. (Can basically wrap around peek/poke): http://dpaste.dzfl.pl/6f8ca6d7e7b8 But the problem there is that it produces quite some overhead right now. We'd need to enforce at least these points: * property value must be inlined, there should not be a 'normal' function at all * value must always be treated as in release mode. For example DMD adds an assert(this) into that function which is not acceptable * There should be no TypeInfo for Volatile!T * There should be no initializer data for Volatile!T * WE should be careful with debug info for Volatile!T * We should not emit postblits or similar stuff In the end the struct basically has to 'vanish' from the object file and ' property value' needs to be substituted into the correct places.1. they can be implemented as compiler intrinsics so there is no efficiency cost for using them 2. they make it clear in user code when memory-mapped I/O is accessedWay more dangerous and important is if you forget peek/poke.3. if you really hate peek/poke calls appearing in the code, you can use UFCS to make them look like variablesBut you cant do REGISTER.peek() |= 0b1;4. they are simple to explain and understand, they do not introduce a numbing complexity to the type systemSorry, but I don't buy any of these complexity arguments. A few months ago you suggested namespaces for D - in addition to modules - which would have lead to a complexity disaster. This just shows the priorities of the project leads: Desktop apps matter, we add nogc, c++ namespaces, shared, immutable, const, -cov, ... but for embedded systems we won't even add one qualifier.5. peek/poke will not add to the confusion about the difference between volatile and shared. No sane person would be tempted to use peek/poke to implement concurrency. The semantics of volatile in other languages won't confuse things for D peek/poke.actually peek/poke are not that different from atomicOp from a syntax perspective ;-) You'll still have to educate people about the difference.The one thing peek/poke doesn't offer is transitivity. C/C++ don't offer volatile transitivity either. I'm not at all sure that anyone builds a data structure in memory-mapped registers, so I'm not convinced this is a need.Well such data structures are sometimes used in interrupts, but not that often.
Jul 15 2014
Johannes Pfau:Well as described in the DIP it works just like shared from an implementation point of view, so I doesn't add much complexity in the compiler / type system.I am reading blogs about compiler bugs, and I see that the implementation of volatile is the buggiest part of GCC/Clang (and the Intel and Microsoft compilers). Despite numerous bug fixes, it's still a stubbornly buggy part. So it can't be as simple to implement correctly as you say. volatile fights against the optimization stages all the time. And I recognize that Walter has a significant experience on this topic, perhaps higher than yours.This just shows the priorities of the project leads: Desktop apps matter, we add nogc, c++ namespaces, shared, immutable, const, -cov, ... but for embedded systems we won't even add one qualifier.Having priorities is not something to be ashamed of. D has a GC, works only on 32+ bits, and several of its features are not meant for embedded systems. Even if you can restrict it for such restricted memory and CPU usages, D is not designed primarily for them. Bye, bearophile
Jul 15 2014
On 15 July 2014 22:07, bearophile via Digitalmars-d <digitalmars-d puremagic.com> wrote:Johannes Pfau:The use of volatile can be buggy in C because there is no other safe way to do inter-thread communication.Well as described in the DIP it works just like shared from an implementation point of view, so I doesn't add much complexity in the compiler / type system.I am reading blogs about compiler bugs, and I see that the implementation of volatile is the buggiest part of GCC/Clang (and the Intel and Microsoft compilers). Despite numerous bug fixes, it's still a stubbornly buggy part. So it can't be as simple to implement correctly as you say. volatile fights against the optimization stages all the time. And I recognize that Walter has a significant experience on this topic, perhaps higher than yours.So your saying that embedded systems don't matter? Shame on you. :o)This just shows the priorities of the project leads: Desktop apps matter, we add nogc, c++ namespaces, shared, immutable, const, -cov, ... but for embedded systems we won't even add one qualifier.Having priorities is not something to be ashamed of. D has a GC, works only on 32+ bits, and several of its features are not meant for embedded systems. Even if you can restrict it for such restricted memory and CPU usages, D is not designed primarily for them.
Jul 15 2014
Iain Buclaw:The use of volatile can be buggy in C because there is no other safe way to do inter-thread communication.C11 offers means much better than volatile for multi thread programming: http://en.wikipedia.org/wiki/C11_%28C_standard_revision%29#Changes_from_C99So your saying that embedded systems don't matter?I was explaining that D design has priorities different from the ones for embedded systems. And I'd like D to be fitter for low level coding, two hours ago I have written this: http://forum.dlang.org/post/ftvdiqvjqgmsardoyksp forum.dlang.org Bye, bearophile
Jul 15 2014
On 15 July 2014 22:17, Iain Buclaw <ibuclaw gdcproject.org> wrote:On 15 July 2014 22:07, bearophile via Digitalmars-d <digitalmars-d puremagic.com> wrote:I say safe in the lightest possible terms. Volatile is in no way safe.Johannes Pfau:The use of volatile can be buggy in C because there is no other safe way to do inter-thread communication.Well as described in the DIP it works just like shared from an implementation point of view, so I doesn't add much complexity in the compiler / type system.I am reading blogs about compiler bugs, and I see that the implementation of volatile is the buggiest part of GCC/Clang (and the Intel and Microsoft compilers). Despite numerous bug fixes, it's still a stubbornly buggy part. So it can't be as simple to implement correctly as you say. volatile fights against the optimization stages all the time. And I recognize that Walter has a significant experience on this topic, perhaps higher than yours.
Jul 15 2014
Am Tue, 15 Jul 2014 21:07:21 +0000 schrieb "bearophile" <bearophileHUGS lycos.com>:Johannes Pfau:And how do you think peek/poke are easier to implement in this regard? You still have to be very careful with inlining, instruction scheduling, etc. The main benefit of a volatile type qualifier is that it's similar to the C volatile qualifier, so all these bugs have already been fixed. GDC/LDC could probably just map to the volatile C qualifier. If we invent some homebrew solution we can start at zero with bug fixing. Also what does buggy mean if C doesn't have a clear specification about volatile?Well as described in the DIP it works just like shared from an implementation point of view, so I doesn't add much complexity in the compiler / type system.I am reading blogs about compiler bugs, and I see that the implementation of volatile is the buggiest part of GCC/Clang (and the Intel and Microsoft compilers). Despite numerous bug fixes, it's still a stubbornly buggy part. So it can't be as simple to implement correctly as you say. volatile fights against the optimization stages all the time. And I recognize that Walter has a significant experience on this topic, perhaps higher than yours.Well as long as those priorities are clearly communicated. If you tell me 'we don't care about embedded programming' then I'll shut up and move back to C. But 'D is a systems programming language for low level tasks' and 'we don't want to introduce a type qualifier for low level programming, but nogc is just fine' don't go together. This leaves all contributors and devs hoping to see D on embedded systems in uncertainty.This just shows the priorities of the project leads: Desktop apps matter, we add nogc, c++ namespaces, shared, immutable, const, -cov, ... but for embedded systems we won't even add one qualifier.Having priorities is not something to be ashamed of. D has a GC, works only on 32+ bits, and several of its features are not meant for embedded systems. Even if you can restrict it for such restricted memory and CPU usages, D is not designed primarily for them.
Jul 15 2014
Johannes Pfau:And how do you think peek/poke are easier to implement in this regard?Let's ask Walter.Well as long as those priorities are clearly communicated.Most of the design of D is not geared toward embedded computing. It could be adapted and improved for those purposes, but surely they aren't its main target. Bye, bearophile
Jul 15 2014
On 7/15/2014 2:28 PM, bearophile wrote:Johannes Pfau:peek/poke should be implemented as compiler intrinsics that are then mapped onto whatever the backend does best for 'volatile' accesses.And how do you think peek/poke are easier to implement in this regard?Let's ask Walter.
Jul 15 2014
On 7/15/2014 2:17 PM, Johannes Pfau wrote:Well as long as those priorities are clearly communicated. If you tell me 'we don't care about embedded programming' then I'll shut up and move back to C.Rest assured we definitely do care about embedded systems programming.But 'D is a systems programming language for low level tasks' and 'we don't want to introduce a type qualifier for low level programming, but nogc is just fine' don't go together. This leaves all contributors and devs hoping to see D on embedded systems in uncertainty.It isn't about whether we want to introduce a type qualifier for low level programming or not. It is about what is the best solution for MMIO for low level programming. Note that D already supports inp() and outp() as compiler intrinsics, adding peek() and poke() will complement them nicely:
Jul 15 2014
On Tuesday, 15 July 2014 at 21:57:56 UTC, Walter Bright wrote:Rest assured we definitely do care about embedded systems programming.That's most encouraging to read! D has a lot of potential for embedded programming, we just need to be thrown a few bones.It isn't about whether we want to introduce a type qualifier for low level programming or not. It is about what is the best solution for MMIO for low level programming.Exactly, and I think a type qualifier is the right tool for the job. I was all in favor of peek/poke after our brief conversation at DConf, but DIP62 sold me on a type qualifier... see below.Note that D already supports inp() and outp() as compiler intrinsics, adding peek() and poke() will complement them nicely:OT: Isn't inp/outp and Intel-only thing, though? DIP62 addressed why peek/poke are not the right tool for the job in the "why a type qualifier" section. In summary, one would never want to access volatile memory with non-volatile semantics, and a type qualifier is the only proposal I've seen that enforces that: peek/poke intrinsics do not. The Volatile!(T) workaround is proof that peek/poke is not the right tool. The Volatile!(T) workaround is a jerry-rig on top of peek/poke to provide what a 'volatile' type qualifier would provide, and is necessary because the language lacks the proper tool. From what I've read in this thread, I suspect you favor peek/poke due to the complexity it introduces into the implementation. If a 'volatile' type qualifier was a trivial implementation, and did not introduce complexity into implementation, would your position be different? I and the others in favor of DIP62 will likely concede the following: * Workarounds exist * The workarounds are not overly burdensome * 'volatile' as defined by DIP62 would not be a pervasive feature of the language with high leverage. It is only used in the low level hardware abstractions (i.e. systems programming) ...but that's beside the point. I remember watching a talk you gave where you compared code with an airplane, and how when you get it right, it just looks like it wants to fly. I don't know how many people got that, but I sure did. peek/poke delegates responsibility to the programmer, as at every usage s/he must make sure they employ it properly to beat their code into flying. Volatile!(T) is an inelegant lever to make peek/poke more less prone to disobedience. A type qualifier is an elegant tool for modeling one's hardware because at every usage, it already knows how, and wants to fly. Mike
Jul 15 2014
On 7/15/2014 6:03 PM, Mike wrote:OT: Isn't inp/outp and Intel-only thing, though?Of course. But it's still used on embedded systems.DIP62 addressed why peek/poke are not the right tool for the job in the "why a type qualifier" section. In summary, one would never want to access volatile memory with non-volatile semantics, and a type qualifier is the only proposal I've seen that enforces that: peek/poke intrinsics do not.See my reply to Steve Sobel.From what I've read in this thread, I suspect you favor peek/poke due to the complexity it introduces into the implementation. If a 'volatile' type qualifier was a trivial implementation, and did not introduce complexity into implementation, would your position be different?Unfortunately, it isn't trivial, so your point is moot. Even if it were, "trivial" is not a good reason to put a feature in a language. The language must not become a kitchen sink grab-bag of trivia. Go has demonstrated there is considerable value in having a small set of orthogonal features instead.I remember watching a talk you gave where you compared code with an airplane, and how when you get it right, it just looks like it wants to fly. I don't know how many people got that, but I sure did.You're the only one that likes my airplane metaphors :-)A type qualifier is an elegant tool for modeling one's hardware because at every usage, it already knows how, and wants to fly.Type qualifiers are not elegant. Type qualifiers are fairly burdensome to the user.
Jul 15 2014
On 15 July 2014 22:57, Walter Bright via Digitalmars-d <digitalmars-d puremagic.com> wrote:On 7/15/2014 2:17 PM, Johannes Pfau wrote:s/D/DMD/ GDC doesn't support it as compiler intrinsics. Also, isn't inp/outp all x86-specific? I've never come across a similar thing in other architectures (at least, of the same name)Well as long as those priorities are clearly communicated. If you tell me 'we don't care about embedded programming' then I'll shut up and move back to C.Rest assured we definitely do care about embedded systems programming.But 'D is a systems programming language for low level tasks' and 'we don't want to introduce a type qualifier for low level programming, but nogc is just fine' don't go together. This leaves all contributors and devs hoping to see D on embedded systems in uncertainty.It isn't about whether we want to introduce a type qualifier for low level programming or not. It is about what is the best solution for MMIO for low level programming. Note that D already supports inp() and outp() as compiler intrinsics, adding peek() and poke() will complement them nicely:
Jul 15 2014
On 7/15/2014 11:23 PM, Iain Buclaw via Digitalmars-d wrote:GDC doesn't support it as compiler intrinsics.I know.Also, isn't inp/outp all x86-specific?Yes. This is not an argument that x86 compilers shouldn't support them. After all, D is supposed to be a systems programming language.I've never come across a similar thing in other architectures (at least, of the same name)I have. Special I/O ports and associated special instructions used to be commonplace in microprocessors. Remember, D is supposed to be a systems programming language, not a language restricted to the least common denominator of every CPU now and in the future. One needs access to the system parts of the CPU.
Jul 16 2014
On 16 July 2014 08:16, Walter Bright via Digitalmars-d <digitalmars-d puremagic.com> wrote:On 7/15/2014 11:23 PM, Iain Buclaw via Digitalmars-d wrote:It certainly isn't! And it is never my intention to prevent optimisations on one platform if another doesn't benefit. I think my argument comes down to this, consider the following compiler intrisinsic. --- ubyte inp(uint port_address); It should instead be written as. --- ubyte inp()(uint port_address) { version(ARM) { asm { /* ... */ } } else static assert(false, "Not Implemented"); } What's going on here? 1) If the compiler recognises the function as an intrinsic, it discards the body and does whatever it knows to be correct. 2) If its not an intrinsic, it falls back to the library implementation. 3) So not to hurt targets that do not support inp/outp, it only errors at compile-time if *actually* used. As an alternative, instead of a template, we could use core.internal to do the same thing in this way. core/internal/bitops.d --- module core.bitops; version(ARM) { ubyte inp(uint port_address) { /* ... */ } } core/bitops.d --- module core.bitops; ubyte inp(uint port_address); But then it falls down to a linker error if 1) The compiler didn't expand inp() in the backend 2) core.internal.bitops doesn't implement inp() Which would you prefer? Regards Iain.GDC doesn't support it as compiler intrinsics.I know.Also, isn't inp/outp all x86-specific?Yes. This is not an argument that x86 compilers shouldn't support them. After all, D is supposed to be a systems programming language.
Jul 16 2014
On 7/16/2014 1:35 AM, Iain Buclaw via Digitalmars-d wrote:Which would you prefer?The simplest, which is just to use a version declaration to select whatever works for any particular compiler/CPU.
Jul 16 2014
Johannes Pfau://In runtime: enum int* SOME_REG = 0xFFFF; //In user code peek(SOME_REG); poke(SOME_REG); *SOME_REG = 1; //Oops, forgot poke! This is not acceptablePerhaps this syntax: volatile enum int* SOME_REG = 0xFFFF; Could turn this in a syntax error (only peek/poke are allowed to write and read from this address): *SOME_REG = 1; Bye, bearophile
Jul 15 2014
Am Tue, 15 Jul 2014 21:16:20 +0000 schrieb "bearophile" <bearophileHUGS lycos.com>:Johannes Pfau:You just reinvented this DIP ;-) Once you add a volatile type qualifier, there's no need to use peek/poke. The compiler can then easily detect reads/writes to these variables and do the right thing. The DIP may seem complicated cause it considers overloading, transitivity and similar stuff. In practice almost nobody will use these features. But if I had not specified them Walters first question would have been "How does it interact with overloading" ;-)//In runtime: enum int* SOME_REG = 0xFFFF; //In user code peek(SOME_REG); poke(SOME_REG); *SOME_REG = 1; //Oops, forgot poke! This is not acceptablePerhaps this syntax: volatile enum int* SOME_REG = 0xFFFF; Could turn this in a syntax error (only peek/poke are allowed to write and read from this address): *SOME_REG = 1; Bye, bearophile
Jul 15 2014
On 7/15/2014 2:22 PM, Johannes Pfau wrote:The DIP may seem complicated cause it considers overloading, transitivity and similar stuff. In practice almost nobody will use these features. But if I had not specified them Walters first question would have been "How does it interact with overloading" ;-)You're right that almost nobody would use those features. But they'd have to be defined, documented, implemented and tested, like they are in C++. It negatively impacts users in that D becomes a larger, more complex language with next to no corresponding benefit. (Just grep for 'shared' in the D source code and see how pervasive it is.) This is why I propose that peek/poke have a much better cost/benefit ratio.
Jul 15 2014
On 7/15/2014 1:43 PM, Johannes Pfau wrote:It's not so easy. All the interactions between volatile and other modifiers have to be carefully designed, documented, and implemented.1. Volatile has caused a major increase in the complexity of the C++ type system - a complexity far out of proportion to the value it provides. It would have a similar complex and pervasive effect on the D type system.Well as described in the DIP it works just like shared from an implementation point of view, so I doesn't add much complexity in the compiler / type system.Indeed there's nothing about peek/poke in DIP62, I thought I added that. The main point is that we can not leak pointers to volatile memory to users and pretend it's normal memory. This will lead to many problems:I have a hard time seeing that MMIO is used so much in an embedded system that this will be a significant problem. And if it is, you can build a wrapper type and control access.This is like saying we don't need the shared qualifier as people should use atomicOp to access the data.Shared use, however, is pervasive in concurrent code.But the problem there is that it produces quite some overhead right now.Are you sure? The compiler is pretty good at inlining trivial functions.As you mentioned in the DIP, read-modify-write operations should be split up, as their semantics are murky.3. if you really hate peek/poke calls appearing in the code, you can use UFCS to make them look like variablesBut you cant do REGISTER.peek() |= 0b1;No need to be sorry. Your opinion is not invalid.4. they are simple to explain and understand, they do not introduce a numbing complexity to the type systemSorry, but I don't buy any of these complexity arguments.A few months ago you suggested namespaces for D - in addition to modules - which would have lead to a complexity disaster.C++ namespaces are now implemented in D, and will they lead to complexity disaster? Certainly not in the implementation. For users? I don't think so, but we'll see.This just shows the priorities of the project leads: Desktop apps matter, we add nogc, c++ namespaces, shared, immutable, const, -cov, ... but for embedded systems we won't even add one qualifier.I don't think this is the right argument. It isn't about embedded systems should be acknowledged with a qualifier. I see it as about how often does MMIO logic appear in an actual embedded program? In the embedded systems I've written, it only appears in a handful of places. It is not pervasive. BTW, immutable data in D is designed to be placed in ROM for embedded systems. So there's already a qualifier :-)That's correct.5. peek/poke will not add to the confusion about the difference between volatile and shared. No sane person would be tempted to use peek/poke to implement concurrency. The semantics of volatile in other languages won't confuse things for D peek/poke.actually peek/poke are not that different from atomicOp from a syntax perspective ;-)You'll still have to educate people about the difference.Peek/poke has a 40 year history of being used for MMIO, and I've never heard of it being used for concurrency. I don't think there's endemic confusion, quite unlike volatile.
Jul 15 2014
On Tuesday, 15 July 2014 at 21:28:24 UTC, Walter Bright wrote:I don't think this is the right argument. It isn't about embedded systems should be acknowledged with a qualifier. I see it as about how often does MMIO logic appear in an actual embedded program? In the embedded systems I've written, it only appears in a handful of places. It is not pervasive. BTW, immutable data in D is designed to be placed in ROM for embedded systems. So there's already a qualifier :-)I just wrote some embedded code for a Bluetooth LE microcontroller. The MCU in question has no less than 25 separate peripheral systems, each with its own set of registers, which use MMIO for configuration and operation. Just one of the header files provided by the chip vendor for this SoC defines 524 volatile variables. I'm sure that this complexity is barely anything compared to, say, the SoCs found in smart phones. Of course, one confines MMIO accesses to one layer of the program. Even so, that layer can be very broad given the sheer amount of stuff it has to touch - potentially many thousands of lines of code. With the code leverage that D has, I would not be surprised if this made up 10% or more of an embedded program's size. Also, there are many ways to misconfigure these peripherals and produce an effect physically similar to a compiler-eliminated access. A single omission of peek() or poke() could take hours to track down if one is unfortunate. If one is having trouble, for example, with the bluetooth radio, it could be due to: - An illegal configuration amongst the many registers touching BTLE - A legal configuration which is not accepted by the other device (e.g. causes a timeout) - The chip is damaged - The other device is damaged - The other device has a bug - The compiler eliminated access to the BTLE control registers Finally, the compiler could have eliminated access to other registers that had a side-effect on the BTLE code (for example, eliminating configuration of the memory protection unit). These would probably be incredibly pernicious because the true error is not in the code that has broken, and could feasibly be in any of a large number of systems. I certainly would not have a lot of fun scanning thousands of lines of code for a mistaken omission of peek() or poke(). So in my opinion, and without prescribing the exact solution, volatility is an important enough concept in embedded to merit a standard solution that can be mechanically verified.
Jul 15 2014
On 7/15/2014 4:52 PM, Steve Sobel wrote:So in my opinion, and without prescribing the exact solution, volatility is an important enough concept in embedded to merit a standard solution that can be mechanically verified.There is another way. The peek() and poke() functions are primitive. You can wrap a "pointer to volatile" in its own type, and in that type, control access to the pointer so it cannot be used outside of peek/poke. For example, off the top of my head: struct VolatilePointerToUint { private: size_t ptr; public: this(size_t ptr) { this.ptr = ptr; } uint read() { return peek(cast(uint*)ptr); } void write(uint value) { poke(cast(uint*)ptr, value); } } You'd probably wish to flesh this out a bit more, but it's the general idea. It's a zero cost abstraction. D has very capable abilities to create types that are restricted versions of other types - this should be explored and exhausted before considering language extensions.
Jul 15 2014
Walter Bright:struct VolatilePointerToUint { private: size_t ptr; public: this(size_t ptr) { this.ptr = ptr; } uint read() { return peek(cast(uint*)ptr); } void write(uint value) { poke(cast(uint*)ptr, value); } } You'd probably wish to flesh this out a bit more, but it's the general idea.To be added to Phobos?It's a zero cost abstraction. D has very capable abilities to create types that are restricted versions of other types - this should be explored and exhausted before considering language extensions.See also the fantom suggested by Kagamin. Bye, bearophile
Jul 16 2014
Am Tue, 15 Jul 2014 14:28:23 -0700 schrieb Walter Bright <newshound2 digitalmars.com>:On 7/15/2014 1:43 PM, Johannes Pfau wrote:Yes, I actually tried tested that simple implementation with gdc. With aggressive optimizations the code inlines just fine and it'll generate the same asm as an access to a volatile variable in C. But it's not space-effecient: Even if it inlines, the function is still in the object file as a distinct function. There's TypeInfo and there's the assert(this), there's an initializer symbol and I guess there could be copy constructors as well. I expect many bug reports/fixes till we get a Volatile!T wrapper to be really space-efficient. I think if we decide that Volatile!T is the solution I'd probably hack an attribute TypeWrapper for gdc which simply recognized Volatile!T and does the right thing. The 'standard D' option requires forceinline noTypeinfo noInitializer noDebug and lots of boilerplate (all opAssign methods have to be implemented as we don't rewrite RMW operations into property get/set which also affects alias this to a property).But the problem there is that it produces quite some overhead right now.Are you sure? The compiler is pretty good at inlining trivial functions.But not necessarily in user code. REGISTER |= is quite common in C and it's perfectly safe. If people will have to split this up into peek |= poke that's a step backwards from C.As you mentioned in the DIP, read-modify-write operations should be split up, as their semantics are murky.3. if you really hate peek/poke calls appearing in the code, you can use UFCS to make them look like variablesBut you cant do REGISTER.peek() |= 0b1;You were initially in favor of an option which would have allowed to use namespaces in D code as well (namespace A {}). The current implementation is mostly limited to C++ so that's better, but my point was that nobody cared about complexity back then.A few months ago you suggested namespaces for D - in addition to modules - which would have lead to a complexity disaster.C++ namespaces are now implemented in D, and will they lead to complexity disaster? Certainly not in the implementation. For users? I don't think so, but we'll see.But it's not only about frequency, it's also about convenience and safety. How often is opDispatch actually used? The synchronized statement? Nevertheless they are there, because someone thought these use cases are important enough. The perceived importance of a special qualifier for MMIO code is highly subjective, but I think that most embedded programmers would say it's necessary. In Andrei's Leverage talk he said (5:57): "Consider this: Convenience, modeling power, efficiency: [...] You gonna have to have a large language. This is a fact of life. [...]". We have weak pure/ strong pure & immutable for functional programming, A complete set of OO primitives, builtin unit tests, code coverage, ... I think it's a correct argument to say that you think embedded system programming is not important enough for the additional complexity introduced by a new qualifier.This just shows the priorities of the project leads: Desktop apps matter, we add nogc, c++ namespaces, shared, immutable, const, -cov, ... but for embedded systems we won't even add one qualifier.I don't think this is the right argument. It isn't about embedded systems should be acknowledged with a qualifier. I see it as about how often does MMIO logic appear in an actual embedded program? In the embedded systems I've written, it only appears in a handful of places. It is not pervasive.
Jul 15 2014
"Johannes Pfau" wrote in message news:lq51u4$21qb$1 digitalmars.com...But it's not only about frequency, it's also about convenience and safety. How often is opDispatch actually used? The synchronized statement? Nevertheless they are there, because someone thought these use cases are important enough.None of those affect the type system, this does. If it could be added as a storage class or even a new statement (!) it would be less intrusive.The perceived importance of a special qualifier for MMIO code is highly subjective, but I think that most embedded programmers would say it's necessary. In Andrei's Leverage talk he said (5:57): "Consider this: Convenience, modeling power, efficiency: [...] You gonna have to have a large language. This is a fact of life. [...]". We have weak pure/ strong pure & immutable for functional programming, A complete set of OO primitives, builtin unit tests, code coverage, ... I think it's a correct argument to say that you think embedded system programming is not important enough for the additional complexity introduced by a new qualifier.The problem is not 'this feature is too complex', the problems is that it's more complex than necessary.
Jul 15 2014
Am Wed, 16 Jul 2014 15:59:46 +1000 schrieb "Daniel Murphy" <yebbliesnospam gmail.com>:The problem is not 'this feature is too complex', the problems is that it's more complex than necessary.So what's necessary in this specific case?
Jul 16 2014
"Johannes Pfau" wrote in message news:lq5pv1$2nfb$1 digitalmars.com...Use intrinsics with a nice template wrapper. They get inlined, the unused function bodies get stripped, and the debug info/typeinfo is just as big a problem as it is for every other function in your program. Intrinsics get us away from our current 'there is no portable way to do this' situation. They are _trivial_ to implement for any of the compilers. I have complete faith that it is possible to make a performant wrapper for both the single-register read/write and the struct pattern you posted. And if I'm wrong - we can add a volatile type modifier later! I just don't see volatile pulling it's weight.The problem is not 'this feature is too complex', the problems is that it's more complex than necessary.So what's necessary in this specific case?
Jul 16 2014
On 16 July 2014 15:14, Daniel Murphy via Digitalmars-d <digitalmars-d puremagic.com> wrote:"Johannes Pfau" wrote in message news:lq5pv1$2nfb$1 digitalmars.com...No they don't. Intrinsics make things worse.Use intrinsics with a nice template wrapper. They get inlined, the unused function bodies get stripped, and the debug info/typeinfo is just as big a problem as it is for every other function in your program. Intrinsics get us away from our current 'there is no portable way to do this' situation.The problem is not 'this feature is too complex', the problems is that it's more complex than necessary.So what's necessary in this specific case?They are _trivial_ to implement for any of the compilers.No they aren't, unless you are talking specifically about volatile., in which case, that depends...I have complete faith that it is possible to make a performant wrapper for both the single-register read/write and the struct pattern you posted. And if I'm wrong - we can add a volatile type modifier later! I just don't see volatile pulling it's weight.Hardware access is supposed to be correct, not performant. Regards Iain
Jul 16 2014
On 7/16/2014 7:31 AM, Iain Buclaw via Digitalmars-d wrote:If the compiler back end doesn't support volatile, then it's a problem regardless of how the front end handles it, so the point is moot. In any case, the compiler front end can always implement an intrinsic by just calling a function.They are _trivial_ to implement for any of the compilers.No they aren't, unless you are talking specifically about volatile., in which case, that depends...
Jul 16 2014
On 16 July 2014 19:58, Walter Bright via Digitalmars-d <digitalmars-d puremagic.com> wrote:On 7/16/2014 7:31 AM, Iain Buclaw via Digitalmars-d wrote:I was thinking: http://bugzilla.gdcproject.org/show_bug.cgi?id=84If the compiler back end doesn't support volatile, then it's a problem regardless of how the front end handles it, so the point is moot. In any case, the compiler front end can always implement an intrinsic by just calling a function.They are _trivial_ to implement for any of the compilers.No they aren't, unless you are talking specifically about volatile., in which case, that depends...
Jul 16 2014
On 16 July 2014 20:03, Iain Buclaw <ibuclaw gdcproject.org> wrote:On 16 July 2014 19:58, Walter Bright via Digitalmars-d <digitalmars-d puremagic.com> wrote:By the way, it's 'so-called' bugs like this that make we want to remove any notion of volatile from shared. Regards IainOn 7/16/2014 7:31 AM, Iain Buclaw via Digitalmars-d wrote:I was thinking: http://bugzilla.gdcproject.org/show_bug.cgi?id=84If the compiler back end doesn't support volatile, then it's a problem regardless of how the front end handles it, so the point is moot. In any case, the compiler front end can always implement an intrinsic by just calling a function.They are _trivial_ to implement for any of the compilers.No they aren't, unless you are talking specifically about volatile., in which case, that depends...
Jul 16 2014
On 7/16/2014 12:10 PM, Iain Buclaw via Digitalmars-d wrote:By the way, it's 'so-called' bugs like this that make we want to remove any notion of volatile from shared.Right.
Jul 16 2014
"Iain Buclaw via Digitalmars-d" wrote in message news:mailman.4214.1405521071.2907.digitalmars-d puremagic.com...No they don't. Intrinsics make things worse.? At worst they're useless.Yes I'm talking about the peek/poke intrinsics. As all D backends are C backends, and C supports these semantics already...They are _trivial_ to implement for any of the compilers.No they aren't, unless you are talking specifically about volatile., in which case, that depends...Hardware access is supposed to be correct, not performant.Ideally both!
Jul 17 2014
On 7/16/2014 7:14 AM, Daniel Murphy wrote:Intrinsics get us away from our current 'there is no portable way to do this' situation. They are _trivial_ to implement for any of the compilers. I have complete faith that it is possible to make a performant wrapper for both the single-register read/write and the struct pattern you posted. And if I'm wrong - we can add a volatile type modifier later!I'll add that if it turns out we can't do a wrapper because of some limitations/bugs in D's expressiveness, that is the problem with D we need to fix, not adding another type modifier. It all comes down to leverage. Making user defined types more capable has far, far more productive leverage than welding specifics into the compiler.
Jul 16 2014
Am Wed, 16 Jul 2014 11:49:21 -0700 schrieb Walter Bright <newshound2 digitalmars.com>:On 7/16/2014 7:14 AM, Daniel Murphy wrote:I'll take this as you pre-approve all the mentioned extensions? * way to disable typeinfo for struct * way to disable initializer * force inlining * (way to omit copy constructor function / force inline)Intrinsics get us away from our current 'there is no portable way to do this' situation. They are _trivial_ to implement for any of the compilers. I have complete faith that it is possible to make a performant wrapper for both the single-register read/write and the struct pattern you posted. And if I'm wrong - we can add a volatile type modifier later!I'll add that if it turns out we can't do a wrapper because of some limitations/bugs in D's expressiveness, that is the problem with D we need to fix, not adding another type modifier. It all comes down to leverage. Making user defined types more capable has far, far more productive leverage than welding specifics into the compiler.
Jul 16 2014
On 7/16/2014 1:12 PM, Johannes Pfau wrote:I'll take this as you pre-approve all the mentioned extensions? * way to disable typeinfo for structWorth investigating. It'd probably be much more effective to work the linker angle of removing items for which no references exist. Martin was working on that, but his latest attempt had to be reverted, as it needed a new approach.* way to disable initializerT t = void; // no initialization* force inlininghttp://wiki.dlang.org/DIP56 (Note the author!)* (way to omit copy constructor function / force inline)I don't know what that means
Jul 16 2014
On 17 July 2014 07:09, Walter Bright via Digitalmars-d <digitalmars-d puremagic.com> wrote:On 7/16/2014 1:12 PM, Johannes Pfau wrote:For the time being, GDC has attribute("forceinline") https://github.com/D-Programming-GDC/GDC/blob/master/libphobos/libdruntime/gcc/attribute.dI'll take this as you pre-approve all the mentioned extensions? * way to disable typeinfo for structWorth investigating. It'd probably be much more effective to work the linker angle of removing items for which no references exist. Martin was working on that, but his latest attempt had to be reverted, as it needed a new approach.* way to disable initializerT t = void; // no initialization* force inlininghttp://wiki.dlang.org/DIP56 (Note the author!)disable this(this); ? I'll let Johannes clarify. Iain* (way to omit copy constructor function / force inline)I don't know what that means
Jul 17 2014
Am Thu, 17 Jul 2014 08:05:33 +0100 schrieb Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com>:On 17 July 2014 07:09, Walter Bright via Digitalmars-d <digitalmars-d puremagic.com> wrote:I'm referring to the __initZ symbol/data, it gets emitted even if all fields of a struct are =void initialized.On 7/16/2014 1:12 PM, Johannes Pfau wrote:I'll take this as you pre-approve all the mentioned extensions? * way to disable typeinfo for structWorth investigating. It'd probably be much more effective to work the linker angle of removing items for which no references exist. Martin was working on that, but his latest attempt had to be reverted, as it needed a new approach.* way to disable initializerT t = void; // no initializationYes. We'll have to take a look if we can prevent the function from appearing in the object file (with force inline, there's no need to keep the function, it must always be inlined anyway)For the time being, GDC has attribute("forceinline") https://github.com/D-Programming-GDC/GDC/blob/master/libphobos/libdruntime/gcc/attribute.d* force inlininghttp://wiki.dlang.org/DIP56 (Note the author!)I was referring to the implicitly generated __cpctor not the postblit. As it's implicitly generated you can't put force-inline on it. This applies to all compiler generated functions though, __fieldPostBlit, __fieldDtor autogenerated opAssign and probably some more. However, simple (POD?) types are probably not affected as those don't get a __cpctor IIRC?disable this(this); ? I'll let Johannes clarify.* (way to omit copy constructor function / force inline)I don't know what that means
Jul 17 2014
On 17 July 2014 21:01, Johannes Pfau via Digitalmars-d <digitalmars-d puremagic.com> wrote:Am Thu, 17 Jul 2014 08:05:33 +0100 schrieb Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com>:Just mark them as extern. More important, we need a way to build the bodies of functions in other modules for cross-module inlining (needed for forceinline to work correctly without getting 'function body is not available' errors).On 17 July 2014 07:09, Walter Bright via Digitalmars-d <digitalmars-d puremagic.com> wrote:I'm referring to the __initZ symbol/data, it gets emitted even if all fields of a struct are =void initialized.On 7/16/2014 1:12 PM, Johannes Pfau wrote:I'll take this as you pre-approve all the mentioned extensions? * way to disable typeinfo for structWorth investigating. It'd probably be much more effective to work the linker angle of removing items for which no references exist. Martin was working on that, but his latest attempt had to be reverted, as it needed a new approach.* way to disable initializerT t = void; // no initializationYes. We'll have to take a look if we can prevent the function from appearing in the object file (with force inline, there's no need to keep the function, it must always be inlined anyway)For the time being, GDC has attribute("forceinline") https://github.com/D-Programming-GDC/GDC/blob/master/libphobos/libdruntime/gcc/attribute.d* force inlininghttp://wiki.dlang.org/DIP56 (Note the author!)
Jul 17 2014
On Thursday, 17 July 2014 at 20:42:16 UTC, Iain Buclaw via Digitalmars-d wrote:Just mark them as extern. More important, we need a way to build the bodies of functions in other modules for cross-module inlining (needed for forceinline to work correctly without getting 'function body is not available' errors).Jernej Krempuš put something together for LDC a while back: https://github.com/jerro/ldc/compare/inlining. Unfortunately, it seems like he never got around to submitting the symbol naming fixes to the upstream frontend. Cheers, David
Jul 17 2014
On Wednesday, 16 July 2014 at 20:11:22 UTC, Johannes Pfau wrote:I think it's kinda ridiculous that D embedded code will only be usable with strong optimization flags, but whatever.Do you have numbers? Transitive volatility implies you planned to do very complex things in embedded. If optimizations are so critical, how do you know if complex code can work without them due to its sheer complexity even if typeinfos were removed somehow? On Wednesday, 16 July 2014 at 20:14:38 UTC, Johannes Pfau wrote:I'll take this as you pre-approve all the mentioned extensions? * way to disable typeinfo for structBTW, typeinfos are needed for basic types too and not everything is inline - e.g. array comparison.* way to disable initializer * force inlining * (way to omit copy constructor function / force inline)We have bugzilla for issues and you can refer to them (with appropriate severity) to describe a level of support for embedded (and baremetal). Would be better with numbers to justify severity.
Jul 17 2014
On Wednesday, 16 July 2014 at 20:14:38 UTC, Johannes Pfau wrote:* way to disable typeinfo for struct * way to disable initializer * force inlining * (way to omit copy constructor function / force inline)BTW, can't lto already remove this unused data? LDC inlines simple initializers already, so they are likely to end up unused (same for functions which where inlined).
Jul 17 2014
On Wed, 16 Jul 2014 11:49:21 -0700, Walter Bright wrote:I'll add that if it turns out we can't do a wrapper because of some limitations/bugs in D's expressiveness, that is the problem with D we need to fix, not adding another type modifier. It all comes down to leverage. Making user defined types more capable has far, far more productive leverage than welding specifics into the compiler.A Volatile(T) in stdlib seems to make a lot of sense, I am also wondering if the implementation of shared should be done the same way (if it does already then ignore), just use the compile to enforce shared and have it rewrite it as Shared(T).
Jul 17 2014
On 07/17/2014 04:01 PM, Byron Heads wrote:A Volatile(T) in stdlib seems to make a lot of sense, I am also wondering if the implementation of shared should be done the same way (if it does already then ignore), just use the compile to enforce shared and have it rewrite it as Shared(T). ...I've grepped the DMD source and currently 'shared' is a _bit flag_ set or not set on some uint member field of the 'Type' class. I assume this is also the main reason why adding new type qualifiers is a really unpleasant undertaking: this is a highly non-modular design.
Jul 18 2014
On 7/18/2014 10:53 AM, Timon Gehr wrote:I've grepped the DMD source and currently 'shared' is a _bit flag_ set or not set on some uint member field of the 'Type' class. I assume this is also the main reason why adding new type qualifiers is a really unpleasant undertaking: this is a highly non-modular design.I also recommend that you grep the C++ Standard for 'volatile'. I counted 137 occurrences by page 300 of 1353 pages, at that point I gave up. And those occurrences were not clustered, they were sprinkled everywhere. A C++ compiler is obliged to carefully account for each and every semantic issue in the Standard. Just writing test cases for all those mentions is a major undertaking.
Jul 18 2014
On 7/15/2014 10:15 PM, Johannes Pfau wrote:Yes, I actually tried tested that simple implementation with gdc. With aggressive optimizations the code inlines just fine and it'll generate the same asm as an access to a volatile variable in C. But it's not space-effecient: Even if it inlines, the function is still in the object file as a distinct function. There's TypeInfo and there's the assert(this), there's an initializer symbol and I guess there could be copy constructors as well. I expect many bug reports/fixes till we get a Volatile!T wrapper to be really space-efficient.D is intended to have a robust enough feature set in order to implement types as library types rather than builtin, and have them operate as good as a builtin type would. These are more general issues, and are not specific to volatile.The point is it is not perfectly safe. There is no definition, and certainly no portable definition, as to how many read/write cycles such an operation entails. That's what I meant, and DIP62 says pretty much said the same thing. Please allow me to quote: "CISC machines may allow other operations to memory than simple load/store operations. The question is then if a read-modify-write operation like 'x++' should translate to 'r = load x; r++; x = store r' or to a single add instruction 'add x;'. This DIP does not dictate any specific behaviour, but this is recommended: If it is known that instructions operating on memory do not work for memory mapped IO as expected, compilers should default to generating a load/modify/store sequence. Otherwise compilers should generate the same code sequence as for regular variables. It is recommended to provide a compiler switch on CISC architectures to allow the user to choose between these two different behaviours."But not necessarily in user code. REGISTER |= is quite common in C and it's perfectly safe.As you mentioned in the DIP, read-modify-write operations should be split up, as their semantics are murky.3. if you really hate peek/poke calls appearing in the code, you can use UFCS to make them look like variablesBut you cant do REGISTER.peek() |= 0b1;If people will have to split this up into peek |= poke that's a step backwards from C.Support for op= is fairly simple for VolatilePointerToUint to implement, and it can implement them in terms of peek/poke for precise and portable semantics.I objected to several proposals for namespace support that I regarded as significantly more complex.You were initially in favor of an option which would have allowed to use namespaces in D code as well (namespace A {}). The current implementation is mostly limited to C++ so that's better, but my point was that nobody cared about complexity back then.A few months ago you suggested namespaces for D - in addition to modules - which would have lead to a complexity disaster.C++ namespaces are now implemented in D, and will they lead to complexity disaster? Certainly not in the implementation. For users? I don't think so, but we'll see.The synchronized statement was something I thought would be widely used. It turns out to be largely a failure. opDispatch is actually pretty simple, and I believe it is an "enabling" feature that hasn't, but will, come into its own. Nevertheless, we should not follow the trap of "we made a mistake on A, so that justifies making another mistake B". Nor should we say "feature X has reasons A,B,C,D, therefore reason D is justification enough for other things." Changes to D must stand/fall on their own merits.But it's not only about frequency, it's also about convenience and safety. How often is opDispatch actually used? The synchronized statement? Nevertheless they are there, because someone thought these use cases are important enough.This just shows the priorities of the project leads: Desktop apps matter, we add nogc, c++ namespaces, shared, immutable, const, -cov, ... but for embedded systems we won't even add one qualifier.I don't think this is the right argument. It isn't about embedded systems should be acknowledged with a qualifier. I see it as about how often does MMIO logic appear in an actual embedded program? In the embedded systems I've written, it only appears in a handful of places. It is not pervasive.The perceived importance of a special qualifier for MMIO code is highly subjective,Of course. There is no doubt that both yours and my opinions here are subjective.but I think that most embedded programmers would say it's necessary.I do believe that a wrapper type around peek/poke satisfies essentially all your necessary requirements, except for transitivity.In Andrei's Leverage talk he said (5:57): "Consider this: Convenience, modeling power, efficiency: [...] You gonna have to have a large language. This is a fact of life. [...]". We have weak pure/ strong pure & immutable for functional programming, A complete set of OO primitives, builtin unit tests, code coverage, ... I think it's a correct argument to say that you think embedded system programming is not important enough for the additional complexity introduced by a new qualifier.That is not what I'm saying at all. Please do not conflate disagreement about the best way to achieve a goal with disagreement about the goal. I understand that you and others put a lot of work into DIP62. As I said before, it's a very nice example of a well-done DIP.
Jul 15 2014
Am Tue, 15 Jul 2014 23:38:21 -0700 schrieb Walter Bright <newshound2 digitalmars.com>:On 7/15/2014 10:15 PM, Johannes Pfau wrote:And how do you implement this with a wrapper? This is common in C/C++: struct Timer { uint control; uint data; } enum timerA = (Volatile!Timer)* = cast()oxDEADBEAF; timerA.control |= 0b1; Your peek/poke intrinsics don't even consider this example. If you extend peek/poke to T, then it'll always load the complete type, whereas the C code loads only the field. And you can't even do anything about it, cause peek!T explicitly tells the compiler to load the complete type and prevent optimization of this load... And it seems timerA.control |= 0b1; produces absolute nonsense. It compiles, but * The codegen is horrible. In C this compiles to 3 instructions, in D 30-40 instructions * This doesn't do what it's supposed to do at all. For example it calls the getter twice and never calls the setter. The more I think about it the more corner cases appear. After all implementing this in the library is much more complex than a type qualifier. Some of the complexity is already implemented (alias this, ...) so there's less changes in DMD. But any bug in alias this, templates, etc will immediately leak into Volatile!T.Yes, I actually tried tested that simple implementation with gdc. With aggressive optimizations the code inlines just fine and it'll generate the same asm as an access to a volatile variable in C. But it's not space-effecient: Even if it inlines, the function is still in the object file as a distinct function. There's TypeInfo and there's the assert(this), there's an initializer symbol and I guess there could be copy constructors as well. I expect many bug reports/fixes till we get a Volatile!T wrapper to be really space-efficient.D is intended to have a robust enough feature set in order to implement types as library types rather than builtin, and have them operate as good as a builtin type would. These are more general issues, and are not specific to volatile.In cases where you use |= the exact number of accesses usually does not matter. What matters is that there's at least one access and that it's not reordered with other reads/writes to volatile. For example |= is used on control registers. But I don't care if I read/set the bit twice or once, I just doesn't matter.The point is it is not perfectly safe. There is no definition, and certainly no portable definition, as to how many read/write cycles such an operation entails. That's what I meant, and DIP62 says pretty much said the same thing. Please allow me to quote:But not necessarily in user code. REGISTER |= is quite common in C and it's perfectly safe.As you mentioned in the DIP, read-modify-write operations should be split up, as their semantics are murky.3. if you really hate peek/poke calls appearing in the code, you can use UFCS to make them look like variablesBut you cant do REGISTER.peek() |= 0b1;"CISC machines may allow other operations to memory than simple load/store operations. The question is then if a read-modify-write operation like 'x++' should translate to 'r = load x; r++; x = store r' or to a single add instruction 'add x;'. This DIP does not dictate any specific behaviour, but this is recommended: If it is known that instructions operating on memory do not work for memory mapped IO as expected, compilers should default to generating a load/modify/store sequence. Otherwise compilers should generate the same code sequence as for regular variables. It is recommended to provide a compiler switch on CISC architectures to allow the user to choose between these two different behaviours."And what does peek/poke guarantee in that case? Everything you could specify for peek/poke can be specified for volatile variables as well.See aboveIf people will have to split this up into peek |= poke that's a step backwards from C.Support for op= is fairly simple for VolatilePointerToUint to implement, and it can implement them in terms of peek/poke for precise and portable semantics.I do believe that a wrapper type around peek/poke satisfies essentially all your necessary requirements, except for transitivity.See above. One important point is that this must be as effective as c+volatile (in terms of code size and execution cycles). This is an important aspect for embedded programming and it'll take ages and many language changes to make a type wrapper as efficient. If it's even possible at all.So you do think peek/poke is a better solution than DIP62? Then you have to explain this. All arguments so far have been it's too complex in the compiler implementation, but never that peek/poke is a better way to achieve that goal of accessing volatile memory. In the end it's a judgement call between complexity and more convenience for embedded programming and it's absolutely your decision. But as long as I'm not convinced that peek/poke is a _better_ solution I'll think that embedded programming is not important enough for you to implement the _best_ solution, as it causes some implementation complexity. And this in the end means it's of lower priority to you than keeping some complexity out of the compiler.I think it's a correct argument to say that you think embedded system programming is not important enough for the additional complexity introduced by a new qualifier.That is not what I'm saying at all. Please do not conflate disagreement about the best way to achieve a goal with disagreement about the goal.I understand that you and others put a lot of work into DIP62. As I said before, it's a very nice example of a well-done DIP.That's not the point. If I were convinced there's a better solution, I'd be glad to accept it. But I'm not convinced at all, the only argument against DIP62 so far was complexity.
Jul 16 2014
On 7/16/2014 5:59 AM, Johannes Pfau wrote:And how do you implement this with a wrapper? This is common in C/C++: struct Timer { uint control; uint data; } enum timerA = (Volatile!Timer)* = cast()oxDEADBEAF; timerA.control |= 0b1;auto timerA = VolatileStruct!Timer(0xDEADBEAF); timerA.control |= 1; i.e. with introspection, the template can automatically generate the getters/setters and operator overloads.The more I think about it the more corner cases appear. After all implementing this in the library is much more complex than a type qualifier. Some of the complexity is already implemented (alias this, ...) so there's less changes in DMD. But any bug in alias this, templates, etc will immediately leak into Volatile!T.It's true that VolatileStruct will be more complex than the average template. But it's quite doable, and the user won't see that complexity.I understand, but the pedantic in me notes the word "usually".The point is it is not perfectly safe. There is no definition, and certainly no portable definition, as to how many read/write cycles such an operation entails. That's what I meant, and DIP62 says pretty much said the same thing. Please allow me to quote:In cases where you use |= the exact number of accesses usually does not matter. What matters is that there's at least one access and that it's not reordered with other reads/writes to volatile.For example |= is used on control registers. But I don't care if I read/set the bit twice or once, I just doesn't matter.I understand that it usually does not matter. But sometimes it does. It should be under user control rather than "whatever the compiler does" which can change with optimization settings. C compiler semantics for volatile are "implementation defined", which pretty much means "random".And what does peek/poke guarantee in that case?It guarantees semantics for (A op= B) to be (A = A op B).One important point is that this must be as effective as c+volatile (in terms of code size and execution cycles). This is an important aspect for embedded programmingI fully agree.and it'll take ages and many language changes to make a type wrapper as efficient.This is unduly pessimistic.If it's even possible at all.I see no reason to quit before we start.I think it unfair to impute motives when I've specifically said otherwise.That is not what I'm saying at all. Please do not conflate disagreement about the best way to achieve a goal with disagreement about the goal.So you do think peek/poke is a better solution than DIP62? Then you have to explain this. All arguments so far have been it's too complex in the compiler implementation, but never that peek/poke is a better way to achieve that goal of accessing volatile memory. In the end it's a judgement call between complexity and more convenience for embedded programming and it's absolutely your decision. But as long as I'm not convinced that peek/poke is a _better_ solution I'll think that embedded programming is not important enough for you to implement the _best_ solution, as it causes some implementation complexity. And this in the end means it's of lower priority to you than keeping some complexity out of the compiler.I think it is fair to acknowledge the good work you've done.I understand that you and others put a lot of work into DIP62. As I said before, it's a very nice example of a well-done DIP.That's not the point.If I were convinced there's a better solution, I'd be glad to accept it. But I'm not convinced at all, the only argument against DIP62 so far was complexity.Complexity is not a minor issue. Complexity means time and bugs. I am all too familiar with how volatile in C++ has been costly in a lot of effort, bugs, specification, and code that had no use for volatile, yet had to account for it. BTW, testing of "volatile" behavior in the compiler is problematic as there just doesn't seem to be a reasonable way to do it. This engenders a regular risk of compiler breakage, and shows up to the end user as erratic behavior and lots of cursing at the compiler vendor.
Jul 16 2014
On Wednesday, 16 July 2014 at 05:17:56 UTC, Johannes Pfau wrote:I think if we decide that Volatile!T is the solution I'd probably hack an attribute TypeWrapper for gdc which simply recognized Volatile!T and does the right thing.I'd recommend fantom name, because not only type wrappers can benefit from these optimizations. Should be also applicable to functions and maybe modules.
Jul 16 2014
Kagamin:I'd recommend fantom name, because not only type wrappers can benefit from these optimizations. Should be also applicable to functions and maybe modules.This sounds like something fit for D too, not just gdc. You can use it for std.typecons.Typedef too. Bye, bearophile
Jul 16 2014
Then we can have phantom functions for porting C macros instead of templates :) (oh it's spelled with 'ph').
Jul 16 2014
On Wednesday, 16 July 2014 at 05:17:56 UTC, Johannes Pfau wrote:There's TypeInfo and there's the assert(this)The optimizer should be able to eliminate this assert after inlining as it knows it's not null. It should be looked into, why the optimizer didn't remove it.
Jul 16 2014
On Tuesday, 15 July 2014 at 21:28:24 UTC, Walter Bright wrote:Peek/poke has a 40 year history of being used for MMIO, and I've never heard of it being used for concurrency. I don't think there's endemic confusion, quite unlike volatile.There is a huge confusion about volatile in the native space. No one really knows what it does and when it should be used, because it's kind of rare. I've seen pervasive volatile use make some GPU kernels 100x slower. Programming by coincidence, sure, but _explicit_ barriers, being read/write/optimizer barriers, I find them much better. volatile is more like denial, and importing the history behind the keyword would negative PR. Pascal had no volatile, and using &= from a memory-mapped location doesn't exactly scream "performance".
Jul 16 2014
We had the volatile statement as a compiler barrier in D1. Why not basically that instead of a type qualifier? We pretty much need it back for atomics anyway.
Jul 17 2014
On 17 July 2014 17:16, Sean Kelly via Digitalmars-d <digitalmars-d puremagic.com> wrote:We had the volatile statement as a compiler barrier in D1. Why not basically that instead of a type qualifier? We pretty much need it back for atomics anyway.I don't recall volatile statements ever working properly, or at least working in an agreed way.
Jul 17 2014
On 7/17/14, 12:15 PM, Iain Buclaw via Digitalmars-d wrote:On 17 July 2014 17:16, Sean Kelly via Digitalmars-d <digitalmars-d puremagic.com> wrote:And that goes for volatile in C and C++ as well. -- AndreiWe had the volatile statement as a compiler barrier in D1. Why not basically that instead of a type qualifier? We pretty much need it back for atomics anyway.I don't recall volatile statements ever working properly, or at least working in an agreed way.
Jul 17 2014
On 7/17/2014 9:16 AM, Sean Kelly wrote:We had the volatile statement as a compiler barrier in D1. Why not basically that instead of a type qualifier? We pretty much need it back for atomics anyway.Volatile and atomic semantics are very different, are historically conflated and confused, and I think it's well worth it to use completely distinct mechanisms for both.
Jul 17 2014
On Thursday, 17 July 2014 at 19:31:25 UTC, Walter Bright wrote:On 7/17/2014 9:16 AM, Sean Kelly wrote:Fair enough. My point was simply that volatile as defined for D1 seems potentially useful. I certainly liked it for concurrent programming, even if the guarantees it provided were only ever on paper and not actually implemented.We had the volatile statement as a compiler barrier in D1. Why not basically that instead of a type qualifier? We pretty much need it back for atomics anyway.Volatile and atomic semantics are very different, are historically conflated and confused, and I think it's well worth it to use completely distinct mechanisms for both.
Jul 17 2014
On Tuesday, 15 July 2014 at 20:45:20 UTC, Johannes Pfau wrote:Am Tue, 15 Jul 2014 12:48:21 -0700 schrieb Walter Bright <newshound2 digitalmars.com>:You can create a function which returns a struct that wraps your desired read-modify-write semantics, e.g.: auto myvolatile(size_t* addr) { static struct VolatileWrapper { size_t* addr; // read-modify-write auto opOpAssign(string op)(size_t rhs) { /*...*/ } // write auto opAssign(size_t rhs) { addr.poke(rhs); return this; } // read auto rval() { return addr.peek(); } alias rval this; } return VolatileWrapper(addr); } REGISTER.myvolatile() |= 0b1; Taking it further, I think you could implement something that looked like this: enum : VolatileWrapper { TIMER0 = VolatileWrapper(REGISTER), // etc } Don't get me wrong, I'm in favor of volatile, it's just that that counter-argument isn't extremely convincing.3. if you really hate peek/poke calls appearing in the code, you can use UFCS to make them look like variablesBut you cant do REGISTER.peek() |= 0b1;
Jul 15 2014
I've been jawboning about peek/poke for years - finally decided to implement it. https://issues.dlang.org/show_bug.cgi?id=13138
Jul 15 2014
On 7/15/14, 6:25 PM, Walter Bright via Digitalmars-d wrote:I've been jawboning about peek/poke for years - finally decided to implement it. https://issues.dlang.org/show_bug.cgi?id=13138I don't have any skin in this particular discussion, but it's worth pointing out here that while it's cute that builtin's can be added, when they're not part of the language definition, they're not really part of a portable solution.
Jul 15 2014
On 7/15/2014 9:29 PM, Brad Roberts via Digitalmars-d wrote:I don't have any skin in this particular discussion, but it's worth pointing out here that while it's cute that builtin's can be added, when they're not part of the language definition, they're not really part of a portable solution.They're as portable as we want them to be.
Jul 15 2014
On 16 July 2014 07:12, Walter Bright via Digitalmars-d <digitalmars-d puremagic.com> wrote:On 7/15/2014 9:29 PM, Brad Roberts via Digitalmars-d wrote:I read that as, not portable at all then. :o)I don't have any skin in this particular discussion, but it's worth pointing out here that while it's cute that builtin's can be added, when they're not part of the language definition, they're not really part of a portable solution.They're as portable as we want them to be.
Jul 15 2014
On 7/15/2014 11:32 PM, Iain Buclaw via Digitalmars-d wrote:On 16 July 2014 07:12, Walter Bright via Digitalmars-d <digitalmars-d puremagic.com> wrote:C'mon, guys. It's our spec and we can make it work.On 7/15/2014 9:29 PM, Brad Roberts via Digitalmars-d wrote:I read that as, not portable at all then. :o)I don't have any skin in this particular discussion, but it's worth pointing out here that while it's cute that builtin's can be added, when they're not part of the language definition, they're not really part of a portable solution.They're as portable as we want them to be.
Jul 16 2014
On 16 July 2014 08:08, Walter Bright via Digitalmars-d <digitalmars-d puremagic.com> wrote:On 7/15/2014 11:32 PM, Iain Buclaw via Digitalmars-d wrote:And we are trying to make it work. I actually have quite a bit to be planned out after removing the 'volatility' from shared.On 16 July 2014 07:12, Walter Bright via Digitalmars-d <digitalmars-d puremagic.com> wrote:C'mon, guys. It's our spec and we can make it work.On 7/15/2014 9:29 PM, Brad Roberts via Digitalmars-d wrote:I don't have any skin in this particular discussion, but it's worth pointing out here that while it's cute that builtin's can be added, when they're not part of the language definition, they're not really part of a portable solution.They're as portable as we want them to be.I read that as, not portable at all then. :o)From the following spec:''' Unlike the shared attribute, __gshared provides no safe-guards against data races or other multi-threaded synchronization issues. It is the responsibility of the programmer to ensure that access to variables marked __gshared is synchronized correctly. ''' GDC will likely in future emit atomic loads, stores, and possibly fences around the use of shared data. As there is no specific meaning to what 'safe-guards' come with shared. I interpret that to keep in line with a happens-before style relationship with sequential reads/writes to shared data. Regards Iain
Jul 16 2014
On Wednesday, 16 July 2014 at 01:25:05 UTC, Walter Bright wrote:I've been jawboning about peek/poke for years - finally decided to implement it. https://issues.dlang.org/show_bug.cgi?id=13138If this is indeed the last straw, please mark DIP20 as accepted/implemented and DIP62 as rejected. I have neither the authority nor the stomach for it. Mike DIP20: http://wiki.dlang.org/DIP20 DIP62: http://wiki.dlang.org/DIP62
Jul 15 2014
On Tuesday, 15 July 2014 at 19:48:23 UTC, Walter Bright wrote:The one thing peek/poke doesn't offer is transitivity. C/C++ don't offer volatile transitivity either. I'm not at all sure that anyone builds a data structure in memory-mapped registers, so I'm not convinced this is a need.Here's an example from the STM32F4 ARM Cortex-M peripheral library: #define __IO volatile /** * brief DMA Controller */ typedef struct { __IO uint32_t CR; /*!< DMA stream x configuration register */ __IO uint32_t NDTR; /*!< DMA stream x number of data register */ __IO uint32_t PAR; /*!< DMA stream x peripheral address register */ __IO uint32_t M0AR; /*!< DMA stream x memory 0 address register */ __IO uint32_t M1AR; /*!< DMA stream x memory 1 address register */ __IO uint32_t FCR; /*!< DMA stream x FIFO control register */ } DMA_Stream_TypeDef; typedef struct { __IO uint32_t LISR; /*!< DMA low interrupt status register, Address offset: 0x00 */ __IO uint32_t HISR; /*!< DMA high interrupt status register, Address offset: 0x04 */ __IO uint32_t LIFCR; /*!< DMA low interrupt flag clear register, Address offset: 0x08 */ __IO uint32_t HIFCR; /*!< DMA high interrupt flag clear register, Address offset: 0x0C */ } DMA_TypeDef; There are several hundreds lines of this stuff in the library. It is used to model register banks. Each register bank roughly corresponds to one of the embedded peripherals. Mike
Jul 15 2014
On 7/15/2014 9:30 PM, Mike wrote:On Tuesday, 15 July 2014 at 19:48:23 UTC, Walter Bright wrote:There are no pointers to pointers there, so there is no transitivity issue. A single volatile reference to the whole struct will suffice.The one thing peek/poke doesn't offer is transitivity. C/C++ don't offer volatile transitivity either. I'm not at all sure that anyone builds a data structure in memory-mapped registers, so I'm not convinced this is a need.Here's an example from the STM32F4 ARM Cortex-M peripheral library: #define __IO volatile /** * brief DMA Controller */ typedef struct { __IO uint32_t CR; /*!< DMA stream x configuration register */ __IO uint32_t NDTR; /*!< DMA stream x number of data register */ __IO uint32_t PAR; /*!< DMA stream x peripheral address register */ __IO uint32_t M0AR; /*!< DMA stream x memory 0 address register */ __IO uint32_t M1AR; /*!< DMA stream x memory 1 address register */ __IO uint32_t FCR; /*!< DMA stream x FIFO control register */ } DMA_Stream_TypeDef; typedef struct { __IO uint32_t LISR; /*!< DMA low interrupt status register, Address offset: 0x00 */ __IO uint32_t HISR; /*!< DMA high interrupt status register, Address offset: 0x04 */ __IO uint32_t LIFCR; /*!< DMA low interrupt flag clear register, Address offset: 0x08 */ __IO uint32_t HIFCR; /*!< DMA high interrupt flag clear register, Address offset: 0x0C */ } DMA_TypeDef; There are several hundreds lines of this stuff in the library. It is used to model register banks. Each register bank roughly corresponds to one of the embedded peripherals.
Jul 15 2014
On Tuesday, 15 July 2014 at 16:27:53 UTC, Johannes Pfau wrote:You may have seen Mike's talk about D on embedded systems and were surprised about Iain's comment that using 'shared' as a replacement for 'volatile' is actually only possible because of a bug in gdc. DIP62 describes how to solve this problem and make embedded programming a first-class citizen in D: http://wiki.dlang.org/DIP62Well, this was posted just as I went to sleep for the night (I'm in Asia), so I wasn't able to get my word in until the end. Sorry! DIP62 is the exemplary DIP: well researched and thoroughly specified. I was actually in favor of peek/poke intrinsics until the DIP62 and its citations took me to school. I still believe that the type qualifier is the right tool for the job, because despite having peek/poke intrinsics, users will still navigate towards making a volatile type to wrap peek/poke, as has already been proposed [1] Volatile!(T), [2] struct VolatilePointerToUint, etc... It actually justifies DIP62, IMO. After reading through this thread, the only argument against DIP62 that really made much sense was the cost/benefit tradeoff in the implementation. I can't speak to the complexity of the implementation, but I do concede that the use of volatile will only be used in the lowest layer of hardware abstraction (although it will be used quite often there), and the workarounds are not burdensome. The type qualifier would avoid such nonsense. Anyway, I can see where this is going[3], so I guess I'll have to patch my airplane[4] with a template to make it fly :( [1] http://forum.dlang.org/post/eznqslwpnxfnwlgzlsll forum.dlang.org [2] http://forum.dlang.org/post/lq4hqi$1j77$1 digitalmars.com [3] https://github.com/D-Programming-Language/dmd/pull/3773 [4] http://www.youtube.com/watch?feature=player_detailpage&v=WKRRgcEk0wg#t=65 Mike
Jul 15 2014
On 7/15/14, 9:25 AM, Johannes Pfau wrote:DIP62 describes how to solve this problem and make embedded programming a first-class citizen in D: http://wiki.dlang.org/DIP62[snip] The good: This is a very crisply-written and comprehensive proposal. It could use a little work (e.g. define the typing relationships among volatile, other qualifiers, and unqualified if any) but completeness is outstanding. The bad: I think we should reject this proposal. Moreover, the comprehensiveness of the DIP gives me confidence to assess that no amount of work on the proposal could take it to acceptance. The ugly: There's none! Actually there's some nice stuff: we could use a bunch of this proposal to specify alternative approaches such as peek/poke. ============= My rationale: * I reject he premise ("The volatile qualifier is often used for embedded programming in C/C++[1]".) At most I agree with "often" as in "you'll see at least one use of volatile in some, maybe many, embedded programs" but not "volatile is present many times in a typical embedded application". Hardware/firmware providers often focus on giving access to low-level hardware facilities by using classic encapsulation and exposing APIs, as opposed to just publishing the raw addresses of various ports. Furthermore, the link given as a justification (http://www.barrgroup.com/Embedded-Systems/How-To/C-Volatile-Keyword) is not support for the statement in text, but instead a mere description of volatile, which is also technically incorrect (viz. the use of volatile with threads). * "volatile" has been one of the most awkward parts of the C and C++ specifications. If we're to learn of anything about it, it's we must stay away from it. It is extremely heavy for what it provides. * Currently C++ defines observable behavior as "reads and writes of volatile variables and sequences of calls to certain library functions". There is only a small burden associated with sequences of function calls, but the volatile parts are complicating the specification enormously. We'd be better off specifying sequences of calls to functions as defined behavior. * Adding qualifiers is heavy for the language definition and implementation. Probably they're the most impactful artifact to add. * Transitivity is only partially justified. For immutable and shared data, transitivity is motivated by code access patterns that chase pointers, but for volatile data there's no hardware reality reflected by transitivity: MMIO only pertains simple address ranges, not node-based volatile structures. The justification in section "Transitivity" is valid but puts undue emphasis on obscure interrupt service routines; even C doesn't go that far (instead it just limits the functions such routines may call). * The DIP is correct in mentioning that "volatility" is a property of the memory location so it should apply to the respective variable's type, as opposed to just letting the variable have a regular type and then unsafely relying on calls to e.g. peek and poke. But here's a practicality argument that's been already mentioned - such volatile data is rare and classic encapsulation techniques can be applied to it. * The scalability aspect should be emphasized: a large system may have more need to synchronize threads or to characterize parts of it as functional (pure + immutable). In contrast, a large embedded system does not manifest a proportionally larger presence of volatile data. ============== I think an approach based on functions peek/poke is a lot more promising. D programs must define sequences of std calls anyway, otherwise even the simplest programs that use writeln("What's your name?") followed by a readln() are incorrect. So in a way peek/poke come for "free". Clearly an approach that adds a qualifier would have superior abstraction capabilities, but my experience with C++ (both specification and use) has been that the weight to power ratio of volatile is enormous. Andrei
Jul 17 2014
On Thursday, 17 July 2014 at 15:58:05 UTC, Andrei Alexandrescu wrote:I think an approach based on functions peek/poke is a lot more promising. D programs must define sequences of std calls anyway, otherwise even the simplest programs that use writeln("What's your name?") followed by a readln() are incorrect. So in a way peek/poke come for "free".Could you expand on this "sequences of calls"? What exactly do you mean by defining it?
Jul 17 2014
On 7/17/14, 9:06 AM, John Colvin wrote:On Thursday, 17 July 2014 at 15:58:05 UTC, Andrei Alexandrescu wrote:For example if you run this C program: printf("ur name: "); fflush(stdout); scanf("%s", &name); printf("Hello, %s!", name); it's guaranteed the order of calls is preserved. AndreiI think an approach based on functions peek/poke is a lot more promising. D programs must define sequences of std calls anyway, otherwise even the simplest programs that use writeln("What's your name?") followed by a readln() are incorrect. So in a way peek/poke come for "free".Could you expand on this "sequences of calls"? What exactly do you mean by defining it?
Jul 17 2014
Am Thu, 17 Jul 2014 11:43:04 -0700 schrieb Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>:On 7/17/14, 9:06 AM, John Colvin wrote:Function calls are actually compiler barriers in every C (like) compiler, I guess that's even in the standard. Relying on this too much can be dangerous with inlining though. Back to topic: Thanks for your response, so let's go with peek/poke / volatileLoad/Store then.On Thursday, 17 July 2014 at 15:58:05 UTC, Andrei Alexandrescu wrote:For example if you run this C program: printf("ur name: "); fflush(stdout); scanf("%s", &name); printf("Hello, %s!", name); it's guaranteed the order of calls is preserved.I think an approach based on functions peek/poke is a lot more promising. D programs must define sequences of std calls anyway, otherwise even the simplest programs that use writeln("What's your name?") followed by a readln() are incorrect. So in a way peek/poke come for "free".Could you expand on this "sequences of calls"? What exactly do you mean by defining it?
Jul 17 2014
On Thursday, 17 July 2014 at 19:56:23 UTC, Johannes Pfau wrote:Am Thu, 17 Jul 2014 11:43:04 -0700 schrieb Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>:only because rearranging those cannot be proved to create the same observable semantics (in this case of course it can be actively disproved). That doesn't apply to all sequences of function calls, thankfully.On 7/17/14, 9:06 AM, John Colvin wrote:On Thursday, 17 July 2014 at 15:58:05 UTC, Andrei Alexandrescu wrote:For example if you run this C program: printf("ur name: "); fflush(stdout); scanf("%s", &name); printf("Hello, %s!", name); it's guaranteed the order of calls is preserved.I think an approach based on functions peek/poke is a lot more promising. D programs must define sequences of std calls anyway, otherwise even the simplest programs that use writeln("What's your name?") followed by a readln() are incorrect. So in a way peek/poke come for "free".Could you expand on this "sequences of calls"? What exactly do you mean by defining it?Function calls are actually compiler barriers in every C (like) compiler, I guess that's even in the standard. Relying on this too much can be dangerous with inlining though.As I understand it (I'm not a compiler writer or a C standard expert): *Opaque* function calls are a compiler memory barrier in C and that's only by necessity (that opaque function could contain a memory barrier, amongst other problems). If the compiler can guarantee to maintain the same visible semantics of defined behaviour it can rearrange function calls just as freely as anything else.
Jul 17 2014
On 07/17/2014 05:58 PM, Andrei Alexandrescu wrote:* "volatile" has been one of the most awkward parts of the C and C++ specifications. If we're to learn of anything about it, it's we must stay away from it. It is extremely heavy for what it provides. ... ============== I think an approach based on functions peek/poke is a lot more promising.I am not saying that we should add volatile as a type qualifier, but it should be noted that programs with volatile type qualifiers can be lowered to programs using peek/poke and possibly other API calls with limited effort. (it is especially limited since the effort could be mostly shared with 'shared', both in specification and implementation, as it is another type qualifier that (will) (presumably) influence(s) the behaviour of loads and stores.)
Jul 18 2014
On 7/18/2014 11:06 AM, Timon Gehr wrote:On 07/17/2014 05:58 PM, Andrei Alexandrescu wrote:That doesn't change the heavy effort in the semantic front end and language specification to support volatile. Take the C++ Standard and grep for 'volatile' and 'cv-qualifier'.* "volatile" has been one of the most awkward parts of the C and C++ specifications. If we're to learn of anything about it, it's we must stay away from it. It is extremely heavy for what it provides. ... ============== I think an approach based on functions peek/poke is a lot more promising.I am not saying that we should add volatile as a type qualifier, but it should be noted that programs with volatile type qualifiers can be lowered to programs using peek/poke and possibly other API calls with limited effort. (it is especially limited since the effort could be mostly shared with 'shared', both in specification and implementation, as it is another type qualifier that (will) (presumably) influence(s) the behaviour of loads and stores.)
Jul 18 2014
On 07/18/2014 08:37 PM, Walter Bright wrote:On 7/18/2014 11:06 AM, Timon Gehr wrote:What does this mean in detail? Is it just about the unfortunate AST representation chosen or are there more sources of heavy effort in the front end?On 07/17/2014 05:58 PM, Andrei Alexandrescu wrote:That doesn't change the heavy effort in the semantic front end* "volatile" has been one of the most awkward parts of the C and C++ specifications. If we're to learn of anything about it, it's we must stay away from it. It is extremely heavy for what it provides. ... ============== I think an approach based on functions peek/poke is a lot more promising.I am not saying that we should add volatile as a type qualifier, but it should be noted that programs with volatile type qualifiers can be lowered to programs using peek/poke and possibly other API calls with limited effort. (it is especially limited since the effort could be mostly shared with 'shared', both in specification and implementation, as it is another type qualifier that (will) (presumably) influence(s) the behaviour of loads and stores.)and language specification to support volatile. ...The language specification would just state what the rules for lowering are. In any case, I think the other points that Andrei raised are convincing enough.
Jul 18 2014
On 7/18/2014 4:58 PM, Timon Gehr wrote:On 07/18/2014 08:37 PM, Walter Bright wrote:Again, take a look at the hundreds of references to volatile all through the C++ specification for some idea.That doesn't change the heavy effort in the semantic front endWhat does this mean in detail? Is it just about the unfortunate AST representation chosen or are there more sources of heavy effort in the front end?
Jul 18 2014
On 07/19/2014 02:07 AM, Walter Bright wrote:On 7/18/2014 4:58 PM, Timon Gehr wrote:I am not suggesting to do what C++ does.On 07/18/2014 08:37 PM, Walter Bright wrote:Again, take a look at the hundreds of references to volatile all through the C++ specification for some idea.That doesn't change the heavy effort in the semantic front endWhat does this mean in detail? Is it just about the unfortunate AST representation chosen or are there more sources of heavy effort in the front end?
Jul 18 2014
On 7/18/2014 6:16 PM, Timon Gehr wrote:On 07/19/2014 02:07 AM, Walter Bright wrote:That is what DIP62 suggests - the only difference being transitivity.On 7/18/2014 4:58 PM, Timon Gehr wrote:I am not suggesting to do what C++ does.On 07/18/2014 08:37 PM, Walter Bright wrote:Again, take a look at the hundreds of references to volatile all through the C++ specification for some idea.That doesn't change the heavy effort in the semantic front endWhat does this mean in detail? Is it just about the unfortunate AST representation chosen or are there more sources of heavy effort in the front end?
Jul 18 2014