www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Discussion on static reflection syntax in C++

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Of possible interest:

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2320r0.pdf
Feb 22
next sibling parent reply 12345swordy <alexanderheistermann gmail.com> writes:
On Monday, 22 February 2021 at 16:27:49 UTC, Andrei Alexandrescu 
wrote:
 Of possible interest:

 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2320r0.pdf
Great, let's make c++ more confusing to newbies. -Alex
Feb 22
parent reply 12345swordy <alexanderheistermann gmail.com> writes:
On Monday, 22 February 2021 at 16:43:34 UTC, 12345swordy wrote:
 On Monday, 22 February 2021 at 16:27:49 UTC, Andrei 
 Alexandrescu wrote:
 Of possible interest:

 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2320r0.pdf
Great, let's make c++ more confusing to newbies. -Alex
Oh great! The named parameter proposal for c++ is got to be the worst version of named parameters that I ever seen! https://groups.google.com/a/isocpp.org/g/std-proposals/c/3dUkwyp2Ie4/m/rZ8dgxVlCgAJ -Alex
Feb 22
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 23/02/2021 7:22 AM, 12345swordy wrote:
 Oh great! The named parameter proposal for c++ is got to be the worst 
 version of named parameters that I ever seen!
 https://groups.google.com/a/isocpp.org/g/std-proposals/c/3dUkwyp2Ie4/m/rZ8dgxVlCgAJ
They really went with the kitchen sink approach didn't they? Yikes.
Feb 22
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Feb 23, 2021 at 12:35:31PM +1300, rikki cattermole via Digitalmars-d
wrote:
 On 23/02/2021 7:22 AM, 12345swordy wrote:
 Oh great! The named parameter proposal for c++ is got to be the worst
 version of named parameters that I ever seen!
 https://groups.google.com/a/isocpp.org/g/std-proposals/c/3dUkwyp2Ie4/m/rZ8dgxVlCgAJ
They really went with the kitchen sink approach didn't they? Yikes.
Seeing as this is C++ we're talking about, a kitchen sink approach doesn't surprise me. (Even if it does disgust me. :-D) T -- Life is too short to run proprietary software. -- Bdale Garbee
Feb 22
prev sibling next sibling parent reply bachmeier <no spam.net> writes:
On Monday, 22 February 2021 at 16:27:49 UTC, Andrei Alexandrescu 
wrote:
 Of possible interest:

 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2320r0.pdf
Good this is proposed for C++ and not for D. The complexity/benefit ratio of adding another meaning of ^ and [:refl:] is rather high. Citing * and & as models for ^ is reasonable only to someone that has not tried to teach others to program. At least to me, this is horrible: f<([:Refl:])>(); In complete seriousness, it would be better to use emoji than to write things like that.
Feb 22
parent Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Monday, 22 February 2021 at 16:47:13 UTC, bachmeier wrote:
 On Monday, 22 February 2021 at 16:27:49 UTC, Andrei 
 Alexandrescu wrote:
 Of possible interest:

 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2320r0.pdf
Good this is proposed for C++ and not for D. The complexity/benefit ratio of adding another meaning of ^ and [:refl:] is rather high. Citing * and & as models for ^ is reasonable only to someone that has not tried to teach others to program. At least to me, this is horrible: f<([:Refl:])>();
aka as <:Rofl:> <º))))><
 In complete seriousness, it would be better to use emoji than 
 to write things like that.
Feb 23
prev sibling next sibling parent reply IGotD- <nise nise.com> writes:
On Monday, 22 February 2021 at 16:27:49 UTC, Andrei Alexandrescu 
wrote:
 Of possible interest:

 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2320r0.pdf
Thank you for posting this. Oh my god, let's hope this goes through and let C++ destroy itself. It seems like the C++ maintainers are becoming more and more obsessed with colons.
Feb 22
parent reply Dukc <ajieskola gmail.com> writes:
On Monday, 22 February 2021 at 16:54:31 UTC, IGotD- wrote:
 Oh my god, let's hope this goes through and let C++ destroy 
 itself.
I can't agree with this wish - not everybody can switch. If something like this goes through it'd only make life more difficult to those who don't have the luxury of D (or Rust, Go, Nim, etc.). I believe C++ is still an improvement over C (at least if you don't go crazy with all the complex features). But I too kind of fail to see how `f(...[:tuple:]..., a, b)` would cut it when `tuple.expand.f(a,b)` has been possible in "another language" for like 10 years.
Feb 22
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Feb 22, 2021 at 07:27:29PM +0000, Dukc via Digitalmars-d wrote:
 On Monday, 22 February 2021 at 16:54:31 UTC, IGotD- wrote:
 Oh my god, let's hope this goes through and let C++ destroy itself.
I can't agree with this wish - not everybody can switch. If something like this goes through it'd only make life more difficult to those who don't have the luxury of D (or Rust, Go, Nim, etc.).
That would perhaps put more pressure on C++ devs to switch to a saner language! ;-)
 I believe C++ is still an improvement over C (at least if you don't go
 crazy with all the complex features).
That is precisely the problem: all the complex features are still there, beckoning every new hire to use them as a quick-fix to make a release deadline. Unless you keep a very tight control over what features can be used -- essentially balkanizing the language, which has already happened for at least a decade, probably longer -- the codebase becomes a gigantic, unmaintainable mess. Years ago, I worked in a team project where there was a C++-based infrastructure so fancy and over-engineered, that after a while nobody knew how to use it properly and started spending more time working around it than using it. Eventually, we ditched the C++ portion completely and rewrote it from scratch in C. It was a refreshing change. C's relative dearth of features was certainly limiting, but limitation is not a bad thing in a team where people are constantly coming and going. No matter how horribly the code devolved, there was still only a small set of features it could abuse, and the scope of abuse is well-known and manageable. It wasn't *enjoyable* to go back to C per se (D takes the cake on that one :-D), but it was definitely better than drowning in the ocean of badly-interacting misfeatures that is C++.
 But I too kind of fail to see how `f(...[:tuple:]..., a, b)` would cut it
 when `tuple.expand.f(a,b)` has been possible in "another language" for like
 10 years.
That's why C++'s early death would do everyone some good. Maybe some of the survivors would discover D. :-D T -- Programming is not just an act of telling a computer what to do: it is also an act of telling other programmers what you wished the computer to do. Both are important, and the latter deserves care. -- Andrew Morton
Feb 22
parent reply Dukc <ajieskola gmail.com> writes:
On Monday, 22 February 2021 at 21:16:07 UTC, H. S. Teoh wrote:
 That would perhaps put more pressure on C++ devs to switch to a 
 saner language! ;-)
But who would maintain the C++ programs then? There may be a reasonable case to discourage new projects in C++, but's that's hardly the same as trying to kill C++ programming. Until we can automatically transpile everything I quess.
 Years ago, I worked in a team project where there was a 
 C++-based infrastructure so fancy and over-engineered, that 
 after a while nobody knew how to use it properly and started 
 spending more time working around it than using it.  
 Eventually, we ditched the C++ portion completely and rewrote 
 it from scratch in C.  It was a refreshing change.  C's 
 relative dearth of features was certainly limiting, but 
 limitation is not a bad thing in a team where people are 
 constantly coming and going. No matter how horribly the code 
 devolved, there was still only a small set of features it could 
 abuse, and the scope of abuse is well-known and manageable.  It 
 wasn't *enjoyable* to go back to C per se (D takes the cake on 
 that one :-D), but it was definitely better than drowning in 
 the ocean of badly-interacting misfeatures that is C++.
Hmm, I see. The engineer in me says it's foolish to ditch all C++, keep the function overloading and simple templates at least (I'm thinking something like `T binaryFun<T>(T arg1, T arg2)`). But I can kinda see that rules like that might have weaker authority than a simple language change. I'm still suspicious about the tradeoff you made but at least I see potential reason to accept it.
 But I too kind of fail to see how `f(...[:tuple:]..., a, b)` 
 would cut it
 when `tuple.expand.f(a,b)` has been possible in "another 
 language" for like
 10 years.
That's why C++'s early death would do everyone some good. Maybe some of the survivors would discover D. :-D
I meant that I don't see how it would cut to add as a new language feature. But for the hapless programmer who is down to either it, the C preprocessor, or copypasta, it just might make sense IMO. Just barely, and just maybe.
Feb 22
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Feb 22, 2021 at 09:47:48PM +0000, Dukc via Digitalmars-d wrote:
 On Monday, 22 February 2021 at 21:16:07 UTC, H. S. Teoh wrote:
 That would perhaps put more pressure on C++ devs to switch to a
 saner language! ;-)
But who would maintain the C++ programs then? There may be a reasonable case to discourage new projects in C++, but's that's hardly the same as trying to kill C++ programming.
Relax, nobody is "trying to kill C++ programming". But the more the C++ committee tries to add more fat to an already overweight beast of a language, its inevitable demise draws nearer, and bystanders like myself enjoy speculating on the benefits that everyone would enjoy once the beast is finally, fully, dead.
 Until we can automatically transpile everything I quess.
I don't think you need to worry; the industry has enough C++ code around that it will last for a *long* time yet. And if the beast ever actually dies, I'm sure more than just a handful of clever heads would come up with a good way of migrating everything over. (More like, everyone and their neighbour's dog will invent a way, of which may be a handful will survive the test of real-world code and become widely disseminated as the solution(s) to the migration problem.)
 Years ago, I worked in a team project where there was a C++-based
 infrastructure so fancy and over-engineered, that after a while
 nobody knew how to use it properly and started spending more time
 working around it than using it.  Eventually, we ditched the C++
 portion completely and rewrote it from scratch in C.  It was a
 refreshing change.  C's relative dearth of features was certainly
 limiting, but limitation is not a bad thing in a team where people
 are constantly coming and going. No matter how horribly the code
 devolved, there was still only a small set of features it could
 abuse, and the scope of abuse is well-known and manageable.  It
 wasn't *enjoyable* to go back to C per se (D takes the cake on that
 one :-D), but it was definitely better than drowning in the ocean of
 badly-interacting misfeatures that is C++.
Hmm, I see. The engineer in me says it's foolish to ditch all C++, keep the function overloading and simple templates at least (I'm thinking something like `T binaryFun<T>(T arg1, T arg2)`).
Some of the programmers on the team were so badly burned by the experience that they flinch at the mere sight of templates at a distance of 10 feet away.
 But I can kinda see that rules like that might have weaker authority
 than a simple language change.
We're talking about real world code here, written in a real company by real people (most of whom have moved on to better things) and running in real customer environments. Experience has shown time and again that such "rules" do not work. Never did work. It's programming by convention, as Walter often refers to, and everyone knows what happens when the release deadline is looming and everyone has more on their plate than they could possibly hope to finish. You just reach out and use whatever straw you could grasp to get it working and move on. If a feature is in the language, people will use it. The boss isn't going to stop a release just because somebody broke the rules and as a result got the code working by the deadline. (As opposed to, y'know, sticking to the rules and *not* shipping on time.) Besides, having features in the language that nobody ever uses is really pointless. Better throw it out and end up with a better, slimmer language.
 I'm still suspicious about the tradeoff you made but at least I see
 potential reason to accept it.
[...] I wasn't the one who made the decision. If it were up to me, I'd have stuck with C++ somehow. But I can totally sympathize with why the PTB chose to ditch it. It's hard not to sympathize when I've had the dubious privilege of writing a last-minute hack function call that involved 6 layers of abstraction, one of which involves an RPC bridge that uses fwrite, fread, and a temporary file in order to bridge the gap between the C++ and non-C++ parts of the code. (You *know* something has gone horribly wrong when the act of making a function call involves disk I/O and a temporary file (a *hard-coded* one...). And that's on top of several other layers of marshalling/unmarshalling. Don't ask, you do *not* want to know.) The day we finally could delete the entire C++ branch of the source there was a veritable celebration. We typed every `rm -rf` command with relish and exquisite delight. :-P T -- Why do conspiracy theories always come from the same people??
Feb 22
prev sibling next sibling parent reply SealabJaster <sealabjaster gmail.com> writes:
On Monday, 22 February 2021 at 16:27:49 UTC, Andrei Alexandrescu 
wrote:
 Of possible interest:

 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2320r0.pdf
"Readability. Obviously, we’d like our programs to be readable. We want syntax that is both visually distinctive yet comprehensible." Can anyone more experienced with C++ confirm that this is in any way readable and easy to understand? Because my definition of "readability" appears to be vastly different, especially when I imagine it being used alongside the rest of C++'s symbol spam. "f<([:Refl:])>();" wat "(: R :). Looks like smiley faces" But apparently "[:" and ":]" don't. Part of me feels like the designer(s) really wanted to use square brackets for that extra *artistic touch*. I don't really have anything constructive to say about this (presumably) draft proposal, because the actual feature itself is of course pretty useful, and I'm not really qualified to say whether this proposal is a good fit or not. But why is it that C++ uses such unusual syntax for everything?
Feb 22
next sibling parent SealabJaster <sealabjaster gmail.com> writes:
On Monday, 22 February 2021 at 18:01:34 UTC, SealabJaster wrote:
 On Monday, 22 February 2021 at 16:27:49 UTC, Andrei 
 Alexandrescu wrote:
 Of possible interest:

 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2320r0.pdf
Also for that first example under 5.3.3, I hope that they made a mistake when writing it, because I really don't see how `cout << %x;` prints the value of "V" while `cout << t.%x;` prints "t.m". I ask this because I have a slight concern that this might actually be intentional.
Feb 22
prev sibling parent reply Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Monday, 22 February 2021 at 18:01:34 UTC, SealabJaster wrote:
 On Monday, 22 February 2021 at 16:27:49 UTC, Andrei 
 Alexandrescu wrote:
 Of possible interest:

 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2320r0.pdf
"Readability. Obviously, we’d like our programs to be readable. We want syntax that is both visually distinctive yet comprehensible." Can anyone more experienced with C++ confirm that this is in any way readable and easy to understand? Because my definition of "readability" appears to be vastly different, especially when I imagine it being used alongside the rest of C++'s symbol spam. "f<([:Refl:])>();"
🤣 Is this real life
Feb 23
parent reply Guillaume Piolat <first.name spam.org> writes:
On Tuesday, 23 February 2021 at 10:14:50 UTC, Imperatorn wrote:
 Is this real life
C++ is quickly becoming that annoying legacy langage that no one likes. I would be incredibly wary starting a C++ project in 2021 ; in my exprience literally anything else is a more productive use of your time. Even in the audio space which is 95% C++, the top product is FLStudio, a product that was originally from Delphi. Friendlier languages lead to friendlier products for some reasons.
Feb 23
next sibling parent reply Paulo Pinto <pjmlp progtools.org> writes:
On Tuesday, 23 February 2021 at 10:24:26 UTC, Guillaume Piolat 
wrote:
 On Tuesday, 23 February 2021 at 10:14:50 UTC, Imperatorn wrote:
 Is this real life
C++ is quickly becoming that annoying legacy langage that no one likes. I would be incredibly wary starting a C++ project in 2021 ; in my exprience literally anything else is a more productive use of your time. Even in the audio space which is 95% C++, the top product is FLStudio, a product that was originally from Delphi. Friendlier languages lead to friendlier products for some reasons.
The effort to avoid C++ in some domains is very high, for example on Windows, it seems every effort from DevDiv to improve the overall experience gets botched by WinDev pushing their C++ agenda. Now it remains to be seen what uphill battles the newly created Microsoft Rust team will need to face as well. This is just one example among many, either one is prepared to be the snowflake that does everything alone, and spends development money getting stuff to work, or just integrate the existing ecosystem and proceed to spend customer's money on actual product development. I cannot bill hours on "working around existing C++ SDK, because there is better out there".
Feb 23
parent Guillaume Piolat <first.name spam.org> writes:
On Tuesday, 23 February 2021 at 13:31:00 UTC, Paulo Pinto wrote:
 spends development money getting stuff to work, or just 
 integrate the existing ecosystem and proceed to spend 
 customer's money on actual product development.
It costs very real development money to live with C++ and "getting stuff to work". If anything, D is much more debt-conscious since frontends are shared and you can use the LLVM target for about everything.
Feb 23
prev sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Tuesday, 23 February 2021 at 10:24:26 UTC, Guillaume Piolat 
wrote:
 On Tuesday, 23 February 2021 at 10:14:50 UTC, Imperatorn wrote:
 Is this real life
C++ is quickly becoming that annoying legacy langage that no one likes. I would be incredibly wary starting a C++ project in 2021 ; in my exprience literally anything else is a more productive use of your time.
I agree that C++ is less productive than many other languages if you want to write the whole application in C++. It is quite ok for "engine parts", e.g. DSP etc. But I don't think this syntax thing is going to be a big issue. Editors will colour them and it static reflection is most likely something library authors will use.
 Even in the audio space which is 95% C++, the top product is 
 FLStudio, a product that was originally from Delphi.
I though most people use Abelton Live? Although Cubase is probably better for composing than both Abelton and FLStudio.
Feb 23
prev sibling next sibling parent reply Max Haughton <maxhaton gmail.com> writes:
On Monday, 22 February 2021 at 16:27:49 UTC, Andrei Alexandrescu 
wrote:
 Of possible interest:

 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2320r0.pdf
Exposing reflection information via CTFE seems like a much more workable solution - i.e. you just write regular D code. This is what Stefan's work effectively culminates in.
Feb 22
parent reply Paul Backus <snarwin gmail.com> writes:
On Monday, 22 February 2021 at 21:34:47 UTC, Max Haughton wrote:
 On Monday, 22 February 2021 at 16:27:49 UTC, Andrei 
 Alexandrescu wrote:
 Of possible interest:

 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2320r0.pdf
Exposing reflection information via CTFE seems like a much more workable solution - i.e. you just write regular D code. This is what Stefan's work effectively culminates in.
Or, in other words: procedural macros. (Shh, don't tell Walter.)
Feb 22
parent reply Max Haughton <maxhaton gmail.com> writes:
On Monday, 22 February 2021 at 21:59:41 UTC, Paul Backus wrote:
 On Monday, 22 February 2021 at 21:34:47 UTC, Max Haughton wrote:
 On Monday, 22 February 2021 at 16:27:49 UTC, Andrei 
 Alexandrescu wrote:
 Of possible interest:

 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2320r0.pdf
Exposing reflection information via CTFE seems like a much more workable solution - i.e. you just write regular D code. This is what Stefan's work effectively culminates in.
Or, in other words: procedural macros. (Shh, don't tell Walter.)
They aren't macros though - at least in my mind a macro is effectively a shortcut into the bowels of the compiler whereas in this case the compiler wouldn't care what you do with the information after it gives it to you since it's just regular D code.
Feb 22
parent reply Paul Backus <snarwin gmail.com> writes:
On Monday, 22 February 2021 at 23:32:39 UTC, Max Haughton wrote:
 On Monday, 22 February 2021 at 21:59:41 UTC, Paul Backus wrote:
 On Monday, 22 February 2021 at 21:34:47 UTC, Max Haughton 
 wrote:
 On Monday, 22 February 2021 at 16:27:49 UTC, Andrei 
 Alexandrescu wrote:
 Of possible interest:

 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2320r0.pdf
Exposing reflection information via CTFE seems like a much more workable solution - i.e. you just write regular D code. This is what Stefan's work effectively culminates in.
Or, in other words: procedural macros. (Shh, don't tell Walter.)
They aren't macros though - at least in my mind a macro is effectively a shortcut into the bowels of the compiler whereas in this case the compiler wouldn't care what you do with the information after it gives it to you since it's just regular D code.
Aren't macros in Lisp just regular Lisp code, too? :) As far as I'm concerned, a procedural macro is a function that 1. runs at compile time, 2. takes AST nodes as input, and 3. produces AST nodes as output. Type functions fit those criteria perfectly. Currently, they only work on a subset of AST nodes (types), so they're not a *complete* implementation of procedural macros, but generalize them enough and that's what you'll get.
Feb 22
next sibling parent reply Bruce Carneal <bcarneal gmail.com> writes:
On Monday, 22 February 2021 at 23:44:45 UTC, Paul Backus wrote:
 As far as I'm concerned, a procedural macro is a function that

 1. runs at compile time,
 2. takes AST nodes as input, and
 3. produces AST nodes as output.

 Type functions fit those criteria perfectly. Currently, they 
 only work on a subset of AST nodes (types), so they're not a 
 *complete* implementation of procedural macros, but generalize 
 them enough and that's what you'll get.
Yes. Given the collective experience that we've had with D's meta programming facilities it would be surprising if we could not imagine something significantly better at this point. By "better" I mostly mean facilities that would allow us to write code that is more readable, more composable, more testable, and more easily debuggable than an equally performant solution using current practice. I'm sympathetic to both type functions and MTVs (monadic type variables), CT type objects with actual memory footprint that function as compiler-guaranteed-safe cursors within the lazily realized dependent type space of a D program. Beyond those two I know that Stefan, and from the above and earlier writings I expect Paul, have thought about more powerful yet still tractable capabilities. It feels like we're within reach of a significant advance.
Feb 22
parent Max Haughton <maxhaton gmail.com> writes:
On Tuesday, 23 February 2021 at 05:17:04 UTC, Bruce Carneal wrote:
 On Monday, 22 February 2021 at 23:44:45 UTC, Paul Backus wrote:
 As far as I'm concerned, a procedural macro is a function that

 1. runs at compile time,
 2. takes AST nodes as input, and
 3. produces AST nodes as output.

 Type functions fit those criteria perfectly. Currently, they 
 only work on a subset of AST nodes (types), so they're not a 
 *complete* implementation of procedural macros, but generalize 
 them enough and that's what you'll get.
Yes. Given the collective experience that we've had with D's meta programming facilities it would be surprising if we could not imagine something significantly better at this point. By "better" I mostly mean facilities that would allow us to write code that is more readable, more composable, more testable, and more easily debuggable than an equally performant solution using current practice. I'm sympathetic to both type functions and MTVs (monadic type variables), CT type objects with actual memory footprint that function as compiler-guaranteed-safe cursors within the lazily realized dependent type space of a D program. Beyond those two I know that Stefan, and from the above and earlier writings I expect Paul, have thought about more powerful yet still tractable capabilities. It feels like we're within reach of a significant advance.
I have been talking to Stefan quite a lot about typefunctions recently, I'm reasonably convinced they are the most natural proposal to explore for D at the moment. One thing I have been toying with is calling them metafunctions instead since they clearly don't only consider types in the general case. Having a well defined type hierarchy to represent the AST in a manner seperate to the actual compiler has an added benefit of making it trivial to write clean programs that analyse D code without bringing in the entire compiler or going without sema.
Feb 22
prev sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 22 February 2021 at 23:44:45 UTC, Paul Backus wrote:
 On Monday, 22 February 2021 at 23:32:39 UTC, Max Haughton wrote:
 On Monday, 22 February 2021 at 21:59:41 UTC, Paul Backus wrote:
 On Monday, 22 February 2021 at 21:34:47 UTC, Max Haughton 
 wrote:
 On Monday, 22 February 2021 at 16:27:49 UTC, Andrei 
 Alexandrescu wrote:
 Of possible interest:

 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2320r0.pdf
Exposing reflection information via CTFE seems like a much more workable solution - i.e. you just write regular D code. This is what Stefan's work effectively culminates in.
Or, in other words: procedural macros. (Shh, don't tell Walter.)
They aren't macros though - at least in my mind a macro is effectively a shortcut into the bowels of the compiler whereas in this case the compiler wouldn't care what you do with the information after it gives it to you since it's just regular D code.
Aren't macros in Lisp just regular Lisp code, too? :) As far as I'm concerned, a procedural macro is a function that 1. runs at compile time, 2. takes AST nodes as input, and 3. produces AST nodes as output. Type functions fit those criteria perfectly. Currently, they only work on a subset of AST nodes (types), so they're not a *complete* implementation of procedural macros, but generalize them enough and that's what you'll get.
They don't exactly take AST nodes. They take a constrained (stabilized) datastructure which reflects a subset of the ast. At least conceptually. They don't create ast nodes, but regular values, which may be converted to ast-nodes in the case where they are statically available. (currently only types) My currently exploration leads me to a path where you can _extend_ open scopes within typefunction by using special functions which map provide a stable mapping to compiler intrinsics. such as __struct* __add_struct(string name, __symbol[] members); the __struct* will not be a proper struct type unless __type__ __add_to_scope(__scope Scope, __struct* s); has been called. At which point any modification of the __struct* is an error and will abort compilation. I am still searching for a solution which is more declarative, but I fear for heavy usage (10_000 new types and declarations added) scenarios the procedural imperative style is just more optimizable. I am always open to syntax suggestions, as I don't like the one I have :)
Feb 23
parent reply Paul Backus <snarwin gmail.com> writes:
On Tuesday, 23 February 2021 at 14:14:13 UTC, Stefan Koch wrote:
 On Monday, 22 February 2021 at 23:44:45 UTC, Paul Backus wrote:
 Aren't macros in Lisp just regular Lisp code, too? :)

 As far as I'm concerned, a procedural macro is a function that

 1. runs at compile time,
 2. takes AST nodes as input, and
 3. produces AST nodes as output.

 Type functions fit those criteria perfectly. Currently, they 
 only work on a subset of AST nodes (types), so they're not a 
 *complete* implementation of procedural macros, but generalize 
 them enough and that's what you'll get.
They don't exactly take AST nodes. They take a constrained (stabilized) datastructure which reflects a subset of the ast. At least conceptually. They don't create ast nodes, but regular values, which may be converted to ast-nodes in the case where they are statically available. (currently only types)
This is also how macros work in some Lisp implementations. For example, in Racket, the AST nodes (called "syntax objects") are converted to S-expressions by the function `syntax->datum`, the S-expressions are manipulated by CTFE, and the result is converted back into an AST node by `datum->syntax`. [1]
 My currently exploration leads me to a path where you can 
 _extend_ open scopes within typefunction by using special 
 functions which map provide a stable mapping to compiler 
 intrinsics.
 such as __struct* __add_struct(string name, __symbol[] members);
 the __struct* will not be a proper struct type unless
 __type__ __add_to_scope(__scope Scope, __struct* s);
 has been called.
I haven't fully thought this through, but would it be possible to overload `mixin` to do this? E.g., import core.ast: Struct, Type; // some type function Struct pair(Type t) { ... } alias PairOfInts = mixin(pair(int)); Conceptually, `mixin` already means "take this stuff and inject it into the current scope," and grammatically it's allowed to appear as both an expression and a type.
 At which point any modification of the __struct* is an error 
 and will abort compilation.
If __struct is only the "user-space" representation, and not the real AST node, I don't see why this is necessary, since modifying the __struct after mixing it in wouldn't have any affect on the actual AST.
 I am still searching for a solution which is more declarative, 
 but I fear for heavy usage (10_000 new types and declarations 
 added) scenarios the procedural imperative style is just more 
 optimizable.

 I am always open to syntax suggestions, as I don't like the one 
 I have :)
In Lisp, the declarative syntax is typically implemented as syntax sugar on top of the procedural syntax, using the `quasiquote` or `backquote` macro. [2][3] In D, one could imagine doing something similar with string mixins: generate the procedural code at compile time from a declarative DSL, then mix it in. [1] https://docs.racket-lang.org/guide/stx-obj.html [2] https://docs.racket-lang.org/guide/qq.html [3] https://www.ccis.northeastern.edu/home/futrelle/teaching/lisp/cltl/clm/node190.html#BACKQUOTE
Feb 23
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 23 February 2021 at 15:35:43 UTC, Paul Backus wrote:
 On Tuesday, 23 February 2021 at 14:14:13 UTC, Stefan Koch wrote:
 On Monday, 22 February 2021 at 23:44:45 UTC, Paul Backus wrote:
I haven't fully thought this through, but would it be possible to overload `mixin` to do this? E.g., import core.ast: Struct, Type; // some type function Struct pair(Type t) { ... } alias PairOfInts = mixin(pair(int)); Conceptually, `mixin` already means "take this stuff and inject it into the current scope," and grammatically it's allowed to appear as both an expression and a type.
Yes it would be possible to reuse mixin, but iirc mixins convert non-string expressions to string which would introduce ambiguity, if I special case it __struct, or __type__.
 At which point any modification of the __struct* is an error 
 and will abort compilation.
If __struct is only the "user-space" representation, and not the real AST node, I don't see why this is necessary, since modifying the __struct after mixing it in wouldn't have any affect on the actual AST.
The restriction is imposed to avoid confusing situations, due to the closeness of the mapping to real compiler entities you cannot finalize the same struct* twice and such things. Modifying a __struct after it has been finalized or 'mixed-in' is either useless or a bug.
Feb 23
parent reply Paul Backus <snarwin gmail.com> writes:
On Tuesday, 23 February 2021 at 15:49:14 UTC, Stefan Koch wrote:
 Yes it would be possible to reuse mixin, but iirc mixins 
 convert non-string expressions to string which would introduce 
 ambiguity, if I special case it __struct, or __type__.
My gut feeling is that it would not be too difficult to understand in practice, but yes, it would require special-casing.
 The restriction is imposed to avoid confusing situations, due 
 to the closeness of the mapping to real compiler entities you 
 cannot finalize the same struct* twice and such things.
 Modifying a __struct after it has been finalized or 'mixed-in' 
 is either useless or a bug.
Actually, now that I think of it, wouldn't the existing CTFE semantics prevent you from doing this anyway? enum __struct s = someTypeFunction(); auto modify(ref __struct s) { ... } enum result = modify(s); // Error: can't modify rvalue Or if you tried to do it inside a type function: auto someTypeFunction(...) { __struct s = ...; alias Before = mixin(s); // Error: can't read `s` at compile time modify(s); }
Feb 23
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 23 February 2021 at 16:10:26 UTC, Paul Backus wrote:
 On Tuesday, 23 February 2021 at 15:49:14 UTC, Stefan Koch wrote:
 [...]
My gut feeling is that it would not be too difficult to understand in practice, but yes, it would require special-casing.
 [...]
Actually, now that I think of it, wouldn't the existing CTFE semantics prevent you from doing this anyway? enum __struct s = someTypeFunction(); auto modify(ref __struct s) { ... } enum result = modify(s); // Error: can't modify rvalue Or if you tried to do it inside a type function: auto someTypeFunction(...) { __struct s = ...; alias Before = mixin(s); // Error: can't read `s` at compile time modify(s); }
... yes you have a point there. ... perhaps I am worrying too much.
Feb 23
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 22 February 2021 at 16:27:49 UTC, Andrei Alexandrescu 
wrote:
 Of possible interest:

 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2320r0.pdf
Hmm at a glance it looks close where I am at with typefunctions. The syntactic difference is that the unary ^ is an either an implicit conversion or an explicit cast. If they play their cards right and integrate their meta-info properly with constexpr it's going to beat D.
Feb 23
parent reply IGotD- <nise nise.com> writes:
On Tuesday, 23 February 2021 at 13:58:34 UTC, Stefan Koch wrote:
 If they play their cards right and integrate their meta-info 
 properly with constexpr it's going to beat D.
Can you explain further why it is going to beat D (I assume in terms of functionality, not aesthetics).
Feb 23
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 23 February 2021 at 15:04:46 UTC, IGotD- wrote:
 On Tuesday, 23 February 2021 at 13:58:34 UTC, Stefan Koch wrote:
 If they play their cards right and integrate their meta-info 
 properly with constexpr it's going to beat D.
Can you explain further why it is going to beat D (I assume in terms of functionality, not aesthetics).
Because the conversion to reflection values meta::info or whatever it was called happens inside the compiler. It's not shackled to templates. And therefore they can have very fast generation of these without inuring recursive evluate-semantic() function calls.
Feb 23
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 23 February 2021 at 15:13:18 UTC, Stefan Koch wrote:
 On Tuesday, 23 February 2021 at 15:04:46 UTC, IGotD- wrote:
 On Tuesday, 23 February 2021 at 13:58:34 UTC, Stefan Koch 
 wrote:
 If they play their cards right and integrate their meta-info 
 properly with constexpr it's going to beat D.
Can you explain further why it is going to beat D (I assume in terms of functionality, not aesthetics).
Because the conversion to reflection values meta::info or whatever it was called happens inside the compiler. It's not shackled to templates. And therefore they can have very fast generation of these without inuring recursive evluate-semantic() function calls.
Actually I take that back. I should have looked more closely. They have polymorphic expressions (changing the static type) which means they can only use those within polymorphic contexts ... which does shackle them to templates again.
Feb 23
prev sibling parent bitwise <bitwise.pvt gmail.com> writes:
On Monday, 22 February 2021 at 16:27:49 UTC, Andrei Alexandrescu 
wrote:
 Of possible interest:

 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2320r0.pdf
 void f(... [:range_of_types:] ...args)
This seems like it could provide a more intuitive intellisense display for things like std::make_shared, but without proper parameter names. Unless this provides a path to including parameter names, I don't see how it's better than <Args&&...>.
Feb 23