www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - DIP 1027---String Interpolation---Format Assessment

reply Mike Parker <aldacron gmail.com> writes:
DIP 1027, "String Interpolation", has been rejected. The decision 
was primarily influenced by the lack of consensus over the 
implementation and the syntax demonstrated in the two review 
threads. As the DIP author, Walter also rejected the suggestion 
to go with an implementation that resolves to a library template. 
He sees that as equivalent to AST macros, a feature which he has 
previously rejected.

https://github.com/dlang/DIPs/blob/4be15bd40381667c0ab1c0aef360d0daa4b8c82c/DIPs/rejected/DIP1027.md
Feb 23
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Sunday, 23 February 2020 at 16:22:46 UTC, Mike Parker wrote:
 The decision was primarily influenced by the lack of consensus 
 over the implementation and the syntax demonstrated in the two 
 review threads.
That's not true, we had consensus minus one - the community rallied around just one small tweak to the proposal. https://forum.dlang.org/post/r1emt0$2rpk$1 digitalmars.com "Now that is something I could use." https://forum.dlang.org/post/gvckofpbecslxwlleynu forum.dlang.org "This looks really good." https://forum.dlang.org/post/bawaaxojdtvsguadefpc forum.dlang.org "A different beast is Adam / Steven proposal ." https://forum.dlang.org/post/mailman.1029.1580903463.31109.digital ars-d puremagic.com "Adam/Steven's proposal is *so* much better," https://forum.dlang.org/post/vhhumlsdimntzxnkxfbn forum.dlang.org "hope we can get instead the proposal from Adam / Steven." https://forum.dlang.org/post/oswkdwpocuvcvdrxpiag forum.dlang.org "The implementation specified here is, more or less, what I would actually want out of string interpolation in D." Each of those posts are from different people. And if you go through the thread, there's even more. Sure, there's some people who would prefer % over $, or {} over (), or implicit over explicit toString/idup, but at the end of the day, we were all willing to put aside our remaining differences and accept that not 100% of people will be 100% happy to get a solution that 90% of people can be 90% happy with. There's just one person who didn't appear to even engage with the idea. Guess who.
 As the DIP author, Walter also rejected the suggestion to go 
 with an implementation that resolves to a library template. He 
 sees that as equivalent to AST macros, a feature which he has 
 previously rejected.
How is `foo!str, args...` a macro when `str, args...` is not? Obviously, to the objective reader, neither is a macro - both are simply argument lists. Just one is (potentially) type-safe and the other isn't.
Feb 23
next sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Sunday, 23 February 2020 at 18:57:55 UTC, Adam D. Ruppe wrote:
 As the DIP author, Walter also rejected the suggestion to go 
 with an implementation that resolves to a library template. He 
 sees that as equivalent to AST macros, a feature which he has 
 previously rejected.
How is `foo!str, args...` a macro when `str, args...` is not?
The one thing that MIGHT motivate this mistaken belief is if you think `foo` is defined in and looked up from random libraries. That is NOT the case. `foo` here is a placeholder name for a template defined once and only once, in druntime. It is a normal lowering, just like is done very successfully in several places throughout the D language.
Feb 23
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Sunday, 23 February 2020 at 18:57:55 UTC, Adam D. Ruppe wrote:
 On Sunday, 23 February 2020 at 16:22:46 UTC, Mike Parker wrote:
 The decision was primarily influenced by the lack of consensus 
 over the implementation and the syntax demonstrated in the two 
 review threads.
That's not true, we had consensus minus one - the community rallied around just one small tweak to the proposal.
I don't think that the proposed change was one small tweak. Even if it were, the DIP that just got rejected didn't have it. There's also the practical question of template instantiations and compile times even if the DIP that was being discussed were to be modified in the way suggested. Would you have preferred the DIP were accepted "as-was"?
Feb 24
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/24/20 4:38 AM, Atila Neves wrote:
 There's also the practical question of template instantiations and 
 compile times even if the DIP that was being discussed were to be 
 modified in the way suggested.
I want to actually explore this. There would be one template instantiation (probably) per usage. Not only that, but the function being called would also have a separate template instantiation per usage. This could potentially explode code to undesirable levels. However, we have some possible mitigation techniques. For example, writefln should validate the format specifiers against the parameters, and then call the real writefln with a format string literal. Second, I would suggest revisiting the idea of "temporary templates" that I talked about here: https://forum.dlang.org/post/qrjls8$13je$1 digitalmars.com. If the "shim" call is inlined, and the template removed, then you are simply left with the call to the standard writefln. This does not reduce compile-times. But I'm hoping we have an answer in the newCTFE update. We should really explore more ways to make generative code more palatable. This is D's strength. I will also note that we did something very similar with switch(string), where prior to this the compiler did all the "heavy lifting" of generating the code to convert strings into integers. Now it's done via a template which accepts all the strings as parameters. I don't think it was a bad change, and I'm sure it increased compile times and compiler memory usage. We shouldn't shy away from this type of stuff, but rather invest in making it better. Finally, I'd say that I would really prefer a tiny bit of compiler magic here. I would love for the template that defines the format string to devolve directly into a string literal if that's the usage -- potentially without actually instantiating the template. It can be a better option if we are super-concerned about template slowdown. -Steve
Feb 24
next sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Monday, 24 February 2020 at 15:41:16 UTC, Steven Schveighoffer 
wrote:
 On 2/24/20 4:38 AM, Atila Neves wrote:
 There's also the practical question of template instantiations 
 and compile times even if the DIP that was being discussed 
 were to be modified in the way suggested.
I want to actually explore this.
<snip> Thanks for the detailed write-up, there are a lot of good things here.
Feb 24
parent Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 24 February 2020 at 16:22:22 UTC, Atila Neves wrote:
 Thanks for the detailed write-up, there are a lot of good 
 things here.
We talked about this in the other thread, too. I doubt the one template here will be a big deal. My experience is templates get bad when we use them in loops or recursion. One line of user code can easily expand a dozen separate instantiations. Like in my jni thing, this one line: `DTypesToJni!(typeof(args))` expands to (args.length * 3) different templates (the impl is basically `alias it = AliasSeq!(convert!T[0], convert![T .. $])`). There's certainly the possibility it will be less, like foo(int, int) will be able to reuse that int one, but it is still work for the compiler to do. Various std.typecons type things and std.algorithm builders can easily grow hugely - a range pipeline has templates returning templates based on templated input types. But back to mine, since its intent is to be used in a __traits(allMembers) loop with different things each time, it can easily be hundreds of templates in two lines of user code. Now, contrast that to a tuple builder literal, which has no internal recursion at all. It is just one instantiation. Even in a loop we might use in code generation: static foreach(member; __traits(allMembers, T)) mixin(i"void $member() {}".idup); Well, each unique i"" only gives one template. That call to idup is a second one. So we are at two instantiations... no matter what the input is. It is certainly possible to write code that does a bad job with it. But it is harder than you think. Even doing what I so often tell people not to do: static foreach(member; __traits(allMembers, T)) mixin(i"$(__traits(getMember, T, member).stringof) $member() {}".idup); stuff like that with stringof... is still only two instantiations for any T. The string is still a literal and the types of the arguments are still the same. I think you'd have to actively try to make more than a constant two template instantiations out of the i"" string. Natural uses will tend to give just one for the string itself and one for the function you call with it (which btw is already a cost we're paying - writeln's variadic template list does the same as it is! So it is really an increase of one template per use, constant. No multiplicative growth like with typical recursive templates, or worse as we see in the really pathological cases.)
Feb 24
prev sibling next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Feb 24, 2020 at 10:41:16AM -0500, Steven Schveighoffer via
Digitalmars-d-announce wrote:
[...]
 I will also note that we did something very similar with
 switch(string), where prior to this the compiler did all the "heavy
 lifting" of generating the code to convert strings into integers. Now
 it's done via a template which accepts all the strings as parameters.
 I don't think it was a bad change, and I'm sure it increased compile
 times and compiler memory usage.
No kidding, at one point it was making Phobos uncompilable except for machines with huge gobs of RAM, because std.datetime had this gigantic switch statement over, like, *every* locale name that exists in the world, and it was causing a gigantic explosion of template bloat. I think the solution that was eventually adopted was to replace that switch with something else, which is not altogether a bad thing I suppose, but still, it does put to question the practicality of having a template generate code for a switch statement with >1000 string items.
 We shouldn't shy away from this type of stuff, but rather invest in
 making it better.
[...] Agreed. I think after newCTFE is finally merged, the next major item to tackle is how to improve the performance of template-heavy code. T -- Don't modify spaghetti code unless you can eat the consequences.
Feb 24
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/24/20 12:31 PM, H. S. Teoh wrote:
 On Mon, Feb 24, 2020 at 10:41:16AM -0500, Steven Schveighoffer via
Digitalmars-d-announce wrote:
 [...]
 I will also note that we did something very similar with
 switch(string), where prior to this the compiler did all the "heavy
 lifting" of generating the code to convert strings into integers. Now
 it's done via a template which accepts all the strings as parameters.
 I don't think it was a bad change, and I'm sure it increased compile
 times and compiler memory usage.
No kidding, at one point it was making Phobos uncompilable except for machines with huge gobs of RAM, because std.datetime had this gigantic switch statement over, like, *every* locale name that exists in the world, and it was causing a gigantic explosion of template bloat. I think the solution that was eventually adopted was to replace that switch with something else, which is not altogether a bad thing I suppose, but still, it does put to question the practicality of having a template generate code for a switch statement with >1000 string items.
The solution was to forward templates over a certain number of string elements to a single runtime function with an array created from the tuple list (which is a minimal wrapper) instead of a binary search via unrolled templates: https://github.com/dlang/druntime/blob/87bf88ba88d2926d7a470b5e126e0a5b1a1e266f/src/core/interna /switch_.d#L60-L73. It's unfortunate we even have to use CTFE to generate the immutable strings. I wouldn't imagine a much different approach for this proposal as well -- convert to something useful at runtime and then call a runtime-only implementation. As I said before, if we can make it so such "wrappers" are just forwarders, it would make the resulting code less bloated, and probably save on memory as well. -Steve
Feb 24
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
Having the compiler lower string interpolation to some hidden template is - AST 
macros. We're not doing AST macros.

Hidden user-defined semantics are not for D. Every language I'm familiar with 
that supports it wound up with users creating their own completely undocumented 
personal language that nobody else can use or has the remotest interest in
using.

Including me.

Exhibit A: https://www.boost.org/doc/libs/1_72_0/libs/preprocessor/doc/index.html

Everyone agrees, but says "but I know how to do AST macros in a sane way." 
They're wrong.

If someone sees an i"abc" in the source code, they should be able to look in
the 
language spec and know exactly what it does. They should not have to go
trolling 
through imports looking for the definition of some template they never heard of.
Feb 24
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 24 February 2020 at 19:35:16 UTC, Walter Bright wrote:
 Having the compiler lower string interpolation to some hidden 
 template is - AST macros. We're not doing AST macros.
This is untrue.
 Hidden user-defined semantics are not for D.
We are NOT calling for this. What, exactly, do you think we're proposing?
Feb 24
next sibling parent 12345swordy <alexanderheistermann gmail.com> writes:
On Monday, 24 February 2020 at 19:45:49 UTC, Adam D. Ruppe wrote:
 On Monday, 24 February 2020 at 19:35:16 UTC, Walter Bright 
 wrote:
 Having the compiler lower string interpolation to some hidden 
 template is - AST macros. We're not doing AST macros.
This is untrue.
 Hidden user-defined semantics are not for D.
We are NOT calling for this. What, exactly, do you think we're proposing?
I knew it. You needs an actual implementation here, to show walter what you are talking about. This reminds me of the whole mangle only discussion a while back. It took an actual PR for Walter to accept it instead of days arguing with him. Let get to work on this. -Alex
Feb 24
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/24/2020 11:45 AM, Adam D. Ruppe wrote:
 On Monday, 24 February 2020 at 19:35:16 UTC, Walter Bright wrote:
 Having the compiler lower string interpolation to some hidden template is - 
 AST macros. We're not doing AST macros.
This is untrue.
 Hidden user-defined semantics are not for D.
We are NOT calling for this.
https://digitalmars.com/d/archives/digitalmars/D/DIP_1027--String_Interpolation--Final_Review_Discussion_Thread_335065.html#N335132
 * put the format literal in a wrapper type.
 i"$foo %"
 is translated to the tuple:
 __d_format_literal!("%s %%"), foo
and also you wrote:
 if we can
 all agree to amend it to put in the template thing we can fix
 everything. Even with just the one string arg, we can do a lot
 with it...
and proposed a lowering to:
 i"your hex data is ${%02x}someByte"

 (_d_interpolated_string!("your hex data is ",
 _d_interpolated_format_spec("%02x"))(), someByte)
and proposed:
 i"$foo %"
 is translated to the tuple:
 __d_format_literal!("%s %%"), foo
 struct __d_format_literal(string fmt) {
          enum f = fmt;
          alias f this;
 }
Feb 24
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 24 February 2020 at 20:55:16 UTC, Walter Bright wrote:
 and proposed a lowering to:

 i"your hex data is ${%02x}someByte"

 (_d_interpolated_string!("your hex data is ",
 _d_interpolated_format_spec("%02x"))(), someByte)
Do you understand that `_d_interpolated_string` and `_d_interpolated_format_spec` are to be defined EXCLUSIVELY inside druntime? There's nothing user-defined about this.
Feb 24
parent reply aliak <something something.com> writes:
On Monday, 24 February 2020 at 21:23:43 UTC, Adam D. Ruppe wrote:
 On Monday, 24 February 2020 at 20:55:16 UTC, Walter Bright 
 wrote:
 and proposed a lowering to:

 i"your hex data is ${%02x}someByte"

 (_d_interpolated_string!("your hex data is ",
 _d_interpolated_format_spec("%02x"))(), someByte)
Do you understand that `_d_interpolated_string` and `_d_interpolated_format_spec` are to be defined EXCLUSIVELY inside druntime? There's nothing user-defined about this.
Does that mean no betterC support if it's in druntime? Actually, was it different with the pure tuple approach in DIP1027?
Feb 24
next sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/24/20 4:41 PM, aliak wrote:
 On Monday, 24 February 2020 at 21:23:43 UTC, Adam D. Ruppe wrote:
 On Monday, 24 February 2020 at 20:55:16 UTC, Walter Bright wrote:
 and proposed a lowering to:

 i"your hex data is ${%02x}someByte"

 (_d_interpolated_string!("your hex data is ",
 _d_interpolated_format_spec("%02x"))(), someByte)
Do you understand that `_d_interpolated_string` and `_d_interpolated_format_spec` are to be defined EXCLUSIVELY inside druntime? There's nothing user-defined about this.
Does that mean no betterC support if it's in druntime?
It will be supported in betterC, as long as you have object.d available. This does not require any D features such as TypeInfo, ModuleInfo, static constructors, GC, etc. -Steve
Feb 24
prev sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 24 February 2020 at 21:41:22 UTC, aliak wrote:
 Does that mean no betterC support if it's in druntime?
It would actually still work there because there is no actual code to link, just a compile-time definition to look up. For example, the name `string` itself is a druntime definition, but it still works in betterC because it does not link anything at run time. This is very similar. There is a known bug <https://issues.dlang.org/show_bug.cgi?id=19268> with betterC not allowing CTFE operations though, which imposes some unnecessary limitations, but we could and should fix that separately. However, even with that bug, this proposal's i"" strings still partially work in betterC. And it will all work (except for the idup function, for obvious reasons) once that bug is fixed.
Feb 24
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/24/20 2:35 PM, Walter Bright wrote:
 Having the compiler lower string interpolation to some hidden template 
 is - AST macros. We're not doing AST macros.
How can you possibly arrive at this conclusion? We lower to templates all the time. By this definition all operator overloading in D is AST macros. This means all the operations on associative arrays are AST macros. array dup and idup -- AST macro. destroy is an AST macro.
 
 Hidden user-defined semantics are not for D. Every language I'm familiar 
 with that supports it wound up with users creating their own completely 
 undocumented personal language that nobody else can use or has the 
 remotest interest in using.
Except that's not what's happening here. It's lowering to a template defined by the language, not the user. The user creates their own API (not language) by accepting such a template and unsurprisingly can react differently to different parameters to said function or template. If you have an example that proves me wrong, I'd love to see it. How can this feature do something that is "like AST macros"?
 If someone sees an i"abc" in the source code, they should be able to 
 look in the language spec and know exactly what it does. They should not 
 have to go trolling through imports looking for the definition of some 
 template they never heard of.
I don't know where to begin here. The very argument rebuts itself -- The reason you look in the spec is to find out about the type of an interpolated string. Then you've heard of it. Are we supposed to only implement features that people have heard of before reading the spec? How would they have heard of it if they don't read the spec? Why do they have to go "trolling through imports" (whatever that means -- this template will be in object.d), and read the implementation? I mean, people compare 2 objects all the time without reading the definition for object.opEquals, I'm sure they could manage understanding that there's something different about interpolated strings, and still use writefln with them. This whole response continues to imply that we do not understand each other's point of view. -Steve
Feb 24
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/24/2020 12:19 PM, Steven Schveighoffer wrote:
 How can you possibly arrive at this conclusion? We lower to templates all the 
 time.
The language is nowhere defined as lowering to specific templates. There are indeed some lowerings to templates in the implementation of the language, but those are NOT user defined templates, they are simply an implementation convenience.
 By this definition all operator overloading in D is AST macros.
Operator overloading is not supported for basic types. It's fundamentally different from what was proposed for interpolated strings.
 Except that's not what's happening here. It's lowering to a template defined
by 
 the language, not the user. The user creates their own API (not language) by 
 accepting such a template and unsurprisingly can react differently to
different 
 parameters to said function or template.
"creates their own API" => AST macros
 If you have an example that proves me wrong, I'd love to see it. How can this 
 feature do something that is "like AST macros"?
Because the proposed templates modify the behavior of basic types. From the discussion:
 i"$apples and ${%d}bananas"
 =>
 (__d_format_literal!(Format.init, " and ", Format("%d")),
apples, bananas)
Feb 24
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/24/20 4:10 PM, Walter Bright wrote:
 On 2/24/2020 12:19 PM, Steven Schveighoffer wrote:
 How can you possibly arrive at this conclusion? We lower to templates 
 all the time.
The language is nowhere defined as lowering to specific templates. There are indeed some lowerings to templates in the implementation of the language, but those are NOT user defined templates, they are simply an implementation convenience.
typeid expressions lower to a TypeInfo object. That is a specific named object in object.d. All we're saying is that it lowers to a specified type in object.d.
 By this definition all operator overloading in D is AST macros.
Operator overloading is not supported for basic types. It's fundamentally different from what was proposed for interpolated strings.
Not at all. In fact, operator overloading lowers to a specific template on that type, which allows the user to write whatever code they want. This is how lowering works. Our proposal is even more restrictive, as the template and its API are actually defined by the language.
 Except that's not what's happening here. It's lowering to a template 
 defined by the language, not the user. The user creates their own API 
 (not language) by accepting such a template and unsurprisingly can 
 react differently to different parameters to said function or template.
"creates their own API" => AST macros
No. It's overloading, not AST macros. How can an overload affect the AST other than to generate code specific to the type being accepted?
 If you have an example that proves me wrong, I'd love to see it. How 
 can this feature do something that is "like AST macros"?
Because the proposed templates modify the behavior of basic types. From the discussion: > i"$apples and ${%d}bananas" > => > (__d_format_literal!(Format.init, " and ", Format("%d")),  apples, bananas)
Where is the basic type modification here? i"..." is not a basic type, it's a string interpolation. It's never been in the language before. It doesn't translate into any basic type, it translates into a compile-time list of expressions. How is this an AST macro, but (string-literal, apples, bananas) not? --- Probably this is being misunderstood. Let me rephrase the proposal. Let's assume we agree on the format string grammar that was presented in the DIP: i"$apples and $%{d}bananas" will be transformed into: (interpolation_spec, apples, bananas) Where interpolation spec will be an instance of a language-defined type. It will have the following API: 1. spec.formatString!(s) where s is a string, will be a compile-time constant format string (guaranteed to end in a null character) where all the interpolated expressions are handled as follows: a. If an interpolation expression is preceded with {x}, where x is the string literal in the braces, the entire interpolation expression will be replaced with the string literal x. b. If an interpolation expression is not preceded with {...}, then the entire interpolation expression will be replaced with the string `s` passed into the formatString member template. 2. spec.formatStringZ() will return the equivalent of `formatString!("%s").ptr`. 3. If all interpolations are determined to have a {...} specifier, then the interpolation_spec can be implicitly converted to a null-terminated const(char)*. 4. There will be an isInterpolationSpec(T) template added to object.d which will return true if the parameter T is an interpolation spec type. --- That's it. The whole definition of the template and whatever is simply implementation details. -Steve
Feb 24
next sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/24/20 4:45 PM, Steven Schveighoffer wrote:
 i"$apples and $%{d}bananas" will be transformed into:
Of course that should have read i"$apples and ${%d}bananas" -Steve
Feb 24
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/24/2020 1:45 PM, Steven Schveighoffer wrote:
 Our proposal is even more restrictive, as the template and its API are
actually 
 defined by the language.
API is defined by the language, but not the behavior.
 No. It's overloading, not AST macros. How can an overload affect the AST other 
 than to generate code specific to the type being accepted?
"generate code" is how.
 How is this an AST macro, but (string-literal, apples, bananas) not?
Because its behavior is defined by a template, not the language spec.
 That's it. The whole definition of the template and whatever is simply 
 implementation details.
No, it isn't. It's leaving things up to templates like "formatString". There are other instances of the compiler lowering things to templates, but the behavior is still defined by the language, not the template. The templates were not strictly necessary, they were just a convenience. Operator overloading is for user defined types, not builtin types. --- The semantics of an interpolated string must be defined by the DIP, not deferred to some template. If the implementation of those defined language features is done by a template, that is an implementation choice, not part of the DIP or spec. My inference of the discussion about this in the n.g. was the templates would be used so users could customize the behavior to be whatever they wanted.
Feb 24
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/24/20 5:11 PM, Walter Bright wrote:
 On 2/24/2020 1:45 PM, Steven Schveighoffer wrote:
 Our proposal is even more restrictive, as the template and its API are 
 actually defined by the language.
API is defined by the language, but not the behavior.
The language doesn't define my function's API, I do. I get to decide the types of my parameters, which ones are accepted, which ones aren't, what the names of my functions are, etc. Maybe you mean ABI?
 
 
 No. It's overloading, not AST macros. How can an overload affect the 
 AST other than to generate code specific to the type being accepted?
"generate code" is how.
This generates code (and AST): int foo() { return 1; } You can see it from the disassembly that a return statement is "generated". That's what I meant.
 
 
 How is this an AST macro, but (string-literal, apples, bananas) not?
Because its behavior is defined by a template, not the language spec.
Not true. Both are defined by the language spec.
 
 That's it. The whole definition of the template and whatever is simply 
 implementation details.
No, it isn't. It's leaving things up to templates like "formatString".
The result of formatString is defined by the spec. It doesn't matter how the template actually generates it. The fact that there's a template involved has no bearing on what the result should be (which will be 100% defined as I said).
 There are other instances of the compiler lowering things to templates, 
 but the behavior is still defined by the language, not the template. The 
 templates were not strictly necessary, they were just a convenience.
As is the case here.
 
 The semantics of an interpolated string must be defined by the DIP, not 
 deferred to some template. If the implementation of those defined 
 language features is done by a template, that is an implementation 
 choice, not part of the DIP or spec.
This is exactly how the proposal is defined. There is no wiggle room for the template (or whatever we decide) to do anything but follow the spec exactly. You will never get alternative behavior from that type.
 My inference of the discussion about this in the n.g. was the templates 
 would be used so users could customize the behavior to be whatever they 
 wanted.
 
By accepting a different type from string. In other words, an overload. This means you can have code that treats an interpolated string differently than a string. Overloads based on literal types are not a new feature. You can have foo(double) and foo(float), and foo(1.0) will result in something different than foo(1.0f). -Steve
Feb 24
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/24/20 5:45 PM, Steven Schveighoffer wrote:
 My inference of the discussion about this in the n.g. was the 
 templates would be used so users could customize the behavior to be 
 whatever they wanted.
By accepting a different type from string. In other words, an overload.
I have a feeling from this comment that you are expecting that the compiler just literally lowers into the provided generated code, and leaves it up to the user to define the template. This is not the case. The template will be defined in object.d, and the compiler will NOT COMPILE code that cannot find that template in the module object.d. I would actually recommend that it lowers to: .object._d_interpolated_string!(...), apples, bananas To ensure that it cannot be intercepted. -Steve
Feb 24
parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/24/2020 4:07 PM, Steven Schveighoffer wrote:
 To ensure that it cannot be intercepted.
See my reply to H.S. Teoh which addresses this.
Feb 26
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/24/2020 2:45 PM, Steven Schveighoffer wrote:
 My inference of the discussion about this in the n.g. was the templates would 
 be used so users could customize the behavior to be whatever they wanted.
By accepting a different type from string. In other words, an overload. This means you can have code that treats an interpolated string differently than a string. Overloads based on literal types are not a new feature.
Were you proposing that an i"xxxx" be a different type? (DIP 1027 did not assign a type to it at all.) This would be radically different from DIP 1027, and a large increase in complexity (adding any new basic types is a big deal and a very intrusive change, and is tough to justify). This is different enough from DIP 1027 that it would merit a separate DIP. DIPs for the core language specify only behaviors, not implementations. Implementation possibilities can be included in an advisory manner only. Note that nowhere in the (massive and complicated) C++ core language specification is there any description of how the compiler should be implemented. Writing that an implementation must refer to specific templates implies that the behavior is customizable by the user via modifying those templates.
Feb 24
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/25/20 1:54 AM, Walter Bright wrote:
 On 2/24/2020 2:45 PM, Steven Schveighoffer wrote:
 My inference of the discussion about this in the n.g. was the 
 templates would be used so users could customize the behavior to be 
 whatever they wanted.
By accepting a different type from string. In other words, an overload. This means you can have code that treats an interpolated string differently than a string. Overloads based on literal types are not a new feature.
Were you proposing that an i"xxxx" be a different type? (DIP 1027 did not assign a type to it at all.)
No, I proposed that the first element of the tuple be specified as a new spec-defined type instead of a string. I would love to have there be a way to specify that it's a specialized type that is akin to string literals. I'm not qualified to do that, and I'm not sure it's something we want to do (this WOULD be extra complicated and require some compiler magic). I would note that it seems uncharacteristic for string enums to not be equivalent to string literals. If that were the case, this type WOULD be easy to set up effectively as a string literal. Having the type detailed in the spec and implemented in the library is a "low cost" solution. Effectively, even though it doesn't need to be specified how this works, the compiler-library interaction is done via simple lowering, and the existing compiler has all the tools to implement the new type already.
 This would be radically different from 
 DIP 1027, and a large increase in complexity (adding any new basic types 
 is a big deal and a very intrusive change, and is tough to justify).
It's not a basic type in terms of a type defined by the compiler, but a type defined by the spec and implemented in the library (with existing language mechanisms). It's not intrusive at all, the changes to your proposed DIP are minimal, as it's still lowering.
 
 This is different enough from DIP 1027 that it would merit a separate DIP.
Adam is working on this: https://github.com/dlang/DIPs/pull/186
 DIPs for the core language specify only behaviors, not implementations. 
 Implementation possibilities can be included in an advisory manner only. 
 Note that nowhere in the (massive and complicated) C++ core language 
 specification is there any description of how the compiler should be 
 implemented. Writing that an implementation must refer to specific 
 templates implies that the behavior is customizable by the user via 
 modifying those templates.
I understand, and I think we can reword the DIP proposal to reflect that. -Steve
Feb 25
parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/25/2020 7:04 AM, Steven Schveighoffer wrote:
 On 2/25/20 1:54 AM, Walter Bright wrote:
 Were you proposing that an i"xxxx" be a different type? (DIP 1027 did not 
 assign a type to it at all.)
No, I proposed that the first element of the tuple be specified as a new spec-defined type instead of a string. ... It's not a basic type in terms of a type defined by the compiler, but a type defined by the spec
I have no idea what this means. I'll wait until you and Adam come up with a finished DIP.
Feb 26
prev sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Feb 24, 2020 at 10:54:34PM -0800, Walter Bright via
Digitalmars-d-announce wrote:
[...]
 Writing that an implementation must refer to specific templates
 implies that the behavior is customizable by the user via modifying
 those templates.
I think this is where the misunderstanding is. The proposed template is defined by druntime, and is NOT customizable by the user. Unless, of course, they modify druntime, but then, if they're going to do that, they could just as easily modify the meaning of various internal symbols in druntime that dmd refers to, like change _adEq2 to do something completely alien. But the expectation is that users generally would not do this. Similarly, wrapping the interpolated tuple in a template is not customizable by the user. Rather, its sole purpose is to provide a distinct type from `string` that user can overload on, if they so wish. T -- The easy way is the wrong way, and the hard way is the stupid way. Pick one.
Feb 25
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/25/2020 9:44 AM, H. S. Teoh wrote:
 On Mon, Feb 24, 2020 at 10:54:34PM -0800, Walter Bright via
Digitalmars-d-announce wrote:
 [...]
 Writing that an implementation must refer to specific templates
 implies that the behavior is customizable by the user via modifying
 those templates.
I think this is where the misunderstanding is. The proposed template is defined by druntime, and is NOT customizable by the user.
Requiring the compiler to use a specific template that is not specified by the user has no place in a language specification (and therefore no place in a proposed language change). The specification does NOT specify how it should be implemented.
Feb 26
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/26/20 4:57 AM, Walter Bright wrote:
 On 2/25/2020 9:44 AM, H. S. Teoh wrote:
 On Mon, Feb 24, 2020 at 10:54:34PM -0800, Walter Bright via 
 Digitalmars-d-announce wrote:
 [...]
 Writing that an implementation must refer to specific templates
 implies that the behavior is customizable by the user via modifying
 those templates.
I think this is where the misunderstanding is. The proposed template is defined by druntime, and is NOT customizable by the user.
Requiring the compiler to use a specific template that is not specified by the user has no place in a language specification (and therefore no place in a proposed language change).
We can do it without specifying that it's a template or the name of that template. But this statement still belies your misunderstanding. D is full of compiler-defined types that are implemented via templates in the library "not specified by the user".
 The specification does NOT specify how it should be implemented.
No but logic does. It will have to be implemented this way, but we can certainly word the spec to "leave it up to the compiler/runtime implementation" to satisfy the requirements. -Steve
Feb 26
parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/26/2020 5:19 AM, Steven Schveighoffer wrote:
 We can do it without specifying that it's a template or the name of that
template.
That isn't what was proposed. I seriously suggest preparing a DIP. Bits and pieces spread out over multiple posts and multiple threads is not working.
 But this statement still belies your misunderstanding. D is full of 
 compiler-defined types that are implemented via templates in the library "not 
 specified by the user".
None of them are specified/required by the spec. Again, the spec specifies the semantics. Not the implementation method. (This would address my issue with AST macros.)
Feb 26
prev sibling next sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Wednesday, 26 February 2020 at 09:57:58 UTC, Walter Bright 
wrote:
 Requiring the compiler to use a specific template that is not 
 specified by the user has no place in a language specification 
 (and therefore no place in a proposed language change).
I think more naturally in D code rather than spec-ese (in other words, I find writing the implementation and usage examples to be a LOT easier than abstractly describing it), so it might help if you were to offer some specific suggestions. Generally speaking, it'd be great if we could see you as a co-contributor rather than an obstacle.
Feb 26
prev sibling parent Arine <arine123445128843 gmail.com> writes:
On Wednesday, 26 February 2020 at 09:57:58 UTC, Walter Bright 
wrote:
 On 2/25/2020 9:44 AM, H. S. Teoh wrote:
 On Mon, Feb 24, 2020 at 10:54:34PM -0800, Walter Bright via 
 Digitalmars-d-announce wrote:
 [...]
 Writing that an implementation must refer to specific 
 templates
 implies that the behavior is customizable by the user via 
 modifying
 those templates.
I think this is where the misunderstanding is. The proposed template is defined by druntime, and is NOT customizable by the user.
Requiring the compiler to use a specific template that is not specified by the user has no place in a language specification (and therefore no place in a proposed language change). The specification does NOT specify how it should be implemented.
That's just an implementation detail, the specification wouldn't require it. (This has been pointed out many times but you still seem to fall back on the same old argument after it's been debunked). It's how many D features already work and why druntime even exists. You're just arguing semantics at this point, not that it even matters. There's only one implementation of the D frontend + druntime. And the spec is in such a bad state, if someone wanted to create a new D compiler by simply following the spec, they wouldn't come out with anything that is remotely close to being compatible with today's D compilers.
Feb 26
prev sibling parent reply FeepingCreature <feepingcreature gmail.com> writes:
On Monday, 24 February 2020 at 22:11:08 UTC, Walter Bright wrote:
 The semantics of an interpolated string must be defined by the 
 DIP, not deferred to some template. If the implementation of 
 those defined language features is done by a template, that is 
 an implementation choice, not part of the DIP or spec.
I don't understand this. The behavior of formatting strings is *currently* deferred to a template (std.format and co). This lets us do important decisions at compiletime, like writing the format string to a file or a string buffer piecewise without allocating memory. Why are you trying to get rid of this flexibility? As I understand it, the point of a format string template is to keep exactly this kind of flexibility in exactly the same places it is currently available to users, while simultaneously improving syntax legibility. To emphasize this again: the gc format string would have *reduced* flexibility for 90% of our usecases (templated nogc/lowgc log library calls). As proposed, I don't see why I would have ever used it.
Feb 24
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/24/2020 8:24 PM, FeepingCreature wrote:
 The behavior of formatting strings is *currently* 
 deferred to a template (std.format and co). This lets us do important
decisions 
 at compiletime, like writing the format string to a file or a string buffer 
 piecewise without allocating memory. Why are you trying to get rid of this 
 flexibility?
std.format is a library function, with its behaviors entirely defined by the library. It is not a core language feature. There's nothing stopping writing a CTFE-able function for doing formatting. It doesn't need to be built in to the compiler.
 As I understand it, the point of a format string template is to keep exactly 
 this kind of flexibility in exactly the same places it is currently available
to 
 users, while simultaneously improving syntax legibility. To emphasize this 
 again: the  gc format string would have *reduced* flexibility for 90% of our 
 usecases (templated nogc/lowgc log library calls). As proposed, I don't see
why 
 I would have ever used it.
Steven's proposal is radically different from #DIP1027. It should be separately proposed as a coherent DIP, not in bits and pieces over several n.g. postings. All DIP1027 did was turn an istring into a tuple. That's it. The user can then do whatever they want with the tuple, including overloading a custom function based on the tuple arguments. DIP1027 did not actually do ANY formatting at all.
Feb 24
parent reply aliak <something something.com> writes:
On Tuesday, 25 February 2020 at 07:07:50 UTC, Walter Bright wrote:
 All DIP1027 did was turn an istring into a tuple. That's it. 
 The user can then do whatever they want with the tuple, 
 including overloading a custom function based on the tuple 
 arguments. DIP1027 did not actually do ANY formatting at all.
This may have already been answered in the other threads, but I was just wondering if anyone managed to propose a way to avoid this scenario with DIP1027? void f(string s, int i = 0); f(i"hello $a"); // silent unwanted bahviour. ?
Feb 25
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Tuesday, 25 February 2020 at 09:36:25 UTC, aliak wrote:
 This may have already been answered in the other threads, but I 
 was just wondering if anyone managed to propose a way to avoid 
 this scenario with DIP1027?
Yes, that is the key impetus of our amendment, which I also wrote up on a gist weeks ago.... and it is now on github too! https://github.com/dlang/DIPs/pull/186 By putting a new type around the generated format string, the f(i"") is a type mismatch error, so the user can decide what they want to do (I propose having the compiler suggest you call `.idup` on it to copy it into an ordinary GC string for new users to quickly get going with it.) That's all our amendment fundamentally does. Walter's: i"test $foo" -> "test %s", foo Our's: i"test $foo" -> wrapper!("test ", thing(null)), foo The `thing` there represents the interpolated item and user-defined format string. Since there isn't one, it is passed null. The `wrapper` there is a new type. Both `thing` and `wrapper` are defined by the language and their actual implementations live in druntime struct thing { string specifier; } struct wrapper(format_string_pieces...) {} From the language perspective, the rest is the same was Walter's proposal. Just with that wrapper type: foo(string, int) // type error, cannot convert wrapper to string to protect from that.
Feb 25
parent reply Aliak <something something.com> writes:
On Tuesday, 25 February 2020 at 13:04:41 UTC, Adam D. Ruppe wrote:
 On Tuesday, 25 February 2020 at 09:36:25 UTC, aliak wrote:
 [...]
Yes, that is the key impetus of our amendment, which I also wrote up on a gist weeks ago.... and it is now on github too! https://github.com/dlang/DIPs/pull/186 [...]
I should’ve been more specific 😬 I was wondering if the same could be achieved without a introducing a new aggregate type!
Feb 25
next sibling parent nullptr <null p.tr> writes:
On Tuesday, 25 February 2020 at 13:39:40 UTC, Aliak wrote:
 On Tuesday, 25 February 2020 at 13:04:41 UTC, Adam D. Ruppe 
 wrote:
 On Tuesday, 25 February 2020 at 09:36:25 UTC, aliak wrote:
 [...]
Yes, that is the key impetus of our amendment, which I also wrote up on a gist weeks ago.... and it is now on github too! https://github.com/dlang/DIPs/pull/186 [...]
I should’ve been more specific 😬 I was wondering if the same could be achieved without a introducing a new aggregate type!
In my opinion the only really viable solutions are either to lower to a function call to format (or similar) such that typeof(i"") == string or a new type such as Adam describes so that functions can add overloads for it (and specialized compiler errors for common misuse cases). Anything else and you risk users running into inexplicable errors because their string isn't a string.
Feb 25
prev sibling next sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/25/20 8:39 AM, Aliak wrote:
 On Tuesday, 25 February 2020 at 13:04:41 UTC, Adam D. Ruppe wrote:
 On Tuesday, 25 February 2020 at 09:36:25 UTC, aliak wrote:
 [...]
Yes, that is the key impetus of our amendment, which I also wrote up on a gist weeks ago.... and it is now on github too! https://github.com/dlang/DIPs/pull/186 [...]
I should’ve been more specific 😬 I was wondering if the same could be achieved without a introducing a new aggregate type!
I think there is not, unless you wanted to implement special rules in the compiler for string interpolations. And that would be akin to overloading (something like "you have to tag a function as acceptsinterpolation to have it work with interpolated strings"). Much simpler to just have a different type. -Steve
Feb 25
prev sibling next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Tuesday, 25 February 2020 at 13:39:40 UTC, Aliak wrote:
 I should’ve been more specific 😬 I was wondering if the same 
 could be achieved without a introducing a new aggregate type!
Well, compiler magic, possibly with more attributes. But that gets far messier than a simple struct, so we rejected it in the other thread. I just added a section to my dip text explaining this though: https://github.com/dlang/DIPs/pull/186/files#diff-2d3c5bf5c5d1f001279a15e3449b2338R325 basically once we address all the inevitable questions such a new thing would raise, we would essentially reinvent a struct with a new, awkward syntax anyway. So my view is a struct is a simple, understandable, and proven solution.
Feb 25
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/25/20 10:13 AM, Adam D. Ruppe wrote:
 On Tuesday, 25 February 2020 at 13:39:40 UTC, Aliak wrote:
 I should’ve been more specific 😬 I was wondering if the same could be 
 achieved without a introducing a new aggregate type!
Well, compiler magic, possibly with more attributes. But that gets far messier than a simple struct, so we rejected it in the other thread. I just added a section to my dip text explaining this though: https://github.com/dlang/DIPs/pull/186/files#diff-2d3c5bf5c5d1f001 79a15e3449b2338R325 basically once we address all the inevitable questions such a new thing would raise, we would essentially reinvent a struct with a new, awkward syntax anyway. So my view is a struct is a simple, understandable, and proven solution.
I think it's important that the compiler isn't involved with type construction, just lowering. It keeps things simple in the compiler. -Steve
Feb 25
prev sibling parent reply Arine <arine123445128843 gmail.com> writes:
On Tuesday, 25 February 2020 at 13:39:40 UTC, Aliak wrote:
 On Tuesday, 25 February 2020 at 13:04:41 UTC, Adam D. Ruppe 
 wrote:
 On Tuesday, 25 February 2020 at 09:36:25 UTC, aliak wrote:
 [...]
Yes, that is the key impetus of our amendment, which I also wrote up on a gist weeks ago.... and it is now on github too! https://github.com/dlang/DIPs/pull/186 [...]
I should’ve been more specific 😬 I was wondering if the same could be achieved without a introducing a new aggregate type!
``How to distinguish a different type? Use a different type. No, is there another simpler way to do that instead?`` Is this really the line of thinking going on here? It seems Walter has these arbitrary rules he's following which led up to the impractical and buggy solution that was DIP1027. Rules aren't meant to be followed blindly.
Feb 25
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/25/2020 8:04 AM, Arine wrote:
 Is this really the line of thinking going on here? It seems Walter has these 
 arbitrary rules he's following which led up to the impractical and buggy 
 solution that was DIP1027. Rules aren't meant to be followed blindly.
See what I mean about "no consensus emerging"? This thread amply illustrates that.
Feb 26
prev sibling parent reply aliak <something something.com> writes:
On Tuesday, 25 February 2020 at 16:04:59 UTC, Arine wrote:
 ``How to distinguish a different type? Use a different type. 
 No, is there another simpler way to do that instead?``

 Is this really the line of thinking going on here? It seems 
 Walter has these arbitrary rules he's following which led up to 
 the impractical and buggy solution that was DIP1027. Rules 
 aren't meant to be followed blindly.
Calm down. I didn't ask how to distinguish a type. I asked how to prevent a behaviour. There's a slight difference. If you misunderstood that then maybe I just didn't communicate that properly.
Feb 26
parent Arine <arine123445128843 gmail.com> writes:
On Wednesday, 26 February 2020 at 10:36:34 UTC, aliak wrote:
 On Tuesday, 25 February 2020 at 16:04:59 UTC, Arine wrote:
 ``How to distinguish a different type? Use a different type. 
 No, is there another simpler way to do that instead?``

 Is this really the line of thinking going on here? It seems 
 Walter has these arbitrary rules he's following which led up 
 to the impractical and buggy solution that was DIP1027. Rules 
 aren't meant to be followed blindly.
Calm down. I didn't ask how to distinguish a type. I asked how to prevent a behaviour. There's a slight difference. If you misunderstood that then maybe I just didn't communicate that properly.
I was paraphrasing to make a point, hence the quotes. Maybe I should have been more explicit, you are trying to reinvent the wheel cause you are following a rule blindly, when there is an obvious simple and easy solution. Fyi, telling someone to calm down is about the worst thing you can do to make someone calm down.
Feb 26
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/25/2020 1:36 AM, aliak wrote:
 This may have already been answered in the other threads, but I was just 
 wondering if anyone managed to propose a way to avoid this scenario with
DIP1027?
 
 void f(string s, int i = 0);
 f(i"hello $a"); // silent unwanted bahviour.
 
 ?
It is lowered to: f("hello %s", a); as designed. I don't know what's unwanted about it.
Feb 26
next sibling parent reply Juraj Mojzis <mojo frikulin.sk> writes:
On Wednesday, 26 February 2020 at 09:45:55 UTC, Walter Bright 
wrote:
 void f(string s, int i = 0);
 f(i"hello $a"); // silent unwanted bahviour.
 
 ?
It is lowered to: f("hello %s", a); as designed. I don't know what's unwanted about it.
void print_many(string msg, int cnt = 1) { foreach(i; 0 .. cnt) writeln(msg); } int apple_cnt = 4; print_many(i"I have $apple_cnt apples."); expected: I have 4 apples. real (unwanted) output: I have 4 apples. I have 4 apples. I have 4 apples. I have 4 apples.
Feb 26
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/26/2020 2:02 AM, Juraj Mojzis wrote:
 void print_many(string msg, int cnt = 1) {
      foreach(i; 0 .. cnt) writeln(msg);
 }
 
 int apple_cnt = 4;
 print_many(i"I have $apple_cnt apples.");
 
 expected: I have 4 apples.
Doing what you want would require a runtime GC allocated string.
Feb 26
parent Walter Bright <newshound2 digitalmars.com> writes:
You can also write:

     print(format(i"I have $apple_cnt apples"));

     void print(string s) { print_many(s); }

and get the behavior you're looking for.
Feb 26
prev sibling parent Juraj Mojzis <mojo frikulin.sk> writes:
On Wednesday, 26 February 2020 at 10:02:15 UTC, Juraj Mojzis 
wrote:
 On Wednesday, 26 February 2020 at 09:45:55 UTC, Walter Bright 
 wrote:

 void print_many(string msg, int cnt = 1) {
     foreach(i; 0 .. cnt) writeln(msg);
 }

 int apple_cnt = 4;
 print_many(i"I have $apple_cnt apples.");

 expected: I have 4 apples.
 real (unwanted) output:
 I have 4 apples.
 I have 4 apples.
 I have 4 apples.
 I have 4 apples.
Sorry, I made a mistake in my last reply. This is the (unwanted) output: I have %s apples. I have %s apples. I have %s apples. I have %s apples.
Feb 26
prev sibling next sibling parent aliak <something something.com> writes:
On Wednesday, 26 February 2020 at 09:45:55 UTC, Walter Bright 
wrote:
 On 2/25/2020 1:36 AM, aliak wrote:
 This may have already been answered in the other threads, but 
 I was just wondering if anyone managed to propose a way to 
 avoid this scenario with DIP1027?
 
 void f(string s, int i = 0);
 f(i"hello $a"); // silent unwanted bahviour.
 
 ?
It is lowered to: f("hello %s", a); as designed. I don't know what's unwanted about it.
It's unwanted because the lowering calls a function that was not intended with values that were unintended. You can take this article: https://dlang.org/articles/hijack.html and replace it with that lowering behaviour, and have the exact same story. Also unwanted because of what string interpolation means (ref wikipedia and all other languages).
Feb 26
prev sibling next sibling parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Wednesday, 26 February 2020 at 09:45:55 UTC, Walter Bright 
wrote:
 On 2/25/2020 1:36 AM, aliak wrote:
 This may have already been answered in the other threads, but 
 I was just wondering if anyone managed to propose a way to 
 avoid this scenario with DIP1027?
 
 void f(string s, int i = 0);
 f(i"hello $a"); // silent unwanted bahviour.
 
 ?
It is lowered to: f("hello %s", a); as designed. I don't know what's unwanted about it.
In all other languages with string interpolation that I'm familiar with, `a` is not passed to the `i` parameter. --- Code: public class Program { public static void f(string s, int i = 21) { System.Console.WriteLine($"s='{s}' | i='{i}'"); } public static void Main() { int a = 42; f($"hello {a}"); } } Output: s='hello 42' | i='21' Try it online: https://repl.it/repls/ZigzagStickyHardware --- JavaScript Code: function f(s, i = 21) { console.log(`s='${s}' | i='${i}'`); } const a = 42; f(`hello ${a}`); Output: s='hello 42' | i='21' Try it online: https://repl.it/repls/TechnologicalJointDisassembler --- Python Code: def f(s, i = 21): print(f"s='{s}' | i='{i}'") a = 42; f(f"hello {a}"); Output: s='hello 42' | i='21' Try it online: https://repl.it/repls/CrookedOutlandishInstructions --- Ruby: Code: def f(s, i = 21) end a = 42; Output: s='hello 42' | i='21' Try it online: https://repl.it/repls/MidnightblueProudAgent --- Kotlin Code: fun f(s: String, i: Int = 21) { println("s='$s' | i='$i'"); } val a = 42; f("hello $a"); Output: s='hello 42' | i='21' Try it online: https://repl.it/repls/ImpartialPepperyProducts --- Dart Code: void f(String s, [int i = 21]) { print("s='${s}' | i='${i}'"); } void main() { const a = 42; f("hello ${a}"); } Output: s='hello 42' | i='21' Try it online: https://repl.it/repls/AwareSqueakyProlog --- Swift Code: func f(_ s: String, _ i: Int = 21) { print("s='\(s)' | i='\(i)'"); } let a = 42 f("hello \(a)") Output: s='hello 42' | i='21' Try it online: https://repl.it/repls/MulticoloredCulturedRule --- Julia: Code: function f(s, i = 21) print("s='$s' | i='$i'") end a = 42 f("hello $a") Output: s='hello 42' | i='21' Try it online: https://repl.it/repls/StupidAcidicDatabases --- And so on...
Feb 26
next sibling parent reply FeepingCreature <feepingcreature gmail.com> writes:
On Wednesday, 26 February 2020 at 11:13:12 UTC, Petar Kirov 
[ZombineDev] wrote:
 On Wednesday, 26 February 2020 at 09:45:55 UTC, Walter Bright 
 wrote:
 It is lowered to:

   f("hello %s", a);

 as designed. I don't know what's unwanted about it.
In all other languages with string interpolation that I'm familiar with, `a` is not passed to the `i` parameter. --- [snip]
To be fair, none of those languages care about an extra string allocation. But to be fair again, Adam/Steven's proposal can avoid the memory allocation while also not passing i implicitly, at the cost of not matching to the string type. But to be thrice fair, Adam/Steven's proposal would work with the minor extension `f(i"hello $a".format)`/`f(i"hello $a".to!string)`, in keeping with the trend of GC use requiring explicit opt-in.
Feb 26
next sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Wednesday, 26 February 2020 at 12:18:07 UTC, FeepingCreature 
wrote:
 But to be thrice fair, Adam/Steven's proposal would work with 
 the minor extension `f(i"hello $a".format)`/`f(i"hello 
 $a".to!string)`, in keeping with the trend of GC use requiring 
 explicit opt-in.
Actually, thanks to the new type, the sample code in my dip offers an `idup` overload that works really easily for the users. void foo(string s) {} const(char)[] a; foo(a); // doesn't work, but you can .idup it foo(i"..."); // doesn't work, but you can .idup it too! foo(i"...".idup); // indicates you are OK with the GC copy by asking for one so still really easy to use while being in line with D's existing features. No GC if you don't want it, easy GC if you do.
Feb 26
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/26/2020 4:18 AM, FeepingCreature wrote:
 But to be thrice fair, Adam/Steven's proposal would work with the minor 
 extension [...]
So would DIP1027.
Feb 26
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 27 February 2020 at 00:21:36 UTC, Walter Bright 
wrote:
 So would DIP1027.
We know. It is *almost* there, the format string idea is a good one. But DIP1027 had a fatal flaw: it made type safety impossible. One small change - wrapping the format string in a new type while keeping the rest of the tuple exactly the same - fixes that fatal flaw. And as a cool bonus, the simple fact that there's a type around it - with no special rules; it is just a struct to identify that it is a format string - unlocks more potential with it too. I tried to put it as a dip back on Feb 4 (and it is now on github: https://github.com/dlang/DIPs/pull/186 ) but I don't know how to word this in spec language. I'm a D programmer, not a spec author :(
Feb 26
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/26/2020 4:46 PM, Adam D. Ruppe wrote:
 But DIP1027 had a fatal flaw: it made type safety impossible.
I don't see how that is true.
Feb 26
parent reply FeepingCreature <feepingcreature gmail.com> writes:
On Thursday, 27 February 2020 at 03:50:35 UTC, Walter Bright 
wrote:
 On 2/26/2020 4:46 PM, Adam D. Ruppe wrote:
 But DIP1027 had a fatal flaw: it made type safety impossible.
I don't see how that is true.
Because it turned a format string into a list of built-in types indistinguishable from a set of manual parameters. You cannot in principle tell the difference between "test $i" and ("test %s", i) - you cannot write a function that takes a string and then *any other type* and is also protected from being accidentally called with a format string. Features should be as simple as they can be, but no simpler. - The point of simplicity is predictability. DIP1027 is so simple that it crosses back into surprising.
Feb 26
parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/26/2020 10:38 PM, FeepingCreature wrote:
 On Thursday, 27 February 2020 at 03:50:35 UTC, Walter Bright wrote:
 On 2/26/2020 4:46 PM, Adam D. Ruppe wrote:
 But DIP1027 had a fatal flaw: it made type safety impossible.
I don't see how that is true.
Because it turned a format string into a list of built-in types indistinguishable from a set of manual parameters. You cannot in principle tell the difference between "test $i" and ("test %s", i) - you cannot write a function that takes a string and then *any other type* and is also protected from being accidentally called with a format string.
That isn't a type safety error.
Feb 27
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/26/2020 3:13 AM, Petar Kirov [ZombineDev] wrote:
 In all other languages with string interpolation that I'm familiar with, `a`
is 
 not passed to the `i` parameter.
All rely on a garbage collected string being generated as an intermediate variable.
Feb 26
next sibling parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Thursday, 27 February 2020 at 00:20:27 UTC, Walter Bright 
wrote:
 On 2/26/2020 3:13 AM, Petar Kirov [ZombineDev] wrote:
 In all other languages with string interpolation that I'm 
 familiar with, `a` is not passed to the `i` parameter.
All rely on a garbage collected string being generated as an intermediate variable.
I'm well aware that allocation is inevitable if we want this behavior. My argument is that this behavior is so ubiquitous that not following it would be surprising to much more people, than if D didn't follow C's Usual Arithmetic Conversions rules. For example, Rust not following those conversion rules is considered a good thing, while if D decided to be different than all other languages w.r.t. string interpolation, most newcomers would consider this a bad thing and not elegant and innovative as we are aiming for. I agree with Adam, Steven and others that string interpolation expression should yield a distinct type and not a tuple. By doing this we would be able to overload functions so they could accept both strings (which would cause GC allocation when the argument is a interpolated string), and the new distinct type, in which case the allocation could be avoided.
Feb 27
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/27/2020 12:27 AM, Petar Kirov [ZombineDev] wrote:
 I'm well aware that allocation is inevitable if we want this behavior. My 
 argument is that this behavior is so ubiquitous that not following it would be 
 surprising to much more people, than if D didn't follow C's Usual Arithmetic 
 Conversions rules. For example, Rust not following those conversion rules is 
 considered a good thing,
Rust does not follow C syntax at all, so nobody will reasonably expect it to have C semantics. D does follow it, it's a feature, so people will have expectations.
 while if D decided to be different than all other 
 languages w.r.t. string interpolation,
You can make it behave like all those other languages simply with: f(format("hello $a")); and there it is. But having it generate a GC allocated string is not so easy to unwind, i.e. it'll be useless with printf and generate unacceptable garbage with writefln. The extra string will always make it slow. Essentially, it'll be crippled. Making D behave like a scripting language will yield scripting performance. D is a language built up from simple, orthogonal parts (or at least that is a goal). A language built from larger indivisible parts is much, much less user-adaptable. An example of this is the built-in associative array, which has a series of fairly intractable problems as a result. Another example is the built-in complex type in D, which turned out to be a bad idea - a much better one is building it as a library type.
Feb 27
next sibling parent Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Thursday, 27 February 2020 at 09:30:30 UTC, Walter Bright 
wrote:
 On 2/27/2020 12:27 AM, Petar Kirov [ZombineDev] wrote:
 I'm well aware that allocation is inevitable if we want this 
 behavior. My argument is that this behavior is so ubiquitous 
 that not following it would be surprising to much more people, 
 than if D didn't follow C's Usual Arithmetic Conversions 
 rules. For example, Rust not following those conversion rules 
 is considered a good thing,
Rust does not follow C syntax at all, so nobody will reasonably expect it to have C semantics. D does follow it, it's a feature, so people will have expectations.
As shown, string interpolation in other languages (not only 'stript', as you wrote) has a so well established way of performing it that everybody will reasonably expect D to behave the same. Said that, better having no string interpolation at all in D, if the introduced feature is not at least comparable with the solution that the others out there are using: no surprise behaviour, please!
 and there it is. But having it generate a GC allocated string 
 is not so easy to unwind, i.e. it'll be useless with printf and 
 generate unacceptable garbage with writefln. The extra string 
 will always make it slow. Essentially, it'll be crippled. 
 Making D behave like a scripting language will yield scripting 
 performance.
It seems that the other compiled languages doing it in that way have raised no concerns at all on the above matters
Feb 27
prev sibling next sibling parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Thursday, 27 February 2020 at 09:30:30 UTC, Walter Bright 
wrote:
 On 2/27/2020 12:27 AM, Petar Kirov [ZombineDev] wrote:
 I'm well aware that allocation is inevitable if we want this 
 behavior. My argument is that this behavior is so ubiquitous 
 that not following it would be surprising to much more people, 
 than if D didn't follow C's Usual Arithmetic Conversions 
 rules. For example, Rust not following those conversion rules 
 is considered a good thing,
Rust does not follow C syntax at all, so nobody will reasonably expect it to have C semantics. D does follow it, it's a feature, so people will have expectations.
I'm not sure where exactly you draw the line, but I would say doesn't import some of the broken C semantics like implicit narrowing conversions (luckily, neither does D) and allowing mixed sign comparisons (the oldest open D issue :( [0]). My point is that if D didn't follow the usual arithmetic conversions, much fewer newcomers would even notice compared to extremely large backlash that we may get if go with the string interpolation -> raw tuple approach. [0]: https://issues.dlang.org/show_bug.cgi?id=259
 while if D decided to be different than all other languages 
 w.r.t. string interpolation,
You can make it behave like all those other languages simply with: f(format("hello $a")); and there it is. But having it generate a GC allocated string is not so easy to unwind, i.e. it'll be useless with printf and generate unacceptable garbage with writefln. The extra string will always make it slow. Essentially, it'll be crippled. Making D behave like a scripting language will yield scripting performance.
I know, I know. Though I think you misunderstood. There several ways to make printf work with zero allocations. For example: 1. Have a simple pragma(inline, true) wrapper function that will convert the distinct type to printf-style args. This wrapper function can even be named printf as it would work by virtue of function overloading. This is O(1) additional code that once written no one will need to bother with. 2. Have the new type implicitly convert to printf-style args. I think this is what Adam is proposing. While nice to have, I don't think it's necessary. As for std.stdio.write(f)(ln), there's no reason why any garbage would need to be created, as again, a simple overload that accepts the distinct type will be able to handle it just like it would (performance-wise) with DIP1027. However, with DIP1027, only writef and writefln can be used, while write and writeln will produce wrong results (wrong according to people that have used string interpolation in other languages).
 D is a language built up from simple, orthogonal parts (or at 
 least that is a goal). A language built from larger indivisible 
 parts is much, much less user-adaptable.
I appreciate the sentiment. Having orthogonal features in D is important to me as well. However, I think DIP1027 falls short here because it produces a single format string and that way loses all of the structure. This is ok for printf, but not for third-party libraries and even our own std.format, as with a distinct type we won't need to parse the whole format string at all, just the individual format specifiers. In other words, a distinct type would make nothrow std.format a much more tractable problem (at least for some cases).
 An example of this is the built-in associative array, which has 
 a series of
 fairly intractable problems as a result. Another example is the 
 built-in
 complex type in D, which turned out to be a bad idea - a much 
 better one is
 building it as a library type.
AFAIR, most of the problems with D's built-in AAs are that they have an extern (C) interface that relies on typeinfo. If they are fully converted to templated library types, the situation would be much better. IIRC, one of the blocking issues was that D didn't have autovivification [1] operators, so a library type wouldn't be a complete replacement without additional help from the compiler. So in conclusion, having a distinct library-defined type (in druntime) seems the best way to go to me, as it's more flexible than raw tuples, could allow easy GC-backed conversion to string for script-like code and would offer a superset of the functionality that DIP1027 would offer, while still allowing easy (although not 100% direct) calls to printf. [1]: https://en.wikipedia.org/wiki/Autovivification
Feb 27
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 27 February 2020 at 14:32:29 UTC, Petar Kirov 
[ZombineDev] wrote:
 2. Have the new type implicitly convert to printf-style args. I 
 think this is what Adam is proposing. While nice to have, I 
 don't think it's necessary.
You can read my document for more detail https://github.com/dlang/DIPs/pull/186 But basically writefln(i"hi $name, you are visitor ${%2d}(count)"); gets turned into: writefln( // the format string is represented by this type new_type!("hi ", spec(null), ", you are visitor ", spec("%2d"))(), // then the referenced arguments are passed as a tuple name, count ) So very, very, very similar to Walter's proposal, just instead of the compiler generating the format string as a plain string, the format string is represented by a new type, defined by the spec and implemented by druntime. As a result, no more guess work - it is clear that this is meant to be interpreted as a format string. It is clear which parts are placeholders/specifiers for which arguments.
Feb 27
parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Thursday, 27 February 2020 at 14:58:20 UTC, Adam D. Ruppe 
wrote:
 On Thursday, 27 February 2020 at 14:32:29 UTC, Petar Kirov 
 [ZombineDev] wrote:
 2. Have the new type implicitly convert to printf-style args. 
 I think this is what Adam is proposing. While nice to have, I 
 don't think it's necessary.
You can read my document for more detail https://github.com/dlang/DIPs/pull/186 But basically writefln(i"hi $name, you are visitor ${%2d}(count)"); gets turned into: writefln( // the format string is represented by this type new_type!("hi ", spec(null), ", you are visitor ", spec("%2d"))(), // then the referenced arguments are passed as a tuple name, count ) So very, very, very similar to Walter's proposal, just instead of the compiler generating the format string as a plain string, the format string is represented by a new type, defined by the spec and implemented by druntime. As a result, no more guess work - it is clear that this is meant to be interpreted as a format string. It is clear which parts are placeholders/specifiers for which arguments.
Perhaps my assumptions were based on an old version of your proposal. What I want is for: auto s = i"hi $name, you are visitor ${%2d}(count)"; to lower to: auto s = new_type!( "hi ", spec(null), ", you are visitor ", spec("%2d") )(name, count); I.e. the referenced arguments are passed to the constructor of new_type. That way new_type can offer implicit conversion to string, while support for zero-allocation printf, write, writeln, writef, writefln and so on can be done via function overloading.
Feb 27
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 27 February 2020 at 17:41:12 UTC, Petar Kirov 
[ZombineDev] wrote:
     auto s = new_type!(
         "hi ", spec(null), ", you are visitor ", spec("%2d")
     )(name, count);

 I.e. the referenced arguments are passed to the constructor of 
 new_type.
Right, that actually is what my old proposal was (and I fought for it on the first few pages of the last thread), and this is it behind because: 1. It doesn't work very nicely in templates: int a; foo!i"test $a"; // error, a cannot be evaluated at compile time Even if `foo` otherwise could handle it (e.g. an `alias` parameter), passing the data to an intermediate object prohibits this. I personally think that is worth losing in exchange for the other benefits it brings, which you correctly identified at the end of your message, but there's more we lose too: 2. It brings complication with features like `ref` (cannot have a ref struct member so once we pass it, it is gone...) and potentially `scope`, etc., or even UDAs are lost with an intermediary (though UDAs affecting printing might be weird af, the door is totally closed with it and possibly open without). But bottom line: this means it wouldn't work with non-copyable types. --- // in theory printable, has a toString method struct NoCopy { disable this(this); void toString(dg) {...} } NoCopy nc; // but this works with a tuple... not with an intermediary, unless // the intermediary allocates a string upon construction, of course, but // that gets us into the hidden GC worries again print(i"i can print it without copying it $nc"); --- `print` there might take its arguments by `auto ref` and work just fine, but an intermediate object wouldn't have that option. (ref also means you could do an interpolated `readf` string too, though I kinda think that is a little weird anyway personally, but some people in the other thread did have uses where it could be useful, so if we can avoid breaking it, we should. I think the non-copy print is more compelling though.) 3. As Walter pointed out with GC too, it is easy to go from tuple to object (just do `.whatever` to pass it to an object constructor/helper function), but object to tuple is not so simple. (of course there is `.tupleof`, but the details lost through the intermediate object can never be recovered.) So if there's other use cases we miss, the tuple has fewer potential regrets. 4. Just want to point out that some people feel strongly that the implicit conversion to fully interpolated string is altogether a bad idea because it would be a hidden GC allocation. I don't agree - I don't think it is that hidden since `alias this` only triggers if you name `string` and if you want to avoid GC allocs there's plenty of other ways to do it - but still a few people said it last thread so I wanna mention it here too. If you like, we could copy/paste this into my DIP too, might be useful for future reference. I didn't spend much time on this since I was focused on just advocating for my amendment to Walter's and assumed the point was moot anyway (Walter's proposal shares these strengths since it also uses the tuple at its core). But yeah, it is a good idea... just I think my new DIP as written, with a typed format string and a tuple of arguments, is a better idea since it works with more of D's unique features in conversion.
Feb 27
parent Aliak <something something.com> writes:
On Thursday, 27 February 2020 at 18:19:03 UTC, Adam D. Ruppe 
wrote:
 On Thursday, 27 February 2020 at 17:41:12 UTC, Petar Kirov 
 [ZombineDev] wrote:
 [...]
Right, that actually is what my old proposal was (and I fought for it on the first few pages of the last thread), and this is it behind because: [...]
Should this maybe be called tuple interpolation instead of string interpolation? It is unique to D it seems, this feature that is. And then it might help with the “wait why can I not assign an interpolated string to a string”?? And on top maybe change the prefix to t instead of i?
Feb 27
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/27/20 9:32 AM, Petar Kirov [ZombineDev] wrote:
 An example of this is the built-in associative array, which has a 
 series of
 fairly intractable problems as a result. Another example is the built-in
 complex type in D, which turned out to be a bad idea - a much better 
 one is
 building it as a library type.
AFAIR, most of the problems with D's built-in AAs are that they have an extern (C) interface that relies on typeinfo. If they are fully converted to templated library types, the situation would be much better. IIRC, one of the blocking issues was that D didn't have autovivification [1] operators, so a library type wouldn't be a complete replacement without additional help from the compiler.
We're going very off topic here, but I wanted to address this. Large hidden invisible types are not the problem (look at normal dynamic arrays, the large hidden type built into the runtime is a huge success I think). The problem is that the compiler gives special features to such types. In the case of AA, the ONLY missing piece that cannot be implemented by user types is this: int[string][string] aalist; aalist["hello"]["world"] = 5; In essence, the compiler knows how to make this work. The operators available do not allow this expression to be captured properly by user code (i.e. we don't have opIndexIndexAssign or opIndexIndexIndexAssign, nor does that scale). I believe the last person to try and implement a full template type that could replace AAs is H. S. Teoh. He would have a better explanation (and possibly contradict me, I don't know). Other than that, we've ripped out all other "magic" into templates in the language. If we could get that one feature (not sure how to do this in a scalable way), I think we have a full library type that can be further factored out of the compiler. We might even be able to avoid using TypeInfo, and make CTFE AAs compatible with runtime ones. -Steve
Feb 27
next sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Thursday, 27 February 2020 at 15:11:07 UTC, Steven 
Schveighoffer wrote:
 [snip]

 We're going very off topic here, but I wanted to address this.

 Large hidden invisible types are not the problem (look at 
 normal dynamic arrays, the large hidden type built into the 
 runtime is a huge success I think). The problem is that the 
 compiler gives special features to such types.

 In the case of AA, the ONLY missing piece that cannot be 
 implemented by user types is this:

 int[string][string] aalist;

 aalist["hello"]["world"] = 5;

 [snip]
Thanks for writing that. I spent a few minutes reading about autovivification and was a little unsure of what the problem was as D's operator overloading is pretty flexible. However, I don't think I've ever used or seen used multi-dimensional associative arrays. It looks as if you cannot make use of aalist["hello", "world"] and have to use it like aalist["hello"]["world"].
Feb 27
prev sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Feb 27, 2020 at 10:11:07AM -0500, Steven Schveighoffer via
Digitalmars-d-announce wrote:
[...]
 Large hidden invisible types are not the problem (look at normal
 dynamic arrays, the large hidden type built into the runtime is a huge
 success I think). The problem is that the compiler gives special
 features to such types.
 
 In the case of AA, the ONLY missing piece that cannot be implemented
 by user types is this:
 
 int[string][string] aalist;
 
 aalist["hello"]["world"] = 5;
 
 In essence, the compiler knows how to make this work. The operators
 available do not allow this expression to be captured properly by user
 code (i.e. we don't have opIndexIndexAssign or
 opIndexIndexIndexAssign, nor does that scale).
It's not that hard: https://issues.dlang.org/show_bug.cgi?id=7753 Somebody just has to do it, that's all.
 I believe the last person to try and implement a full template type
 that could replace AAs is H. S. Teoh. He would have a better
 explanation (and possibly contradict me, I don't know).
Actually, I may have been the *first* one to try to do this, but I don't think I was the last. Over the years various pieces of the AA implementation has been templatized and moved to druntime, though the core implementation is still in aaA.d. I think, on the basis of this other work, that we're in a far better position now to fully move AA's into a template implementation. I haven't been keeping track, though, so I don't know what issues might remain that hinder this migration.
 Other than that, we've ripped out all other "magic" into templates in
 the language. If we could get that one feature (not sure how to do
 this in a scalable way), I think we have a full library type that can
 be further factored out of the compiler. We might even be able to
 avoid using TypeInfo, and make CTFE AAs compatible with runtime ones.
[...] Yeah, most of the work on removing AA magic from the compiler has been done by someone else, IIRC Martin Nowak among them, and probably others. Making CTFE AAs usable at runtime is somewhat of a different beast, though. The main problem is that you need to be able to instantiate the binary representation of a runtime AA (the main hash table, and each of the buckets) at compile-time, and do so in a way that the compiler can turn into data in the data segment of the executable. Regardless of what the CTFE representation is, it needs an explicit transformation step to turn it into something the runtime code can decipher. Basically, you have to create an .init value for the final object that doesn't require calling a runtime memory allocation function, but nonetheless still points to legit instances of AA buckets and their contents. This cannot be directly cast from the CTFE AA, because CTFE AA buckets won't have legit runtime addresses that can be assigned to the hash table's pointers. I think this *might* be possible to do via a string mixin that creates explicit variables for each AA bucket then the main hash table by taking their addresses. Of course, some hackish casts will probably be required to make it all work with the current runtime AA implementation. T -- Why can't you just be a nonconformist like everyone else? -- YHL
Feb 27
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/27/20 1:42 PM, H. S. Teoh wrote:
 Making CTFE AAs usable at runtime is somewhat of a different beast,
 though.  The main problem is that you need to be able to instantiate the
 binary representation of a runtime AA (the main hash table, and each of
 the buckets) at compile-time, and do so in a way that the compiler can
 turn into data in the data segment of the executable.  Regardless of
 what the CTFE representation is, it needs an explicit transformation
 step to turn it into something the runtime code can decipher.
I think this is not too difficult. This works, and it's not much different: static immutable rbt = new RedBlackTree!int(1,2, 3, 4); In other words, I have pretty much faith that if the AA becomes a template, then whatever call is made for ["hello": 1, "world": 2] can be callable at compile time, and generate a compatible runtime AA. The CTFE AA can be whatever CTFE likes, just when it moves to runtime land, it gets translated to an AA literal. -Steve
Feb 27
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Feb 27, 2020 at 02:20:14PM -0500, Steven Schveighoffer via
Digitalmars-d-announce wrote:
 On 2/27/20 1:42 PM, H. S. Teoh wrote:
 Making CTFE AAs usable at runtime is somewhat of a different beast,
 though.  The main problem is that you need to be able to instantiate the
 binary representation of a runtime AA (the main hash table, and each
 of the buckets) at compile-time, and do so in a way that the
 compiler can turn into data in the data segment of the executable.
 Regardless of what the CTFE representation is, it needs an explicit
 transformation step to turn it into something the runtime code can
 decipher.
I think this is not too difficult. This works, and it's not much different: static immutable rbt = new RedBlackTree!int(1,2, 3, 4); In other words, I have pretty much faith that if the AA becomes a template, then whatever call is made for ["hello": 1, "world": 2] can be callable at compile time, and generate a compatible runtime AA. The CTFE AA can be whatever CTFE likes, just when it moves to runtime land, it gets translated to an AA literal.
[...] Interesting. So it looks like the problem is "merely" that the AA implementation is opaque to the compiler (the current implementation is PIMPL), so it doesn't know how to turn it into static data. What if we explicitly cast it into the implementation type, or into a parallel declaration of the implementation type(s), then forcibly cast it back to V[K]? Something like this: struct AANode { /* implementation here */ } static immutable AANode* aaImpl = /* CTFE AA here */; static immutable aa = cast(V[K]) aaImpl; Not 100% sure about that last line, you probably have to force it into void* or something at some point, I dunno. T -- MASM = Mana Ada Sistem, Man!
Feb 27
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/27/2020 6:32 AM, Petar Kirov [ZombineDev] wrote:

 C's syntax about as much as D does.
DasBetterC is very viable. Hindsight being 20/20, if I was doing a do-over with D I might even make it able to compile unmodified C code.
 Yet it doesn't import some of the broken C 
 semantics like implicit narrowing conversions (luckily, neither does D) and 
 allowing mixed sign comparisons (the oldest open D issue :( [0]).
Java's "solution", which is to not have unsigned types.
 My point is that if D didn't follow the usual arithmetic conversions, much
fewer 
 newcomers would even notice compared to extremely large backlash that we may
get 
 if go with the string interpolation -> raw tuple approach.
As opposed to backlash from another gc-required feature and low performance and not usable with printf nor writef nor any user-built functions like OutBuffer.printf and all the ones used by dmd.
 1. Have a simple pragma(inline, true) wrapper function that will convert the 
 distinct type to printf-style args. This wrapper function can even be named 
 printf as it would work by virtue of function overloading. This is O(1) 
 additional code that once written no one will need to bother with.

 2. Have the new type implicitly convert to printf-style args. I think this is 
 what Adam is proposing. While nice to have, I don't think it's necessary.
It's better to build things from simple components than try to break up a complex feature into components.
 As for std.stdio.write(f)(ln), there's no reason why any garbage would need to 
 be created, as again, a simple overload that accepts the distinct type will be 
 able to handle it just like it would (performance-wise) with DIP1027. However, 
 with DIP1027, only writef and writefln can be used, while write and writeln
will 
 produce wrong results (wrong according to people that have used string 
 interpolation in other languages).
Magic types are not simple and inevitably lead to unexpected corners and unresolvable problems. *cough* associative arrays *cough* You can have everything you want with DIP1027 interpolated strings by wrapping it in a function call to a function you specify. And so can everyone else. DIP1027 will also likely lead to a number of unexpected treasures, as has happened repeatedly when simple building blocks were added, while complex ones just caused trouble.
Feb 27
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Feb 27, 2020 at 11:26:37AM -0800, Walter Bright via
Digitalmars-d-announce wrote:
[...]
 Magic types are not simple and inevitably lead to unexpected corners
 and unresolvable problems. *cough* associative arrays *cough*
[...] For all the trouble they've given us, built-in AA's is one of the primary reasons I love D. IMO, it's inexcusible for a programming language in this day and age not to have built-in AA's, either as part of the language, or as part of the standard library -- but with FULL LANGUAGE SUPPORT. Meaning that all built-in types can be used as AA keys without further ado, unlike C++'s standard hashtable which IMNSHO is a horrible piece of nigh-unusable crap, because you can't even declare a hashtable without defining your own hash function, you can't use structs as keys unless you also manually declare a hash function, a comparison function, and who knows what other onerous detail I can't recall, and then you have to use that horrendous iterator API to lookup anything at all. Once, I wanted to add a simple hashtable as a caching mechanism to one of my C++ projects. After *two days* it still wasn't working properly, because there is no default hash function for anything, not even ints, so I have to write my own, and the hash function must be wrapped in some kind of functional wrapper object because you can't pass functions to the template parameter, then to initialize the hashtable I have to jump through a whole bunch of hoops, mostly involving the fact that to create the hash function object I needed to pass the necessary context all over the place -- so I had to rewrite 70% of the internal API just to be able to pass the context to the right places. Then the dumb thing required C++11 or something like that, which broke my C++98 code in horrendous ways that required a full-scale refactor to fix some of the incompatibilities. Eventually I threw in the towel, and spent about a month's time to rewrite the whole danged project in D -- from scratch. I can't describe how refreshing it was to be able to just write: Value[Key] aa; and that's *it*. No fuss, no muss, it Just Works(tm). Sure, when there's a performance issue then I have to add opHash or opEquals or what-not to my types, but the important thing is, there are *sane defaults* for everything. The compiler isn't going to throw my code back in my face just because I used a custom type instead of string for the AA key (some languages have this problem), and the custom type may contain other custom types but it just knows to compute the default hash value accordingly. If D *didn't* have .opHash built-in, it would've been a totally different story, probably not too unlike my horrible experience with C++. If there wasn't a default hash function, and if I had to manually declare a whole bunch of stuff just to be able to use a struct containing a pair of ints as key, then it would have been a horrendous experience. TL;DR: I'm *glad* D has built-in AA's, in spite of whatever flaws they may have. It was the Right Choice, even though the execution, in retrospect, was a bit wanting. T -- MSDOS = MicroSoft's Denial Of Service
Feb 27
parent reply Atila Neves <atila.neves gmail.com> writes:
On Thursday, 27 February 2020 at 20:00:52 UTC, H. S. Teoh wrote:
 On Thu, Feb 27, 2020 at 11:26:37AM -0800, Walter Bright via 
 Digitalmars-d-announce wrote: [...]
 [...]
[...] For all the trouble they've given us, built-in AA's is one of the primary reasons I love D. [...]
The reason for C++ forcing users to do that is the lack of compile-time reflection.
Feb 28
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Feb 28, 2020 at 05:45:35PM +0000, Atila Neves via
Digitalmars-d-announce wrote:
 On Thursday, 27 February 2020 at 20:00:52 UTC, H. S. Teoh wrote:
[...]
 For all the trouble they've given us, built-in AA's is one of the
 primary reasons I love D.
 [...]
The reason for C++ forcing users to do that is the lack of compile-time reflection.
Whatever the reason may be, the result is a total usability nightmare compared to D's convenience. Even a supposedly "flawed" feature like AA's in D compares favorably to C++; I seriously can't bring myself to write C++ code anymore. D has ruined my life. :-P T -- Those who don't understand D are condemned to reinvent it, poorly. -- Daniel N
Feb 28
prev sibling parent Arine <arine123445128843 gmail.com> writes:
On Thursday, 27 February 2020 at 14:32:29 UTC, Petar Kirov 
[ZombineDev] wrote:
 On Thursday, 27 February 2020 at 09:30:30 UTC, Walter Bright 
 wrote:
 Rust does not follow C syntax at all, so nobody will 
 reasonably expect it to have C semantics. D does follow it, 
 it's a feature, so people will have expectations.
I'm not sure where exactly you draw the line, but I would say doesn't import some of the broken C semantics like implicit narrowing conversions (luckily, neither does D) and allowing mixed sign comparisons (the oldest open D issue :( [0]). My point is that if D didn't follow the usual arithmetic conversions, much fewer newcomers would even notice compared to extremely large backlash that we may get if go with the string interpolation -> raw tuple approach. [0]: https://issues.dlang.org/show_bug.cgi?id=259
Funnily enough the only contribution by Walter in that 14 year old bug is him trying to close it without there having been a solution implemented. The spec is also still wrong, after 14 years. https://dlang.org/spec/expression.html#integer_comparisons
Feb 28
prev sibling parent reply SealabJaster <sealabjaster gmail.com> writes:
On Thursday, 27 February 2020 at 09:30:30 UTC, Walter Bright 
wrote:
 You can make it behave like all those other languages simply 
 with:

     f(format("hello $a"));
At that point, it begs the question of why even bother having string interpolation. I'd like to imagine that most newcomers/returning veterans of D would see "D has string interpolation!" and then expect it to work similar to how it does in most other languages. They'd expect f(format("hello %s", a)) to be replaced by a much more compact f("hello $a"), instead of some D-ism take on interpolation that doesn't really provide much in the way of the main use of the feature: easily creating formatted strings. People more used to D will likely go "Oh, that's kind of weird yet cool" once they learn what it lowers down to, yet newcomers who aren't too caught up with the language's features will simply be annoyed and confused that it 'feels' (not necessarily is) more complicated than needed. It appears the main focus in these dicussions are "What super super cool and niche (in comparison to string formatting) things can we do with this feature!" while mostly blind-siding that it's generally an easy-to-use, easy-to-understand feature in other languages. Similarly, as far as I can tell even with the adjustment of making these strings their own special type, something as simple (for other languages as): ``` void f(string a, string b) { /*...*/ } int foo = 20; string bar = "lalafell"; f(i"You are a foo($foo)", i"I am not a $bar"); ``` Isn't easily achieved without use of an extra helper function such as i"...".format or .str, etc. My view on it is that this is generally a QoL feature, which some principle of interpolation while also granting user code the string interpolation works as-expected, yet check out functions such as FromSqlInterpolated[1] which can directly access the format string & values simply by taking a special `FormattableString` type) To be honest, if D's take on interpolation makes people scratch their head in confusion at the most basic use case, then it's probably not even worth adding to the language. [1] https://docs.microsoft.com/en-us/dotnet/api/microsoft.entityframeworkcore.relationalqueryableextensions.fromsqlinterpolated?view=efcore-3.1
Feb 27
next sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
On 28/02/2020 3:47 AM, SealabJaster wrote:
 Similarly, as far as I can tell even with the adjustment of making these 
 strings their own special type, something as simple (for other languages 
 as):
 
 ```
 void f(string a, string b)
 { /*...*/ }
 
 int foo = 20;
 string bar = "lalafell";
 f(i"You are a foo($foo)", i"I am not a $bar");
 ```
 
 Isn't easily achieved without use of an extra helper function such as 
 i"...".format or .str, etc.
What you have suggested is a GC only language feature. This will cut out numerous use cases for D, and will leave those people swearing and complaining. Unlike new users who may not discover this feature for a while, existing users will complain and will not be happy guranteed. This is something we as a community want to avoid.
Feb 27
prev sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 27 February 2020 at 14:47:55 UTC, SealabJaster wrote:
 At that point, it begs the question of why even bother having 
 string interpolation.
I encourage you to read my document too: https://github.com/dlang/DIPs/pull/186 It addresses all these concerns. Walter's proposal is dead. It has been formally rejected. We shouldn't waste more time talking about it.
 I'd like to imagine that most newcomers/returning veterans of D 
 would see "D has string interpolation!" and then expect it to 
 work similar to how it does in most other languages.
My proposal doesn't work exactly like in other languages - it is can do, except the implicit conversion to string... BUT, if you assume it is the same as other languages, you get a compile error, and the error message tells you how to convert it to a regular string! f(i"hello $a"); error: cannot implicitly convert argument of type interpolated tuple to type string. Tip: use `.idup` to explicitly convert it to string. f(i"hello $a".idup); // works almost like other languages now! f2(i"hello $a", i"hello $b"); // type mismatch error, try idup f2(i"hello $a".idup, i"hello $b".idup); // works So education wise, it is only a few seconds: if you use it like in other languages, it doesn't work, but the compiler tells you how to fix it. If you find the error or `.idup` method unacceptable, then... I'm sorry, but you aren't going to win that one. We've already argued this at length and the community is not willing to lose what we'd lose (ref, scope, compile-time usage, and more) by making that work; the trade-offs are too steep.
Feb 27
parent SealabJaster <sealabjaster gmail.com> writes:
On Thursday, 27 February 2020 at 15:12:23 UTC, Adam D. Ruppe 
wrote:
 error: cannot implicitly convert argument of type interpolated 
 tuple to type string. Tip: use `.idup` to explicitly convert it 
 to string.
Oh, that pretty much sorts out my problem there, sorry if I glanced over it being mentioned previously.
 We've already argued this at length and the community is not 
 willing to lose
 what we'd lose (ref, scope, compile-time usage, and more)
And that also explains to me why having the values packed into a Again, sorry if I missed it being mentioned before. My main other argument was the possible complexity/bugginess of having functions that provide special support for these strings, as from Walter's DIP it seemed each function would have to implement their own parsing of the format string. But I see your DIP already addresses that (_d_interpolated_format_spec("")). I'm kind of struggling to see now why your changes were so vehemently rejected now. Other than that, just pretend my misinformed post doesn't exist.
Feb 27
prev sibling parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 27/02/2020 01:20, Walter Bright wrote:
 On 2/26/2020 3:13 AM, Petar Kirov [ZombineDev] wrote:
 In all other languages with string interpolation that I'm familiar
 with, `a` is not passed to the `i` parameter.
All rely on a garbage collected string being generated as an intermediate variable.
The string buffer could also be stack allocated or manually managed with malloc/free by the string interpolation type.
Feb 27
next sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Thursday, 27 February 2020 at 09:45:06 UTC, Rainer Schuetze 
wrote:
 On 27/02/2020 01:20, Walter Bright wrote:
 On 2/26/2020 3:13 AM, Petar Kirov [ZombineDev] wrote:
 In all other languages with string interpolation that I'm 
 familiar with, `a` is not passed to the `i` parameter.
All rely on a garbage collected string being generated as an intermediate variable.
The string buffer could also be stack allocated or manually managed with malloc/free by the string interpolation type.
Don't forget LDC does GC to stack optimisations. There ought to be no need to do anything manually if you care about perf.
Feb 27
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/27/2020 1:45 AM, Rainer Schuetze wrote:
 The string buffer could also be stack allocated or manually managed with
 malloc/free by the string interpolation type.
It's quite a big deal to make that work, and does not address the inherent inefficiency of it. printf, for all its faults, is very efficient, and one reason is it does not require arbitrary temporary storage. Another is it does not require exception handlers. I, for one, simply would not use such when printf is available. People often overlook how *expensive* RAII is. Turn exception handling on and have a look at the generated code. It's a minor syntactic convenience with an unexpected large and hidden cost. That's not what D is about. Leave this at the user's discretion with: f(format("hello %betty")); where the user chooses via the format function which method of handling temporaries he finds appropriate. --- Note that the reason outbuffer.d has a printf member function is because it is fast and efficient to format data directly into the output buffer rather than going through a series of temporary strings first.
Feb 27
parent Arine <arine123445128843 gmail.com> writes:
On Thursday, 27 February 2020 at 18:58:40 UTC, Walter Bright 
wrote:
 On 2/27/2020 1:45 AM, Rainer Schuetze wrote:
 The string buffer could also be stack allocated or manually 
 managed with
 malloc/free by the string interpolation type.
It's quite a big deal to make that work, and does not address the inherent inefficiency of it. printf, for all its faults, is very efficient, and one reason is it does not require arbitrary temporary storage. Another is it does not require exception handlers. I, for one, simply would not use such when printf is available.
It isn't that efficient. There are a lot of implementations faster than it, and funnily enough they allocate memory, use exceptions, and RAII.
 People often overlook how *expensive* RAII is. Turn exception 
 handling on and have a look at the generated code.
There was a whole other thread about this, and people found the assembly generated isn't all that different. Yes throwing exceptions is expensive, but what you are talking about, the cost of not throwing them, it really isn't. You are making it out to be a bigger problem than it actually is. If it caused such a huge performance hit, then exceptions simply wouldn't be used at all in C++. But that's not the case.
 It's a minor syntactic convenience with an unexpected large and 
 hidden cost. That's not what D is about.

 Leave this at the user's discretion with:

     f(format("hello %betty"));

 where the user chooses via the format function which method of 
 handling temporaries he finds appropriate.
There are quite a few places that D has large hidden costs. Hell GC and any feature or otherwise that uses it is a large and hidden cost. You can't really say D is not about unexpected large and hidden costs when it has a GC integrated into quite a few features that make it invisible.
Feb 28
prev sibling parent reply Arine <arine123445128843 gmail.com> writes:
On Wednesday, 26 February 2020 at 09:45:55 UTC, Walter Bright 
wrote:
 On 2/25/2020 1:36 AM, aliak wrote:
 This may have already been answered in the other threads, but 
 I was just wondering if anyone managed to propose a way to 
 avoid this scenario with DIP1027?
 
 void f(string s, int i = 0);
 f(i"hello $a"); // silent unwanted bahviour.
 
 ?
It is lowered to: f("hello %s", a); as designed. I don't know what's unwanted about it.
Wow. There were multiple examples in the review thread. void CreateWindow(string title, int w = -1, int h = -1); int a; CreateWindow(i"Title $a"); Yah, what's unwanted about that?
Feb 26
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Wednesday, 26 February 2020 at 15:41:48 UTC, Arine wrote:
 Yah, what's unwanted about that?
To follow up on this, I expect a reply will be "the user ought to know how the feature works". This isn't a realistic expectation. This is why I put in my little narrative in the new DIP, though it is deleted now since everyone else thought it was too anecdotal... But as many of you know, I try to help D users pretty regularly, both existing D users with tricky questions and new D users who just aren't familiar with the language yet. I've answered about 1/5 of all stack overflow D questions myself, am often in the IRC channel to answer user questions, and read every post in the D.learn forum, again answering many myself. So I see a lot of frequently asked questions and have developed a bit of a feel for where people common have trouble. mismatched expectations coming from another language. A compile error is a chance to educate them as to D's difference, why it is that way, and how they can make it work for them. That's why I want a compile error on misuse. A type mismatch is the standard way we achieve this. A quick run time error (e.g. null pointer segfault) results in users figuring it out on their own or asking us fairly quickly, but risks additional frustration since it is more delayed from the build. But code that looks right to someone coming from another language yet silently doing weird stuff at runtime... I guarantee you would-be converts to D will find that really confusing and frustrating. Every single book or tutorial on D will have to call this out specifically. Every single usage example will have to point out the mismatched expectations. Of course the spec will say it, but people often don't read the spec. (How many times have you seen people ask if D's `private` keyword is buggy?) And a lot of new users will make the mistake anyway. We can prevent that with this simple, proven solution.
Feb 26
parent Arine <arine123445128843 gmail.com> writes:
On Wednesday, 26 February 2020 at 16:08:53 UTC, Adam D. Ruppe 
wrote:
 On Wednesday, 26 February 2020 at 15:41:48 UTC, Arine wrote:
 Yah, what's unwanted about that?
To follow up on this, I expect a reply will be "the user ought to know how the feature works". This isn't a realistic expectation.
Yah I don't understand how he's fine on one hand introducing weird edge cases like this with String Interpolation. And then on the other hand he's removing brace initializers with a justification that it makes the language simpler and easier to learn. The examples given are obviously errors, and aren't the intended use for string interpolation. Rather than allowing errors to exist, it should just be made so that it doesn't happen.
Feb 26
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/26/2020 7:41 AM, Arine wrote:
 Yah, what's unwanted about that?
1. unwanted extra string allocation 2. poor performance 3. doesn't work with printf 4. doesn't work with writef 5. non-default formats require extra temp strings to be generated
Feb 27
next sibling parent reply aliak <something something.com> writes:
On Thursday, 27 February 2020 at 09:34:23 UTC, Walter Bright 
wrote:
 On 2/26/2020 7:41 AM, Arine wrote:
 Yah, what's unwanted about that?
1. unwanted extra string allocation 2. poor performance 3. doesn't work with printf 4. doesn't work with writef 5. non-default formats require extra temp strings to be generated
Pretty sure he meant that this call: int a; CreateWindow(i"Title $a"); Would call CreateWindow like: CreateWindow("Title %s", a); Which is what is unwanted. Btw: with the adam-steve-twist you fix everything that is unwanted, including enforcing explicitness when someone wants it to act as a string, without the danger of mistakenly calling the wrong overloads of functions because of switching to string interpolation. And it seems to me there's precedent in typeid i.e. typeid is an construct in D that lowers to a TypeInfo type, which is defined somewhere (is that object.d?) Anyway. The same would happen with the interpolated string. Btw: Swift does this for string interpolation and it works very well -> https://www.hackingwithswift.com/articles/178/super-powered-string-interpolation-in-swift-5-0
Feb 27
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/27/2020 3:44 AM, aliak wrote:
 Btw: Swift does this for string interpolation and it works very well -> 
 https://www.hackingwithswift.com/articles/178/super-powered-string-interp
lation-in-swift-5-0 
I don't know Swift, but this looks like the "generate strings and concatenate them" approach.
Feb 27
parent reply Jacob Carlborg <doob me.com> writes:
On Friday, 28 February 2020 at 03:10:48 UTC, Walter Bright wrote:

 I don't know Swift, but this looks like the "generate strings 
 and concatenate them" approach.
No, it basically lowers to bunch of method calls. Here's an example of how it could look like with D syntax: auto a = 3; auto b = i"foo $a bar"; Could be lowered to: auto _temp = StringInterpolation(8 /* literal capacity */, 1 /* interpolation count */); _temp.appendLiteral("foo "); _temp.appendInterpolation(a); _temp.appendLiteral(" bar"); auto b = _temp.toString(); There's nothing here that says that this needs to use the GC to allocate the final string. "StringInterpolation" could contain something like OutBuffer from DMD or, if the arguments were passed as template arguments, it could allocate on the stack. -- /Jacob Carlborg
Feb 28
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/28/20 5:17 AM, Jacob Carlborg wrote:
 On Friday, 28 February 2020 at 03:10:48 UTC, Walter Bright wrote:
 
 I don't know Swift, but this looks like the "generate strings and 
 concatenate them" approach.
No, it basically lowers to bunch of method calls. Here's an example of how it could look like with D syntax: auto a = 3; auto b = i"foo $a bar"; Could be lowered to: auto _temp = StringInterpolation(8 /* literal capacity */, 1 /* interpolation count */); _temp.appendLiteral("foo "); _temp.appendInterpolation(a); _temp.appendLiteral(" bar"); auto b = _temp.toString();
I think Walter's point is that swift is still appending strings and then returning that. This requires allocations, and is not as preferable as directly processing the data. Not only that, but it's generating temporary strings just to add them to the larger thing that will be printed (I'm assuming this is not just a big string but an array/list, due to the beginning of the video and s1+s2+s3+s4). I'd much prefer for example, printing using DIP1027 than constructing a string (even if the memory is reasonably fast, like malloc) just to throw it away. I watched a lot of that video, it didn't impress me much. I use swift interpolation strings quite a bit, and they are useful. But I think D's will be much more performant and more straightforward for hooking (if they ever get in). -Steve
Feb 28
parent reply aliak <something something.com> writes:
On Friday, 28 February 2020 at 19:16:08 UTC, Steven Schveighoffer 
wrote:
 On 2/28/20 5:17 AM, Jacob Carlborg wrote:
 On Friday, 28 February 2020 at 03:10:48 UTC, Walter Bright 
 wrote:
 
 I don't know Swift, but this looks like the "generate strings 
 and concatenate them" approach.
No, it basically lowers to bunch of method calls. Here's an example of how it could look like with D syntax: auto a = 3; auto b = i"foo $a bar"; Could be lowered to: auto _temp = StringInterpolation(8 /* literal capacity */, 1 /* interpolation count */); _temp.appendLiteral("foo "); _temp.appendInterpolation(a); _temp.appendLiteral(" bar"); auto b = _temp.toString();
I think Walter's point is that swift is still appending strings and then returning that. This requires allocations, and is not as preferable as directly processing the data. Not only that, but it's generating temporary strings just to add them to the larger thing that will be printed (I'm assuming this is not just a big string but an array/list, due to the beginning of the video and s1+s2+s3+s4). I'd much prefer for example, printing using DIP1027 than constructing a string (even if the memory is reasonably fast, like malloc) just to throw it away. I watched a lot of that video, it didn't impress me much. I use swift interpolation strings quite a bit, and they are useful. But I think D's will be much more performant and more straightforward for hooking (if they ever get in). -Steve
I actually didn't realize it was a video, thought it was just an article! - But anyway, it was just to point out that swift lowers to specialized types when it comes to interpolation (which is what you and adam are trying to get through). And therefor you can detect interpolations being given to you and deal with them the way you want and you can do a lot when you know you're getting an interpolation. You can create types like let example: SQLStatment = "select * from blah where a=\(a), b=\(b) ... " I also didn't realize the takeaway would be that swift does appending 😆- which by the way, is not completely accurate. And it does not generate temporaries (unless you mean passing in parameters? There's no way around that if you want to end up with a string based on runtime values - it'll have to be processed in to a string somewhere). You can also get an interpolated string directly in to "print processing" if you wanted to: https://swift.godbolt.org/z/muAzgm And for the record I think the tuple generation is great as well. I highly doubt it'll be easier to use than swift (case in point: no need to call idup or whatever to convert to a string, since a string in swift is a type that is "interpolation aware"). Hook in to maybe, it depends on the APIs provided to hook in to them. An opaque type will not be easier to hook in to and a "concrete" named interface (aka protocol in swift). When it comes to printing it really doesn't matter if you construct a string on the stack and pass it along. You're IO bound anyway. By the by: if you or anyone is interested in swift's string interpolation design decisions (for inspiration or whatever) then here's the full discussion: https://forums.swift.org/t/string-interpolation-revamp-design-decisions/12624 One very interesting thing of note is the way they combine named arguments with string interpolations. Also another note, this tuple expansion should really not be called string interpolation, since it does not result in a string :/ It's more string expansion really.
Feb 28
next sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Saturday, 29 February 2020 at 00:57:54 UTC, aliak wrote:
 Also another note, this tuple expansion should really not be 
 called string interpolation, since it does not result in a 
 string :/ It's more string expansion really.
Yeah, me and Steven agreed on this too in the other thread, and I thought I updated my DIP but I guess not. wait a minute yes i did https://gist.github.com/adamdruppe/a58f097d974b364ae1cbc8c050dd9a3f on the gist version, I called it "Formatted string tuple literals", but I never saved tht back to my local or github versions. oops. But basically I see our new thing as being a string builder rather than a string per se.
Feb 28
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/28/20 7:57 PM, aliak wrote:
 
 I actually didn't realize it was a video, thought it was just an 
 article! - But anyway, it was just to point out that swift lowers to 
 specialized types when it comes to interpolation (which is what you and 
 adam are trying to get through). And therefor you can detect 
 interpolations being given to you and deal with them the way you want 
 and you can do a lot when you know you're getting an interpolation. You 
 can create types like
 
 let example: SQLStatment = "select * from blah where a=\(a), b=\(b) ... "
I didn't get to this part of the video, but that is indeed pretty cool. I'm assuming that this generates placeholders for the SQL statement and attaches a and b as parameters? However, D cannot do something like this exactly, because expressions define the tuple, not how they are used. But this is possible (with the proposed DIP or ours): alias sql = "select * from blah where a=$a, b=$b"; // aliased to the tuple connection.query(sql); a = 5; connection.query(sql); // another query with `a` set to 5 now. Swift can do some pretty cool things due to the type resolver, but it comes at a cost (some expressions that are trivial in D make the compiler complain about them taking too long to resolve).
 I also didn't realize the takeaway would be that swift does appending 
 😆- which by the way, is not completely accurate. And it does not 
 generate temporaries (unless you mean passing in parameters? There's no 
 way around that if you want to end up with a string based on runtime 
 values - it'll have to be processed in to a string somewhere).
For example, the part where they change the date formatting, they use a date formatter to generate a string for the date, which then is appended to the string interpolation. Yes, you need to allocate a string. But you should only allocate one.
 You can also get an interpolated string directly in to "print 
 processing" if you wanted to: https://swift.godbolt.org/z/muAzgm
Hm... I'm not too impressed with this when compared to writefln(i"hello $("hello"), $x"); which works without such extra mechanics or strange call syntax.
 
 When it comes to printing it really doesn't matter if you construct a 
 string on the stack and pass it along. You're IO bound anyway.
IO is not usually processed directly to the device. Usually it's buffered. Writing directly to the buffer is preferable to generating a string on the stack, and just copying that to the buffer.
 One very interesting thing of note is the way they combine named 
 arguments with string interpolations.
Yeah, that part is cool. But that comes for free with Swift. Potentially with the right additions to the named parameters DIP, it would be feasible to do something similar with this DIP.
 Also another note, this tuple expansion should really not be called 
 string interpolation, since it does not result in a string :/ It's more 
 string expansion really.
I recommended calling it a "formatted tuple" to avoid conflation with existing string interpolation implementations. -Steve
Feb 29
parent aliak <something something.com> writes:
On Saturday, 29 February 2020 at 14:41:16 UTC, Steven 
Schveighoffer wrote:
 On 2/28/20 7:57 PM, aliak wrote:
 
 I actually didn't realize it was a video, thought it was just 
 an article! - But anyway, it was just to point out that swift 
 lowers to specialized types when it comes to interpolation 
 (which is what you and adam are trying to get through). And 
 therefor you can detect interpolations being given to you and 
 deal with them the way you want and you can do a lot when you 
 know you're getting an interpolation. You can create types like
 
 let example: SQLStatment = "select * from blah where a=\(a), 
 b=\(b) ... "
I didn't get to this part of the video, but that is indeed pretty cool. I'm assuming that this generates placeholders for the SQL statement and attaches a and b as parameters?
I'm not sure if it's in the video (I haven't seen the video). But it was just an example, implementations would vary I'd assume. Here's one from a quick search for e.g.: https://github.com/groue/GRDB.swift/blob/4c8574aa4c08ff715ce0e63b38502ac1e8529069/GRDB/Core/SQLInterpolation.swift
 However, D cannot do something like this exactly, because 
 expressions define the tuple, not how they are used.
Yep, and similarly expressions define the interpolation pattern in swift, not how they are used as well. So the tuple thing is also pretty sweet! I'd imaging you'd be able to do something like: auto sql = SQLStatment(i"select * from blah where a=$a, b=$b"); where struct SQLStatement { string formatted; // What template magic is needed here? This is where I get a little worried. this(S : _interpolated_object!Specs, Specs, Args...)(S s, Args args) { // build formatted...? } alias formatter this; }
 But this is possible (with the proposed DIP or ours):

 alias sql = "select * from blah where a=$a, b=$b"; // aliased 
 to the tuple

 connection.query(sql);
 a = 5;
 connection.query(sql); // another query with `a` set to 5 now.
Yeah, and that's pretty sweet too!
 Swift can do some pretty cool things due to the type resolver, 
 but it comes at a cost (some expressions that are trivial in D 
 make the compiler complain about them taking too long to 
 resolve).
Yeah, swift ha a few problems with its type solver during semantic I think. let a: Double = -(1 + 2) + -(3 + 4) + -(5) Will fail to compile on my machine, and you need to break it up. But, on the other hand, swift's constraint solver avoids backwards situations like: void f(bool) { "bool".writeln; } void f(int) { "int".writeln; } enum A { one } void main() { f(0); f(A.one); } And: void main() { struct B { B opBinary(string op : "+")(int b) { return this; } } static if (is(typeof(B.init + size_t.init))) { size_t x = 1; B b1, b2; b1 = b2 + x; // fails here } }
 I also didn't realize the takeaway would be that swift does 
 appending 😆- which by the way, is not completely accurate. And 
 it does not generate temporaries (unless you mean passing in 
 parameters? There's no way around that if you want to end up 
 with a string based on runtime values - it'll have to be 
 processed in to a string somewhere).
For example, the part where they change the date formatting, they use a date formatter to generate a string for the date, which then is appended to the string interpolation. Yes, you need to allocate a string. But you should only allocate one.
Generating temporaries is not a constraint that the interpolation system puts on you.
 You can also get an interpolated string directly in to "print 
 processing" if you wanted to: 
 https://swift.godbolt.org/z/muAzgm
Hm... I'm not too impressed with this when compared to writefln(i"hello $("hello"), $x"); which works without such extra mechanics or strange call syntax.
Indeed.
Mar 01
prev sibling next sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 27 February 2020 at 09:34:23 UTC, Walter Bright 
wrote:
 On 2/26/2020 7:41 AM, Arine wrote:
 Yah, what's unwanted about that?
[snip]
You're arguing against a strawman. The other poster's comment was showing a likely problem with the (rejected) dip 1027, that our new proposal fixes. The new proposal does not allocate a new string, so none of your points apply to it.
Feb 27
prev sibling parent reply Arine <arine123445128843 gmail.com> writes:
On Thursday, 27 February 2020 at 09:34:23 UTC, Walter Bright 
wrote:
 On 2/26/2020 7:41 AM, Arine wrote:
 Yah, what's unwanted about that?
1. unwanted extra string allocation 2. poor performance 3. doesn't work with printf 4. doesn't work with writef 5. non-default formats require extra temp strings to be generated
Sometimes I wonder if you even bother to read posts to understand. This is a problem with YOUR DIP. Where you said, "what's wrong with that, it's working as intended". void CreateWindow(string title, int w = -1, int h = -1); int a; CreateWindow(i"Title $a"); // becomes CreateWindow("Title %s", a); That's what your fine with, that's what the DIP your wrote would allow. It's like you are just reading what you want to, instead of actually understanding what people are saying. I'd have more luck talking to a brick wall at the rate this thread is going, jesus.
Feb 27
parent Meta <jared771 gmail.com> writes:
On Thursday, 27 February 2020 at 18:07:19 UTC, Arine wrote:
 On Thursday, 27 February 2020 at 09:34:23 UTC, Walter Bright 
 wrote:
 On 2/26/2020 7:41 AM, Arine wrote:
 Yah, what's unwanted about that?
1. unwanted extra string allocation 2. poor performance 3. doesn't work with printf 4. doesn't work with writef 5. non-default formats require extra temp strings to be generated
Sometimes I wonder if you even bother to read posts to understand. This is a problem with YOUR DIP. Where you said, "what's wrong with that, it's working as intended". void CreateWindow(string title, int w = -1, int h = -1); int a; CreateWindow(i"Title $a"); // becomes CreateWindow("Title %s", a); That's what your fine with, that's what the DIP your wrote would allow. It's like you are just reading what you want to, instead of actually understanding what people are saying. I'd have more luck talking to a brick wall at the rate this thread is going, jesus.
Come on, this isn't Reddit. Be more civil.
Feb 27
prev sibling parent reply =?iso-8859-1?Q?Robert_M._M=FCnch?= <robert.muench saphirion.com> writes:
On 2020-02-23 16:22:46 +0000, Mike Parker said:

 DIP 1027, "String Interpolation", has been rejected. The decision was 
 primarily influenced by the lack of consensus over the implementation 
 and the syntax demonstrated in the two review threads. As the DIP 
 author, Walter also rejected the suggestion to go with an 
 implementation that resolves to a library template. He sees that as 
 equivalent to AST macros, a feature which he has previously rejected.
 
 https://github.com/dlang/DIPs/blob/4be15bd40381667c0ab1c0aef360d0daa4b8c82c/DIPs
rejected/DIP1027.md 
 
Out of curiosity, how and who makes such a decision? Is there a voting? Is there a committee? Is there a structured pro/con overview and highlight of blocking-points that need to be resolved? I mean, people spend a lot of time thinking, making suggestions, etc. and the end result is: we now have nothing. Which, IMO is the worst result for all. As a community with highly skilled people I think there should be a way to come to a good solution, not only a good enough. If not, this doesn't shed light on D and the community... -- Robert M. Mnch http://www.saphirion.com smarter | better | faster
Feb 24
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/24/2020 12:43 AM, Robert M. Münch wrote:
 Out of curiosity, how and who makes such a decision? Is there a voting? Is
there 
 a committee? Is there a structured pro/con overview and highlight of 
 blocking-points that need to be resolved?
I talk it over with Atila after the review threads are done, then decide. Voting is a terrible way to do engineering design, for many reasons I can list if you like.
 I mean, people spend a lot of time thinking, making suggestions, etc. and the 
 end result is: we now have nothing. Which, IMO is the worst result for all.
Not totally. I was unable to convince people that printf format validation was an orthogonal feature. So I went ahead and implemented that to show it had nothing to do with the proposal: https://github.com/dlang/dmd/pull/10812 It turned out kinda sweet, and found a huge number of latent bugs in our own code, proving we shoulda done that long ago :-) Bad printfs turned out to be much more common than I'd anticipated.
 As a community with highly skilled people I think there should be a way to
come 
 to a good solution, not only a good enough. If not, this doesn't shed light on
D 
 and the community...
Community input is what the review threads are for. Consensus on a good solution did not emerge.
Feb 24
next sibling parent reply Arine <arine123445128843 gmail.com> writes:
On Monday, 24 February 2020 at 09:51:16 UTC, Walter Bright wrote:
 On 2/24/2020 12:43 AM, Robert M. Münch wrote:
 I mean, people spend a lot of time thinking, making 
 suggestions, etc. and the end result is: we now have nothing. 
 Which, IMO is the worst result for all.
Not totally. I was unable to convince people that printf format validation was an orthogonal feature. So I went ahead and implemented that to show it had nothing to do with the proposal: https://github.com/dlang/dmd/pull/10812
With the way that the DIP you wrote up, it really isn't orthogonal. How people *expect* string interpolation to work is like this: float a; int b; printf(i"$a $b"); // should just work What your DIP did is this: float a; int b; // printf(i"$a $b"); becomes: printf("%s %s", a, b); printf(i"${%d}a ${%f}b"); // need to do formatting on your own anyways The only way you wouldn't want printf format validation for such a feature is if you are fine with buggy code. You literally still require the user to format the strings themselves, this is no different than simply using printf. And guess what:
 It turned out kinda sweet, and found a huge number of latent 
 bugs in our own code, proving we shoulda done that long ago :-) 
 Bad printfs turned out to be much more common than I'd 
 anticipated.
No shit.
Feb 24
parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/24/2020 9:24 AM, Arine wrote:
 No [...]
Using unprofessional language will result in your posts being deleted. Please stop.
Feb 24
prev sibling parent =?iso-8859-1?Q?Robert_M._M=FCnch?= <robert.muench saphirion.com> writes:
On 2020-02-24 09:51:16 +0000, Walter Bright said:

 I talk it over with Atila after the review threads are done, then 
 decide. Voting is a terrible way to do engineering design, for many 
 reasons I can list if you like.
You don't need to, I'm not a fan of such a voting approach too. I just added a bunch of variants as I didn't know how the process works nor who is involved at what step.
 Not totally. I was unable to convince people that printf format 
 validation was an orthogonal feature. So I went ahead and implemented 
 that to show it had nothing to do with the proposal:
 
 https://github.com/dlang/dmd/pull/10812
 
 It turned out kinda sweet, and found a huge number of latent bugs in 
 our own code, proving we shoulda done that long ago :-) Bad printfs 
 turned out to be much more common than I'd anticipated.
Proving by showing the results... Ok, looking forward how things develop. -- Robert M. Mnch http://www.saphirion.com smarter | better | faster
Feb 24
prev sibling next sibling parent reply Mike Parker <aldacron gmail.com> writes:
On Monday, 24 February 2020 at 08:43:57 UTC, Robert M. Münch 
wrote:

 
Out of curiosity, how and who makes such a decision? Is there a voting? Is there a committee? Is there a structured pro/con overview and highlight of blocking-points that need to be resolved?
The DIP review process is outlined here: https://github.com/dlang/DIPs The final decision rests with the language maintainers. Now, that means Walter and Atila.
 I mean, people spend a lot of time thinking, making 
 suggestions, etc. and the end result is: we now have nothing. 
 Which, IMO is the worst result for all.
Not at all. In this case, as the DIP author, Walter could have chosen to revise the DIP with a new implementation. He chose not to. He wasn't persuaded by the arguments in the thread. Anyone else in the community is free to submit an alternative DIP for the same feature. Adam Ruppe has already done so: https://github.com/dlang/DIPs/pull/186 If that proposal and its review manage to convince Walter and Atila, then the feature gets in.
 As a community with highly skilled people I think there should 
 be a way to come to a good solution, not only a good enough. If 
 not, this doesn't shed light on D and the community...
The DIP process is one way to approach that. It allows everyone to provide feedback and influence the drafting of the DIP. That requires convincing the DIP author to revise the DIP and ultimately convincing the language maintainers to accept it. It means there's a high barrier for acceptance, but in my own opinion that's how it should be.
Feb 24
next sibling parent Basile B. <b2.temp gmx.com> writes:
On Monday, 24 February 2020 at 10:02:26 UTC, Mike Parker wrote:
 I mean, people spend a lot of time thinking, making 
 suggestions, etc. and the end result is: we now have nothing. 
 Which, IMO is the worst result for all.
Not at all. In this case, as the DIP author, Walter could have chosen to revise the DIP with a new implementation. He chose not to. He wasn't persuaded by the arguments in the thread.
Yeah, not to mention that in first place I think that he made a DIP for named args because the commuity was unable to formulate a good propostion, while he was himself not "pro-named args"... (maybe, I think i read this in the past)
Feb 24
prev sibling parent =?iso-8859-1?Q?Robert_M._M=FCnch?= <robert.muench saphirion.com> writes:
On 2020-02-24 10:02:26 +0000, Mike Parker said:

 The DIP review process is outlined here:
 
 https://github.com/dlang/DIPs
 
 The final decision rests with the language maintainers. Now, that means 
 Walter and Atila.
Thanks, and sorry for my ignorance... How about adding a note who the "language maintainers" are?
 ... If that proposal and its review manage to convince Walter and 
 Atila, then the feature gets in.
Ok. I understand.
 The DIP process is one way to approach that. It allows everyone to 
 provide feedback and influence the drafting of the DIP. That requires 
 convincing the DIP author to revise the DIP and ultimately convincing 
 the language maintainers to accept it. It means there's a high barrier 
 for acceptance, but in my own opinion that's how it should be.
I agree. My impression and point was (after following the topic loosly) that the DIP might be close to find a good solution and that it was suddenly withdrawn. But anyway... -- Robert M. Mnch http://www.saphirion.com smarter | better | faster
Feb 24
prev sibling parent bachmeier <no spam.net> writes:
On Monday, 24 February 2020 at 08:43:57 UTC, Robert M. Münch 
wrote:

 I mean, people spend a lot of time thinking, making 
 suggestions, etc. and the end result is: we now have nothing. 
 Which, IMO is the worst result for all.
We don't have nothing. There's an option value of not making a decision at this time. Once you decide to go forward with a string interpolation proposal, you're deciding not to go forward with any other similar proposal for many years, or even forever. This is IMO a minor feature (not a big deal if we don't have it). If you're going to do it, do it right. In some cases, there's really no way to know until you have a formal proposal and discuss the heck out of it whether it's worth going forward.
Feb 24