www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - String Interpolation Compare - DIP1027 and YAIDIP

reply Walter Bright <newshound2 digitalmars.com> writes:
https://github.com/dlang/DIPs/blob/master/DIPs/rejected/DIP1027.md

https://github.com/John-Colvin/YAIDIP

Both designs fulfill their basic function of string interpolation, and both are 
user extensible to enable most anything. They look rather alike. Beyond that, 
they are quite different. DIP1027 is based around the
printf()/writef()/format() 
model, while YAIDIP is based on the write() model. DIP1027 generates a format 
string followed by arguments to it, YAIDIP generates a template instance 
followed by a sequence of string arguments.

The rest is about differences that matter. Of course I'm biased about this - I 
went looking for fault and found it. It's better to find fault in advance
rather 
than belatedly, because whatever we pick we're going to be stuck with. I 
strongly favor designs that are self-evident and require minimal to no 
documentation. I prefer designs that are simple building blocks, where the user 
can combine such blocks to form more complex designs. Designs that have minimal 
special cases and fit with the rest of the language are better.

As with Ddoc and unittest, designs that don't address every need, but cover the 
vast bulk of needs with a simple self-evident design, are better designs.

I apologize for any typos and mistakes due to misunderstanding YAIDIP.





1. Is less susceptible (but not immune) to accidentally matching the wrong 
function or template.



1. It's inefficient because of more than double the number of arguments that 
have to be passed.

2. It's inefficient because it requires the arguments to be converted to string 
temporaries, and then the strings are appended to the result. This is both
slow, 
and requires string memory allocations. In contrast, a formatter (like printf 
and writef) does not require string intermediates, the generated characters can 
be sent directly to the sink.

3. To get anything other than the default conversion, such as adding leading 
spaces, writing a formatting conversion function is necessary, and then called. 
This means the i-string is going to get quite long. Consider the difference
between:

```
     i"axy:  ${%03d}a ${%e}x ${%20.10f}y"  // DIP1027
```
```
     i"axy:  $leadingZero$digits(3)$a $scientific$x $fixed(20,10)$y" // YAIDIP
```

If there are more than a small number of embedded arguments, the length of the 
i-string becomes untenable and unreadable. The compact formatting language, 
which has been pretty standard for nearly 50 years, is a lot easier to deal
with.

4. If the manipulators are not in scope, there's no way to qualify them with
the 
'.' syntax. Temporary aliases are necessary:

```
     alias leadingZero = std.istring.leadingZero;
     alias digits = std.istring.digits;
     alias scientific = std.istring.scientific;
     alias fixed = std.istring.fixed;
     i"axy:  $leadingZero$digits(3)$a $scientific$x $fixed(20,10)$y"
```

5. The tuple output, which the user is sooner or later going to be confronted 
with, looks like (copied from YAIDIP):

```
     writeln(.object.imported!"core.interpolation".InterpolationHeader!("",
"x", 
" can be written as ", scientific, "", x, " or ", fixed(20, 10), "", x, ".")(),
   x, " can be written as ", scientific, "", x, " or ", fixed(20, 10), "", x,
".");
```

This will show up in error messages, and the user is confronted with how the 
sausage is made. I had to read YAIDIP several times to try to figure out what 
that text actually meant. Contrast that with:

```
     writefln("%s can be written as %e or %20.10f", x, y);
```

where it is simple and self-evident what is happening.

6. It requires another template added to object.d, which is already so bloated 
with templates it consumes a large compile time penalty. It requires the user
to 
understand this template.

7. It requires the creation of another module, call it `std.istring`, to
contain 
all the manipulators a user will expect to see (such as `scientific`).

8. There will be difficulties using this with betterC because of the required 
string allocations mentioned earlier.

9. i-strings are special case behaviors for pragma, mixin, assert, static 
assert, function calls, constructor calls, and template instantiations. They
are 
not general purpose. They cannot be used for things like generating tuples for 
other uses.

----------------------------------------------





1. does not require memory allocation

2. does not require object.d support

3. does not require a library of manipulators

4. works out of the box with all the existing formatted string functions, and 
leverages the decades of optimizations that the C formatted string functions 
employ, along with existing knowledge of those functions and how they work

5. can do everything YAIDIP does

6. is faster

7. uses less memory

8. does not emit template bloat into the object files

9. does not emit structs into the object files

10. the i-strings are much more compact

11. generates a tuple that is self-evident

12. generates a tuple that can be used anywhere tuples work



1. Is more susceptible to inadvertent matching with the wrong function, because 
the tuples generated are tuples of strings, integers, and other ordinary types.

-----------------------------

Nothing is perfect, but DIP1027 is a much better fit for D's style of simple 
elegance.
Oct 20 2023
next sibling parent reply monkyyy <crazymonkyyy gmail.com> writes:
On Saturday, 21 October 2023 at 01:50:33 UTC, Walter Bright wrote:

 


airnt both implimented?
even if you hate it, can you just throw one of them behind a 
compiler flag and mark it as "not final"?
Oct 20 2023
parent bachmeier <no spam.net> writes:
On Saturday, 21 October 2023 at 02:10:06 UTC, monkyyy wrote:

 On Saturday, 21 October 2023 at 01:50:33 UTC, Walter Bright 
 wrote:

 


 airnt both implimented?
 even if you hate it, can you just throw one of them behind a 
 compiler flag and mark it as "not final"?


Given that I'm not able to follow the details of either proposal 
(after the first 50 pages I fall asleep) I think it would be 
helpful to try both with an experimental flag. My impression is 
that many of the arguments in favor of one proposal or another 
are speculative.
Oct 20 2023
prev sibling next sibling parent reply Adam D Ruppe <destructionator gmail.com> writes:
Note to readers: you don't have to believe second-hand nonsense. 
There's functional implementations to play with already:

YAIDIP's first draft itself: 
https://github.com/dlang/dmd/pull/15714

YAIDIP is not perfect, so I made a better iteration on the 
concept too:
https://github.com/dlang/dmd/pull/15715

Some things in there could still be changed, of course, but even 
in its current draft state, you can easily see what is true and 
false in these comparisons.




On Saturday, 21 October 2023 at 01:50:33 UTC, Walter Bright wrote:

 YAIDIP generates a template instance followed by a sequence of 
 string arguments.


Not true. The only string arguments are the strings interpolated 
with the arguments.


 2. It's inefficient because it requires the arguments to be 
 converted to string temporaries, and then the strings are 
 appended to the result.


Not true. You have shown zero understanding of what it is, even 
though there's an implementation you could look at in addition to 
the document!


     i"axy:  $leadingZero$digits(3)$a $scientific$x 
 $fixed(20,10)$y" // YAIDIP


This is one possible use case of it, but there's several others, 
including using format strings. It is 100% up to the library what 
to do; the compiler is agnostic to these specifics.


 4. If the manipulators are not in scope, there's no way to 
 qualify them with the '.' syntax. Temporary aliases are 
 necessary:


Not true. This might be the stupidest of your ignorant 
statements, since even basic knowledge of how D modules work show 
your examples are nonsense (you can `import std.istring;`!), and 
if you read the DIP or played with the implementation, you'd also 
know you can enclose a fully-qualified name in parenthesis too.


 5. The tuple output, which the user is sooner or later going to 
 be confronted with, looks like (copied from YAIDIP):


This is half true. We could make the compiler error messages 
nicer with a bit of implementation effort, but currently they 
messages do indeed look like:

test.d(51): Error: function `test.foos(string s)` is not callable 
using argument types `(InterpolationHeader, 
InterpolatedLiteral!"Hello, ", InterpolatedExpression!"name", 
string, InterpolatedLiteral!"", InterpolationFooter)`
test.d(51):        cannot pass argument `InterpolationHeader()` 
of type `InterpolationHeader` to parameter `string s`



     writefln("%s can be written as %e or %20.10f", x, y);


But this is still a dishonest comparison. If you passed that to a 
`void foos(string s)` you'd get an error message along the lines 
of:

test.d(52): Error: function `test.foos(string s)` is not callable 
using argument types `(string, string)` test.d(52):        
expected 1 argument(s), not 2

Which would still require the user to understand how the thing 
works, since if they are thinking they ARE passing a string, 
they'd wonder why the compiler thinks there's two.


 6. It requires another template added to object.d, which is 
 already so bloated with templates it consumes a large compile 
 time penalty. It requires the user to understand this template.


Not true.


 7. It requires the creation of another module, call it 
 `std.istring`, to contain all the manipulators a user will 
 expect to see (such as `scientific`).


Not true. (But this is an option, but it is not a requirement).


 8. There will be difficulties using this with betterC because 
 of the required string allocations mentioned earlier.


Not true.


 9. i-strings are special case behaviors for pragma, mixin, 
 assert, static assert, function calls, constructor calls, and 
 template instantiations. They are not general purpose. They 
 cannot be used for things like generating tuples for other uses.


Half true, they would be special case for pragma, etc., if you 
wanted to (I don't btw, I'd tell the user to call the function), 
but they *can* be used for other users. THAT'S THE WHOLE POINT.






 3. does not require a library of manipulators


Not true - that library is `format` or `printf` etc., but it is 
there.

Of course, the other proposal also works with those functions. We 
even provided implementations - it is a short function. One short 
example is here: 
https://github.com/dlang/dmd/pull/15715#issuecomment-1773146760


 5. can do everything YAIDIP does


Completely and utterly false, as we've gone over many, many, many 
times.


 10. the i-strings are much more compact


False.


 11. generates a tuple that is self-evident


That's just like, your opinion, man.


 12. generates a tuple that can be used anywhere tuples work


False.






 1. Is more susceptible to inadvertent matching with the wrong 
 function, because the tuples generated are tuples of strings, 
 integers, and other ordinary types.


Hey, you finally said something that's 100% true!


Again, to see the truth, look at the implementations:

YAIDIP itself: https://github.com/dlang/dmd/pull/15714

And a better iteration of the concept:
https://github.com/dlang/dmd/pull/15715

DIP texts are hard to read. Experimenting with it is easy. Don't 
believe second-hand nonsense.
Oct 20 2023
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
The examples I used are copied verbatim from the YAIDIP. If they're all false, 
please write a better proposal.
Oct 20 2023
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On Saturday, 21 October 2023 at 03:48:49 UTC, Walter Bright wrote:

 The examples I used are copied verbatim from the YAIDIP. If 
 they're all false, please write a better proposal.


Your interpretation is false, not the examples.

They are examples of what's possible, not requirements.

That being said, the proposal needs much work. To be fair it's 
not an official proposal. It should be made clear enough to be 
understood by the language maintainers.

-Steve
Oct 20 2023
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/20/2023 8:56 PM, Steven Schveighoffer wrote:

 On Saturday, 21 October 2023 at 03:48:49 UTC, Walter Bright wrote:

 The examples I used are copied verbatim from the YAIDIP. If they're all false, 
 please write a better proposal.


 
 Your interpretation is false, not the examples.
 
 They are examples of what's possible, not requirements.


Then, again, a far more thorough proposal is necessary. YAIDIP says nothing at 
all about `{03}` formats. It gives no examples of them. `{ }` isn't even in the 
grammar! I'm not making false statements about YAIDIP by saying it doesn't have 
them, because it doesn't.

How it works needs to be clearly and completely written in the specification.
Oct 20 2023
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On Saturday, 21 October 2023 at 04:36:05 UTC, Walter Bright wrote:

 On 10/20/2023 8:56 PM, Steven Schveighoffer wrote:

 On Saturday, 21 October 2023 at 03:48:49 UTC, Walter Bright 
 wrote:

 The examples I used are copied verbatim from the YAIDIP. If 
 they're all false, please write a better proposal.


 
 Your interpretation is false, not the examples.
 
 They are examples of what's possible, not requirements.


 Then, again, a far more thorough proposal is necessary. YAIDIP 
 says nothing at all about `{03}` formats. It gives no examples 
 of them. `{ }` isn't even in the grammar! I'm not making false 
 statements about YAIDIP by saying it doesn't have them, because 
 it doesn't.


That's because it's not part of the proposed feature. It's just 
an example of how the proposed feature could be used. As is the 
"scientific" et. al. usage. If you understand the proposal you 
can understand the possibilities. Hopefully examples help to 
understand the possibilities without having to enumerate all 
future infinite possibilities.

That being said, this example in the DIP is poor IMO. It doesn't 
sell the feature at all, and to me looks like a step down. 
Technically some people might prefer that style of formatting, 
but I don't count myself as one of them.


 How it works needs to be clearly and completely written in the 
 specification.


Agreed. But it is important to note that YAIDIP and 1036 describe 
a *language feature*. The section on "Use Cases" is not a set of 
requirements for the proposal. They are just examples.

I think the DIP could be clearer about what is part of the 
proposal and what is just an example. It should have a clear 
statement such as "That concludes the proposal. The following are 
descriptions of examples that can be realized with the proposed 
feature, and are in no way requirements of the proposal."

-Steve
Oct 20 2023
parent reply Walter Bright <newshound2 digitalmars.com> writes:
I expect the grammar would be part of the proposal, not a use case.

In any case, I don't see a point in discussing YAIDIP any further, and will 
await the real proposal.
Oct 20 2023
next sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Saturday, 21 October 2023 at 06:04:21 UTC, Walter Bright wrote:

 I expect the grammar would be part of the proposal, not a use 
 case.

 In any case, I don't see a point in discussing YAIDIP any 
 further, and will await the real proposal.


I’m not really sure I grok the differences between the two 
proposals at the moment, but the people who support YAIDIP seem 
to think you have a lot of misconceptions and have left the door 
open for you to look at the implementations or play with it to 
have them corrected. If a bunch of people told me I was badly 
mistaken about something, I would want to find out why…

I’d say put them both behind preview switches and have the 
proponents of each put together some showcases of what they can 
do, or can’t do.
Oct 21 2023
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/21/2023 7:20 AM, jmh530 wrote:

 I’m not really sure I grok the differences between the two proposals at the 
 moment, but the people who support YAIDIP seem to think you have a lot of 
 misconceptions and have left the door open for you to look at the 
 implementations or play with it to have them corrected. If a bunch of people 
 told me I was badly mistaken about something, I would want to find out why…


I know why. YAIDIP is not an actual specification for it. When Adam and Steven 
prepare a specification then I will look at it again.

(For example, Steven mentions a { } syntax, which does not exist in YADIP.)
Oct 21 2023
parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Saturday, 21 October 2023 at 17:27:01 UTC, Walter Bright wrote:

 On 10/21/2023 7:20 AM, jmh530 wrote:

 I’m not really sure I grok the differences between the two 
 proposals at the moment, but the people who support YAIDIP 
 seem to think you have a lot of misconceptions and have left 
 the door open for you to look at the implementations or play 
 with it to have them corrected. If a bunch of people told me I 
 was badly mistaken about something, I would want to find out 
 why…


 I know why. YAIDIP is not an actual specification for it. When 
 Adam and Steven prepare a specification then I will look at it 
 again.

 (For example, Steven mentions a { } syntax, which does not 
 exist in YADIP.)


So simply move the discussion on the implementation pull request 
of YAIDIP [1]: no misunderstanding about what can and can't do. 
It's the best way to grok it.

https://github.com/dlang/dmd/pull/15715

Discuss about the code, not about description of what the code is 
supposed to do.

/P
Oct 21 2023
parent Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Saturday, 21 October 2023 at 17:38:00 UTC, Paolo Invernizzi 
wrote:

 On Saturday, 21 October 2023 at 17:27:01 UTC, Walter Bright 
 wrote:

 On 10/21/2023 7:20 AM, jmh530 wrote:

 [...]


 I know why. YAIDIP is not an actual specification for it. When 
 Adam and Steven prepare a specification then I will look at it 
 again.

 (For example, Steven mentions a { } syntax, which does not 
 exist in YADIP.)


 So simply move the discussion on the implementation pull 
 request of YAIDIP [1]: no misunderstanding about what can and 
 can't do. It's the best way to grok it.

 https://github.com/dlang/dmd/pull/15715

 Discuss about the code, not about description of what the code 
 is supposed to do.

 /P


I think so too, or just provide a set of examples.

Like DIP x:
string s =<something>

DIP y:
string s = <something>

etc etc
Oct 21 2023
prev sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On Saturday, 21 October 2023 at 06:04:21 UTC, Walter Bright wrote:

 I expect the grammar would be part of the proposal, not a use 
 case.

 In any case, I don't see a point in discussing YAIDIP any 
 further, and will await the real proposal.


I'm getting exhausted with this thread, as you seem not to pay 
attention to anything I'm saying. So this set of replies will be 
my last.

The YAIDIP discusses *completely* the grammar of the solution, 
the requirements of the solution, and *THEN* goes into possible 
use cases.

The C++-style stream manipulators *USE CASE* is not part of the 
proposal at all. In no way, will phobos, or druntime, gain some 
std.iformat module as a *REQUIREMENT* of this DIP.

Neither are the hypothetical format specifiers that I posted 
earlier. They are *POSSIBLE* things you might do if given the 
power that this DIP provides, not an integral part of the DIP.

-Steve
Oct 21 2023
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On Saturday, 21 October 2023 at 02:22:49 UTC, Adam D Ruppe wrote:

 On Saturday, 21 October 2023 at 01:50:33 UTC, Walter Bright 
 wrote:

 2. It's inefficient because it requires the arguments to be 
 converted to string temporaries, and then the strings are 
 appended to the result.


 Not true. You have shown zero understanding of what it is, even 
 though there's an implementation you could look at in addition 
 to the document!


You can even call printf with betterC (no runtime memory 
allocation whatsoever, not even malloc) with introspection of 
parameters, something that 1027 *cannot do*.


     i"axy:  $leadingZero$digits(3)$a $scientific$x 
 $fixed(20,10)$y" // YAIDIP


 This is one possible use case of it, but there's several 
 others, including using format strings. It is 100% up to the 
 library what to do; the compiler is agnostic to these specifics.


YAIDIP:

```d
i"axy:  $a{03} $x{e} $y{20.10}"
```

Compare that to dip 1027 (how verbose!)

```d
i"axy:  ${%03d}a ${%e}x ${%20.10f}y"
```


 5. The tuple output, which the user is sooner or later going 
 to be confronted with, looks like (copied from YAIDIP):


 This is half true. We could make the compiler error messages 
 nicer with a bit of implementation effort, but currently they 
 messages do indeed look like:

 ```
 test.d(51): Error: function `test.foos(string s)` is not 
 callable using argument types `(InterpolationHeader, 
 InterpolatedLiteral!"Hello, ", InterpolatedExpression!"name", 
 string, InterpolatedLiteral!"", InterpolationFooter)`
 test.d(51):        cannot pass argument `InterpolationHeader()` 
 of type `InterpolationHeader` to parameter `string s`
 ```


YAIDIP is here to help!

```
Perhaps use `std.conv.text` to make a string from an 
interpolation tuple?
```

And of course we can clean up the parameter types to be more 
readable (we already do this for `string`).


 8. There will be difficulties using this with betterC because 
 of the required string allocations mentioned earlier.




How can you possibly think that YAIDIP requires allocations? The 
only proposal that requires allocations is DIP1027, since you 
must rebuild the format string if your function doesn't support 
printf style format specifiers (i.e. mysql). You have it exactly 
backwards.






 3. does not require a library of manipulators


 Not true - that library is `format` or `printf` etc., but it is 
 there.


Not even printf works as expected, only writef.

Try doing printf with a string parameter (not a string literal) 
with DIP1027.

---

I want to stress that I'm not in love with YAIDIP. If we want to 
go with a templated interpolation tuple, I'd wish for 1036, but 
without the auto-string conversion.

-Steve
Oct 20 2023
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/20/2023 8:51 PM, Steven Schveighoffer wrote:

 YAIDIP:
 
 ```d
 i"axy:  $a{03} $x{e} $y{20.10}"
 ```


You and I must be reading a different version of 
https://github.com/John-Colvin/YAIDIP

May I quote:

"There is no need for defining, implementing, and memorizing a sui generis 
mini-language of encoded format specifiers --- all formatting can be done with
D 
language expressions. Continuing the example, the library can just as easily 
define parameterized formatting for floating-point numbers, such as width, 
precision, and scientific notation:"

```
void fun(double x) {
     writeln(i"$x can be written as $scientific$x or $(fixed(20, 10))$x.");
     // Lowering: --->
     // writeln(.object.imported!"core.interpolation".InterpolationHeader!("", 
"x", " can be written as ", scientific, "", x, " or ", fixed(20, 10), "", x,
".")(),
     //     x, " can be written as ", scientific, "", x, " or ", fixed(20, 10), 
"", x, ".");
}
```
Oct 20 2023
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On Saturday, 21 October 2023 at 03:59:22 UTC, Walter Bright wrote:

 On 10/20/2023 8:51 PM, Steven Schveighoffer wrote:

 YAIDIP:
 
 ```d
 i"axy:  $a{03} $x{e} $y{20.10}"
 ```


 You and I must be reading a different version of 
 https://github.com/John-Colvin/YAIDIP

 May I quote:


That portion is a *possible* usage. Perhaps the original authors 
of YAIDIP didn't realize that it's still worth pointing out that 
it effectively can also be solved with format specifiers. I 
personally find the C++ style of formatting not as appealing as 
the writef style. That portion of the DIP lists it as a benefit, 
but I'm with you on the terseness of format specifications. It 
really should be more clearly identified as an option, and not an 
integral part of the proposal.

But that's the whole thing, with a tuple-based solution, you have 
options in the library. You *can* use the C++ style or you *can* 
use formatting encoded in the string literal. Both are available 
depending on what you, the library author, would prefer.

And of course, YAIDIP (and 1036) has the benefit that runtime 
parsing of the string-based format specifiers is not necessary, 
whereas it is in 1027.

-Steve
Oct 20 2023
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/20/2023 8:51 PM, Steven Schveighoffer wrote:

 Not even printf works as expected, only writef.
 
 Try doing printf with a string parameter (not a string literal) with DIP1027.


To use printf with i-strings, it's implicitly necessary to use arguments that 
are compatible with printf. That means const(char)* arguments, not string 
arguments. This is something that should be obvious once how i-string
conversion 
to tuples works.

DIP1027 has zero knowledge of `writef` or `printf` or `format`.

--- However ---

Ever since we added printf format checking to the language, I've toyed with the 
idea of going a step further and adjusting the arguments (and formats) of the 
calls to printf to make them work.

For example, a D string argument `s` would be split into two parameters, 
`cast(int)s.length, s.ptr`. The format `%s` would be replaced with `%.*s`. In 
other words, you could do things like:

```
printf("integer %s", 67);
```

would be rewritten as:

```
printf("integer %d", 67);
```

After all, if the format checker can give an error message with the correct 
format specifier, why not just fix it?

However, this would be independent of string interpolation. It's just that the 
two separate initiatives would fit together handsomely when dealing with printf.
Oct 20 2023
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On Saturday, 21 October 2023 at 04:10:35 UTC, Walter Bright wrote:

 On 10/20/2023 8:51 PM, Steven Schveighoffer wrote:

 Not even printf works as expected, only writef.
 
 Try doing printf with a string parameter (not a string 
 literal) with DIP1027.


 To use printf with i-strings, it's implicitly necessary to use 
 arguments that are compatible with printf. That means 
 const(char)* arguments, not string arguments. This is something 
 that should be obvious once how i-string conversion to tuples 
 works.

 DIP1027 has zero knowledge of `writef` or `printf` or `format`.


Yes and no. It's very clearly geared towards writef (or else we 
would have some other default other than %s), even though it 
doesn't require hooking with writef. But in effect, writef and 
format (and other functions like it, such as loggers) are the 
only functions it works with "out of the box". There are many 
other functions that use the same style of "blueprint + data" 
calls which will utterly fail, and can't be made to work with 
DIP1027.

BTW, I count a failure as having to specify the correct 
substitution as the user. That is the domain of the function or 
the type system.


 --- However ---

 Ever since we added printf format checking to the language, 
 I've toyed with the idea of going a step further and adjusting 
 the arguments (and formats) of the calls to printf to make them 
 work.


Nobody, and I really truly mean, NOBODY, is clamoring for better 
ways to call printf (specifically printf).

And if you are, YAIDIP or DIP1036 is your guy. It can do it, 
without any specialized compiler help, without allocations, and 
in betterC mode (as I have demonstrated).

-Steve
Oct 20 2023
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/20/2023 9:42 PM, Steven Schveighoffer wrote:

 Yes and no. It's very clearly geared towards writef (or else we would have
some 
 other default other than %s), even though it doesn't require hooking with 
 writef. But in effect, writef and format (and other functions like it, such as 
 loggers) are the only functions it works with "out of the box".


That's no surprise, since Phobos has adopted that style for more than just 
writef, such as std.format.format(). There's no reason to invent a different
one 
for loggers.

https://dlang.org/phobos/std_format.html#format

Besides, I had modified DIP1027 to enable any format string between { } 
specifically to accommodate your database use case.



 There are many 
 other functions that use the same style of "blueprint + data" calls which will 
 utterly fail, and can't be made to work with DIP1027.


Example, please. I already showed how to do Adam's example of things that 
wouldn't work with DIP1027.



 BTW, I count a failure as having to specify the correct substitution as the 
 user. That is the domain of the function or the type system.


I don't know what that means.


 Nobody, and I really truly mean, NOBODY, is clamoring for better ways to call 


printf (specifically printf).

Which is too bad. printf has many advantages:

1. small
2. only the call is emitted to the object file
3. several decades spent optimizing it
4. ubiquitous
5. very well documented
6. very battle-tested

(2) is very important when trying to track down a bug, and one needs to examine 
the object code. D's format checking has eliminated much of the problems using
it.


 NOBODY


Nobody wanted ImportC, either. However, several people have told me that they 
have been seduced into using it and found it delightful and very productive. 
Heck, in another posting I discovered it could be used to translate C code to D!
Oct 20 2023
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On Saturday, 21 October 2023 at 06:44:22 UTC, Walter Bright wrote:

 On 10/20/2023 9:42 PM, Steven Schveighoffer wrote:

 Yes and no. It's very clearly geared towards writef (or else 
 we would have some other default other than %s), even though 
 it doesn't require hooking with writef. But in effect, writef 
 and format (and other functions like it, such as loggers) are 
 the only functions it works with "out of the box".


 That's no surprise, since Phobos has adopted that style for 
 more than just writef, such as std.format.format(). There's no 
 reason to invent a different one for loggers.

 https://dlang.org/phobos/std_format.html#format

 Besides, I had modified DIP1027 to enable any format string 
 between { } specifically to accommodate your database use case.


This is woefully inadequate. It requires users to participate in 
the details of the SQL format that are irrelevant to the function 
call, and it also allows people to do the wrong thing quite 
easily, with no compiler help. Statically typed languages are 
better than this, and especially D is. We don't want to make 
users have to know how the "sausage is made".

This is why I consider it a failure.

consider the even worse case of postgres -- all parameters 
require a positional index, so:

```
INSERT INTO tbl (name, age) VALUES ($1, $2)
```

Now, the user not only has to change the %s to $N, but they have 
to maintain the ordering of the indexes manually. These are the 
kinds of things that people "get used to" because they have no 
other options. Much better to let the function itself handle the 
proper placeholders that correspond to the implementation details 
of the usage.

You might say, "hey, that's just the way it is, you are using 
postgres". But what about a DB abstraction library? It might want 
to change the placeholders based on the underlying backend.


 There are many other functions that use the same style of 
 "blueprint + data" calls which will utterly fail, and can't be 
 made to work with DIP1027.


 Example, please. I already showed how to do Adam's example of 
 things that wouldn't work with DIP1027.


The point I was making was that there are no other functions 
except writef and format that work, or can be made to work, 
directly with this mechanism (of course, without sausage-making 
by the user). Even printf requires the user to become involved.

Of course, it is possible to write a completely new function with 
a new name that will handle (poorly) the puzzle that the compiler 
constructed in its quirky writef-like way (oh, and make sure you 
don't change those placeholders, because we can't detect that!)


 BTW, I count a failure as having to specify the correct 
 substitution as the user. That is the domain of the function 
 or the type system.


 I don't know what that means.


The {} thing you mentioned above. It does not suffice as 
discussed above. The user shouldn't have to be involved with 
figuring out placeholders when we have a perfectly capable 
compiler to do it for us.


 Nobody, and I really truly mean, NOBODY, is clamoring for


 better ways to call printf (specifically printf).

 Which is too bad. printf has many advantages:


I was saying printf is fine the way it is, nobody needs new ways 
to call printf.


 NOBODY


 Nobody wanted ImportC, either. However, several people have 
 told me that they have been seduced into using it and found it 
 delightful and very productive. Heck, in another posting I 
 discovered it could be used to translate C code to D!


1) Everybody wanted a way to import C directly. It's been 
probably the most requested language change for D ever. ImportC 
is turning out to be a great success, and I think it will change 
D's future for the better.
2) People can and do successfully and joyfully use printf. Nobody 
is asking for more printf capabilities.
3) YAIDIP and DIP1036 cover this mechanism way way better than 
DIP1027 or some special case compiler workaround would.

-Steve
Oct 21 2023
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/20/2023 8:51 PM, Steven Schveighoffer wrote:

 How can you possibly think that YAIDIP requires allocations?


How else can you make a user-defined dec() work?



 The only proposal 
 that requires allocations is DIP1027, since you must rebuild the format string 
 if your function doesn't support printf style format specifiers (i.e. mysql). 


The example I gave in the other thread shows how to not need to rewrite the 
format string.

https://www.digitalmars.com/d/archives/digitalmars/D/Just_another_example_of_missing_string_interpolation_370542.html#N370696

But even if you did rewrite it, it doesn't escape the template function, and
can 
be RAII'd.

Dealing with a string allocated and returned by `dec()` is significantly harder.
Oct 20 2023
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On Saturday, 21 October 2023 at 04:17:57 UTC, Walter Bright wrote:

 On 10/20/2023 8:51 PM, Steven Schveighoffer wrote:

 How can you possibly think that YAIDIP requires allocations?


 How else can you make a user-defined dec() work?


I'm not sure I understand this. What is `dec()`?


 The only proposal that requires allocations is DIP1027, since 
 you must rebuild the format string if your function doesn't 
 support printf style format specifiers (i.e. mysql).


 The example I gave in the other thread shows how to not need to 
 rewrite the format string.

 https://www.digitalmars.com/d/archives/digitalmars/D/Just_another_example_of_missing_string_interpolation_370542.html#N370696


mysql requires a string as the sql for the prepared statement. 
Basically, you pass a string with a different type of placeholder 
specifier "?". It does not accept "%s". This is not something I 
have any control over.

So naturally, since you only get a runtime format string from 
DIP1027, you need to create an equivalent runtime string to pass 
to the library. How can you do that without allocations?


 But even if you did rewrite it, it doesn't escape the template 
 function, and can be RAII'd.


That is not comparable to building the correct string at 
compile-time. It also requires *parsing* the format string at 
runtime.


 Dealing with a string allocated and returned by `dec()` is 
 significantly harder.


You will have to expand on what `dec()` is, I'm unfamiliar with 
it.

-Steve
Oct 20 2023
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/20/2023 9:54 PM, Steven Schveighoffer wrote:

 I'm not sure I understand this. What is `dec()`?


 From https://github.com/John-Colvin/YAIDIP :

"Consider for example using stream manipulators such as dec and hex for writeln 
by using an i-string:+"

```
void fun(int x) {
     writeln(i"$dec$x in hexadecimal is 0x$hex$x.");
     // Lowering: --->
     // writeln(.object.imported!"core.interpolation".InterpolationHeader!("", 
"dec", "", "x", " in hexadecimal is 0x", "hex", "", "x", ".")(),
     //     "", dec, "", x, " in hexadecimal is 0x", hex, "", x, ".");
}
```

And:

"There is no need for defining, implementing, and memorizing a sui generis 
mini-language of encoded format specifiers"

I wasn't making false statements about that, either.



 https://www.digitalmars.com/d/archives/digitalmars/D/Just_another_example_of_missing_string_interpolation_370542.html#N370696


 
 mysql requires a string as the sql for the prepared statement. Basically, you 
 pass a string with a different type of placeholder specifier "?". It does not 
 accept "%s". This is not something I have any control over.


See my example. There's no reason it cannot be extended to replace the %s with 
whatever is needed. Also, DIP1027 provides the { } syntax enabling insertion of 
whatever placeholder specifier is needed.



 So naturally, since you only get a runtime format string from DIP1027, you
need 
 to create an equivalent runtime string to pass to the library. How can you do 
 that without allocations?


The { } syntax. But even if you needed an allocation, a malloc/free pair will 
suffice. The problem with dec() is dec() returns an allocated string, so the 
free gets messy.


 But even if you did rewrite it, it doesn't escape the template function, and 
 can be RAII'd.


 
 That is not comparable to building the correct string at compile-time. It also 
 requires *parsing* the format string at runtime.


I'd agree that parsing it is a last resort if the { } also fails. But having 
implemented formatted writes, I can attest that parsing it is not a big deal. 
It's scanning till you see the %, then substitute your own version. Of course, 
using { } means the compiler does it for you at compile time.
Oct 20 2023
next sibling parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 21/10/2023 7:57 PM, Walter Bright wrote:

 I'd agree that parsing it is a last resort if the { } also fails. But 
 having implemented formatted writes, I can attest that parsing it is not 
 a big deal. It's scanning till you see the %, then substitute your own 
 version. Of course, using { } means the compiler does it for you at 
 compile time.


This also means that one formatter doesn't have to know about another 
formatters format string syntax.

Like how formattedWrite doesn't need to know about GDateTime.

And yes, I have that working today with the ``{:}`` syntax in my 
codebase. Its great.

This is why I do not agree with Adam about his argument against 
supporting the format string specifier as part of the language. Having 
it in the language means it is all nicely unified.
Oct 21 2023
parent Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Saturday, 21 October 2023 at 07:04:35 UTC, Richard (Rikki) 
Andrew Cattermole wrote:

 On 21/10/2023 7:57 PM, Walter Bright wrote:

 [...]


 This also means that one formatter doesn't have to know about 
 another formatters format string syntax.

 Like how formattedWrite doesn't need to know about GDateTime.

 And yes, I have that working today with the ``{:}`` syntax in 
 my codebase. Its great.

 This is why I do not agree with Adam about his argument against 
 supporting the format string specifier as part of the language. 
 Having it in the language means it is all nicely unified.



indeed
Oct 21 2023
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On Saturday, 21 October 2023 at 06:57:30 UTC, Walter Bright wrote:

 On 10/20/2023 9:54 PM, Steven Schveighoffer wrote:

 I'm not sure I understand this. What is `dec()`?


 From https://github.com/John-Colvin/YAIDIP :

 "Consider for example using stream manipulators such as dec and 
 hex for writeln by using an i-string:+"

 ```d
 void fun(int x) {
     writeln(i"$dec$x in hexadecimal is 0x$hex$x.");
     // Lowering: --->
     // 
 writeln(.object.imported!"core.interpolation".InterpolationHeader!("", "dec",
"", "x", " in hexadecimal is 0x", "hex", "", "x", ".")(),
     //     "", dec, "", x, " in hexadecimal is 0x", hex, "", x, 
 ".");
 }
 ```


Oh I see now what you are looking at! This is just like C++ 
though.

```c++
std::cout << "value in hex: " << std::hex << 42
           << ", value in decimal: " << std::dec << 42 << 
std::endl;
```

`dec` is not a function that gets called on the value, it's a 
manipulator to set a flag that says "now do decimal numbers". 
It's no different for this example use case.


 And:

 "There is no need for defining, implementing, and memorizing a 
 sui generis mini-language of encoded format specifiers"

 I wasn't making false statements about that, either.


The false statement is not that these words exist in the 
proposal. It's that you have misinterpreted what these words are 
-- these things are *USE CASES* and not required for the proposal.


 https://www.digitalmars.com/d/archives/digitalmars/D/Just_another_example_of_missing_string_interpolation_370542.html#N370696


 
 mysql requires a string as the sql for the prepared statement. 
 Basically, you pass a string with a different type of 
 placeholder specifier "?". It does not accept "%s". This is 
 not something I have any control over.


 See my example. There's no reason it cannot be extended to 
 replace the %s with whatever is needed.


Hence, requiring allocation and runtime parsing to build the 
string.


 So naturally, since you only get a runtime format string from 
 DIP1027, you need to create an equivalent runtime string to 
 pass to the library. How can you do that without allocations?


 The { } syntax.


No, the {} thing is a horrendous workaround. Nobody will use it, 
they'd just use the original, as this is just a large set of 
footguns that has no redeemable qualities.

Consider that for mysql there are no "format specifiers", the 
entire point for having the {} is negated because it should 
*always be* `?`. You just did `$foo` and not `${?}foo`? Sorry 
user, that's on you. It will compile and fail at runtime, or even 
worse it will succeed and do something completely unexpected. 
This is not good API design. Writing incorrect code with this 
feature is super easy and will happen non-stop. Ironically, 
printf is going to be better because it has a magic compiler 
pragma to do the things that somehow the metaprogramming 
powerhouse of D could not handle.


 But even if you needed an allocation, a malloc/free pair will 
 suffice.


YAIDIP requires no allocations, even to build a format string.


 The problem with dec() is dec() returns an allocated string, so 
 the free gets messy.


First, of course as previously mentioned, The C++ style stream 
manipulators are not a requirement of the DIP (or even proposed 
for the language). And second, as mentioned above, `dec()` does 
not allocate a string.


 But even if you did rewrite it, it doesn't escape the 
 template function, and can be RAII'd.


 
 That is not comparable to building the correct string at 
 compile-time. It also requires *parsing* the format string at 
 runtime.


 I'd agree that parsing it is a last resort if the { } also 
 fails. But having implemented formatted writes, I can attest 
 that parsing it is not a big deal. It's scanning till you see 
 the %, then substitute your own version. Of course, using { } 
 means the compiler does it for you at compile time.


Of course, we can all just switch to C or assembly, what is the 
big deal? The big deal is, I have a 21st century compiler, with 
insanely good metaprogramming capabilities. I shouldn't be 
parsing strings that the compiler built at compile time to give 
me substandard information. Give me the good stuff.

-Steve
Oct 21 2023
parent Paul Backus <snarwin gmail.com> writes:
On Saturday, 21 October 2023 at 17:47:52 UTC, Steven 
Schveighoffer wrote:

 On Saturday, 21 October 2023 at 06:57:30 UTC, Walter Bright 
 wrote:

 On 10/20/2023 9:54 PM, Steven Schveighoffer wrote:

 I'm not sure I understand this. What is `dec()`?


 From https://github.com/John-Colvin/YAIDIP :

 "Consider for example using stream manipulators such as dec 
 and hex for writeln by using an i-string:+"

 ```d
 void fun(int x) {
     writeln(i"$dec$x in hexadecimal is 0x$hex$x.");
     // Lowering: --->
     // 
 writeln(.object.imported!"core.interpolation".InterpolationHeader!("", "dec",
"", "x", " in hexadecimal is 0x", "hex", "", "x", ".")(),
     //     "", dec, "", x, " in hexadecimal is 0x", hex, "", 
 x, ".");
 }
 ```


 Oh I see now what you are looking at! This is just like C++ 
 though.

 ```c++
 std::cout << "value in hex: " << std::hex << 42
           << ", value in decimal: " << std::dec << 42 << 
 std::endl;
 ```

 `dec` is not a function that gets called on the value, it's a 
 manipulator to set a flag that says "now do decimal numbers". 
 It's no different for this example use case.


I think the misleading part of this example is that it uses 
`writeln`. Currently, `writeln` does not support passing this 
kind of "stream manipulator" as an argument--even if you write 
the lowered version out by hand, it will not work the way the 
example implies.

Probably the easiest fix is to replace `writeln` with a made-up 
function like `stream.write`.
Oct 21 2023
prev sibling parent Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Saturday, 21 October 2023 at 02:22:49 UTC, Adam D Ruppe wrote:

 Note to readers: you don't have to believe second-hand 
 nonsense. There's functional implementations to play with 
 already:

 [...]


5. Needs to be fixed with YAIDIP.

Walter, is there currently an implementation of your dip that we 
could try and experiment with?

I'd like to try. As an end user I o ly care about if it works.

I come back to 3 basic requirements :

1. Don't break existing code
2. Be safe
3. Be reasonably performant

That's about it
Oct 21 2023
prev sibling next sibling parent reply Andrea Fontana <nospam example.org> writes:
On Saturday, 21 October 2023 at 01:50:33 UTC, Walter Bright wrote:

 https://github.com/dlang/DIPs/blob/master/DIPs/rejected/DIP1027.md


Can't you "expand" i"..." directly into format(..., ..., ...) 
instead of expanding in a tuple?
Oct 21 2023
parent Hipreme <msnmancini hotmail.com> writes:
On Saturday, 21 October 2023 at 08:25:02 UTC, Andrea Fontana 
wrote:

 On Saturday, 21 October 2023 at 01:50:33 UTC, Walter Bright 
 wrote:

 https://github.com/dlang/DIPs/blob/master/DIPs/rejected/DIP1027.md


 Can't you "expand" i"..." directly into format(..., ..., ...) 
 instead of expanding in a tuple?


People are worried about betterC on that.
To be fair, I'm more worried about having a *simply works* 
feature that would allow me to get a real string when my argument 
receives a string, and having the tuple solution if I want to 
optimize something.
Oct 21 2023
prev sibling next sibling parent reply Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Saturday, 21 October 2023 at 01:50:33 UTC, Walter Bright wrote:


 ...

 1. Is more susceptible to inadvertent matching with the wrong 
 function, because the tuples generated are tuples of strings, 
 integers, and other ordinary types.


2. Requires parsing of format string (always).

Really, except already present *f functions, if you'd like to 
have your own function accepting interpolated strings, DIP 1027 
version would take a lot more time and brain power to write 
compared to YAIDIP.
Oct 21 2023
parent reply Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Saturday, 21 October 2023 at 08:40:11 UTC, Alexandru Ermicioi 
wrote:

 On Saturday, 21 October 2023 at 01:50:33 UTC, Walter Bright 
 wrote:


 ...

 1. Is more susceptible to inadvertent matching with the wrong 
 function, because the tuples generated are tuples of strings, 
 integers, and other ordinary types.


 2. Requires parsing of format string (always).

 Really, except already present *f functions, if you'd like to 
 have your own function accepting interpolated strings, DIP 1027 
 version would take a lot more time and brain power to write 
 compared to YAIDIP.


Is there currently an implementation of 1027 that we can try?
Oct 21 2023
parent Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Saturday, 21 October 2023 at 09:00:54 UTC, Imperatorn wrote:

 Is there currently an implementation of 1027 that we can try?


Dunno if there is.

Still from DIP 1027 it is quite clear that you'd need to parse 
format string somehow. Now this parsing logic could be extracted 
and for example put in phobos, but then we'd stumble on another 
cons for DIP 1027:

3. Requires extra handling code of interpolation strings in 
phobos.

This Walter actually listed wrongly for YAIDIP, albeit for a 
different functionality, so I'd guess extra code for 
interpolation strings for DIP 1027 is also a no go, leaving us 
with original cons I've mentioned.
Oct 21 2023
prev sibling parent Dadoum <dadoum protonmail.com> writes:
On Saturday, 21 October 2023 at 01:50:33 UTC, Walter Bright wrote:

 https://github.com/dlang/DIPs/blob/master/DIPs/rejected/DIP1027.md

 https://github.com/John-Colvin/YAIDIP

 [...]


I personally find the swift approach more natural and pretty.

Instead of using a character like `$` or `%`, it uses `\` as in 
`"Section count: \(sectionCount)"`. \
It does not give a way to format them, so I think that maybe a 
second fmt function should be implemented to wrap toString and 
get something like `"Downloading \(completion.format("%.2f"))%"` 
(here there could be another problem if we can't put quotes in 
here, but I don't know how D parses string in general and if it 
would be a problem).
Oct 21 2023