www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Lazy formatting

reply Steven Schveighoffer <schveiguy gmail.com> writes:
As a response to Andrei in DIP 1036 thread (as this is somewhat off topic):


 On 1/31/21 11:11 AM, Steven Schveighoffer wrote:

 This is how I would do it:

 i"Hello, ${var.withFormat!"%45s"} world";

 (where withFormat provides a toString overload that works with idup). No extra
intermediate allocations necessary.


 
 I wonder if we can define a nice convention by the use of past tense vs.
active.
 
 format!"%45s"(val) -> format using this spec the following values
 
 val.formatted!"%45s" -> value formatted with this spec
 
 We have just a bit of that with sort/sorted.
 
 Maybe flesh that up in phobos 2.0 :o).
 


I have often wanted a lazy formatting feature where you aren't 
allocating but it returns a string-able thing that can be used as an 
input range. I was surprised that phobos doesn't provide this.

Would be really awesome to have, but I think possibly it means we need 
to reengineer the lower level. Probably a phobos 2.0 thing.

-Steve
Jan 31 2021
next sibling parent reply Der <der1029384576 hotmail.com> writes:
On Monday, 1 February 2021 at 00:25:42 UTC, Steven Schveighoffer 
wrote:

 Probably a phobos 2.0 thing.


Is this under serious consideration, or is it merely the 
optimists' fervent dream.
Jan 31 2021
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/31/21 7:43 PM, Der wrote:

 On Monday, 1 February 2021 at 00:25:42 UTC, Steven Schveighoffer wrote:

 Probably a phobos 2.0 thing.


 
 Is this under serious consideration, or is it merely the optimists' 
 fervent dream.


Walter, Andrei, Atila and I are discussing for about 4 months. I have 
only been involved since then, not sure what the plans were before that.

This is not a facetious thing. We are trying to determine the best way 
to create a phobos v2 that does not break existing code. I don't know 
what it will look like, or if we can come up with something that is 
workable soon. But we are working on it. The thing that I'm most 
interested in is no autodecoding. But of course, with a place where we 
can create breaking changes that live in an "opt-in" space, it opens the 
door to a lot of other things that we may want to include.

-Steve
Jan 31 2021
parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Monday, 1 February 2021 at 02:49:37 UTC, Steven Schveighoffer 
wrote:

 On 1/31/21 7:43 PM, Der wrote:

 On Monday, 1 February 2021 at 00:25:42 UTC, Steven 
 Schveighoffer wrote:

 Probably a phobos 2.0 thing.


 
 Is this under serious consideration, or is it merely the 
 optimists' fervent dream.


 Walter, Andrei, Atila and I are discussing for about 4 months. 
 I have only been involved since then, not sure what the plans 
 were before that.

 This is not a facetious thing. We are trying to determine the 
 best way to create a phobos v2 that does not break existing 
 code. I don't know what it will look like, or if we can come up 
 with something that is workable soon. But we are working on it. 
 The thing that I'm most interested in is no autodecoding. But 
 of course, with a place where we can create breaking changes 
 that live in an "opt-in" space, it opens the door to a lot of 
 other things that we may want to include.

 -Steve


I would like to have a clear path to _increment_ it right now:

https://github.com/dlang/phobos/pull/6730

A clear policy and a final word simple additions like that would 
be great.
Feb 02 2021
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/2/21 3:51 AM, Paolo Invernizzi wrote:

 On Monday, 1 February 2021 at 02:49:37 UTC, Steven Schveighoffer wrote:

 On 1/31/21 7:43 PM, Der wrote:

 On Monday, 1 February 2021 at 00:25:42 UTC, Steven Schveighoffer wrote:

 Probably a phobos 2.0 thing.


 Is this under serious consideration, or is it merely the optimists' 
 fervent dream.


 Walter, Andrei, Atila and I are discussing for about 4 months. I have 
 only been involved since then, not sure what the plans were before that.

 This is not a facetious thing. We are trying to determine the best way 
 to create a phobos v2 that does not break existing code. I don't know 
 what it will look like, or if we can come up with something that is 
 workable soon. But we are working on it. The thing that I'm most 
 interested in is no autodecoding. But of course, with a place where we 
 can create breaking changes that live in an "opt-in" space, it opens 
 the door to a lot of other things that we may want to include.


 
 I would like to have a clear path to _increment_ it right now:
 
 https://github.com/dlang/phobos/pull/6730
 
 A clear policy and a final word simple additions like that would be great.


That's a prime example of what a phobos v2 would allow. No code 
breakage, and provides a saner interface for future projects.

-Steve
Feb 02 2021
parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Tuesday, 2 February 2021 at 18:50:00 UTC, Steven Schveighoffer 
wrote:

 On 2/2/21 3:51 AM, Paolo Invernizzi wrote:

 On Monday, 1 February 2021 at 02:49:37 UTC, Steven 
 Schveighoffer wrote:

[...]


 
 I would like to have a clear path to _increment_ it right now:
 
 https://github.com/dlang/phobos/pull/6730
 
 A clear policy and a final word simple additions like that 
 would be great.


 That's a prime example of what a phobos v2 would allow. No code 
 breakage, and provides a saner interface for future projects.

 -Steve


I underline that adding a new better method, is not a breakage ...
Feb 02 2021
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/2/21 2:50 PM, Paolo Invernizzi wrote:

 On Tuesday, 2 February 2021 at 18:50:00 UTC, Steven Schveighoffer wrote:

 On 2/2/21 3:51 AM, Paolo Invernizzi wrote:

 On Monday, 1 February 2021 at 02:49:37 UTC, Steven Schveighoffer wrote:

 [...]


 I would like to have a clear path to _increment_ it right now:

 https://github.com/dlang/phobos/pull/6730

 A clear policy and a final word simple additions like that would be 
 great.


 That's a prime example of what a phobos v2 would allow. No code 
 breakage, and provides a saner interface for future projects.


 
 I underline that adding a new better method, is not a breakage ...
 


Adding  nogc to a method that's overridable that was previously not 
 nogc is breakage. That's the point of the folks in that review.

-Steve
Feb 02 2021
parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Tuesday, 2 February 2021 at 20:03:18 UTC, Steven Schveighoffer 
wrote:

 On 2/2/21 2:50 PM, Paolo Invernizzi wrote:

 On Tuesday, 2 February 2021 at 18:50:00 UTC, Steven 
 Schveighoffer wrote:

 On 2/2/21 3:51 AM, Paolo Invernizzi wrote:

 On Monday, 1 February 2021 at 02:49:37 UTC, Steven 
 Schveighoffer wrote:

 [...]


 I would like to have a clear path to _increment_ it right 
 now:

 https://github.com/dlang/phobos/pull/6730

 A clear policy and a final word simple additions like that 
 would be great.


 That's a prime example of what a phobos v2 would allow. No 
 code breakage, and provides a saner interface for future 
 projects.


 
 I underline that adding a new better method, is not a breakage 
 ...
 


 Adding  nogc to a method that's overridable that was previously 
 not  nogc is breakage. That's the point of the folks in that 
 review.

 -Steve


I agree with you, that's why I simply suggested to _add_ a new 
better method, and pointed also to precedents for that practise 
in Phobos ...

But despite that, the point is NOT having a policy for such 
common improvements, and that why there was a call to authority 
...
Feb 02 2021
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/2/21 3:08 PM, Paolo Invernizzi wrote:

 On Tuesday, 2 February 2021 at 20:03:18 UTC, Steven Schveighoffer wrote:

 Adding  nogc to a method that's overridable that was previously not 
  nogc is breakage. That's the point of the folks in that review.


 
 I agree with you, that's why I simply suggested to _add_ a new better 
 method, and pointed also to precedents for that practise in Phobos ...
 
 But despite that, the point is NOT having a policy for such common 
 improvements, and that why there was a call to authority ...


OK, I was misunderstanding what you were saying (I may still not 
understand it), but yes, we are trying to find a way to do these kinds 
of improvements.

-Steve
Feb 02 2021
parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Tuesday, 2 February 2021 at 20:33:03 UTC, Steven Schveighoffer 
wrote:

 On 2/2/21 3:08 PM, Paolo Invernizzi wrote:

 On Tuesday, 2 February 2021 at 20:03:18 UTC, Steven 
 Schveighoffer wrote:

 Adding  nogc to a method that's overridable that was 
 previously not  nogc is breakage. That's the point of the 
 folks in that review.


 
 I agree with you, that's why I simply suggested to _add_ a new 
 better method, and pointed also to precedents for that 
 practise in Phobos ...
 
 But despite that, the point is NOT having a policy for such 
 common improvements, and that why there was a call to 
 authority ...


 OK, I was misunderstanding what you were saying (I may still 
 not understand it),


Sorry for my bad phrasing ... you are understanding it rightly :-P


 but yes, we are trying to find a way to do  these kinds of 
 improvements.


Steven, it's a two years waiting for a trivial addition to an 
obsoleted (definition coming from the PR discussion) module.

This thing start to smell more like "we don't care" that "we are 
carefully studying how hight to put the bar"
Feb 03 2021
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/3/21 4:03 AM, Paolo Invernizzi wrote:

 On Tuesday, 2 February 2021 at 20:33:03 UTC, Steven Schveighoffer wrote:

 but yes, we are trying to find a way to do  these kinds of improvements.


 
 Steven, it's a two years waiting for a trivial addition to an obsoleted 
 (definition coming from the PR discussion) module.
 
 This thing start to smell more like "we don't care" that "we are 
 carefully studying how hight to put the bar"


Not sure how to answer this. I only got involved 4 months ago. I can't 
spend full time on it, I have an actual job. So I'm sparsely working on it.

And it's important to prove that what we want to do is feasible and 
workable with our system of development. That's not an easy task, I've 
already hit some dead ends.

-Steve
Feb 03 2021
parent Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Wednesday, 3 February 2021 at 14:34:42 UTC, Steven 
Schveighoffer wrote:

 On 2/3/21 4:03 AM, Paolo Invernizzi wrote:

 On Tuesday, 2 February 2021 at 20:33:03 UTC, Steven 
 Schveighoffer wrote:

 but yes, we are trying to find a way to do  these kinds of 
 improvements.


 
 Steven, it's a two years waiting for a trivial addition to an 
 obsoleted (definition coming from the PR discussion) module.
 
 This thing start to smell more like "we don't care" that "we 
 are carefully studying how hight to put the bar"


 Not sure how to answer this. I only got involved 4 months ago. 
 I can't spend full time on it, I have an actual job. So I'm 
 sparsely working on it.


I definitely understand that, don't worry.


 And it's important to prove that what we want to do is feasible 
 and workable with our system of development. That's not an easy 
 task, I've already hit some dead ends.


That's why defining policies and procedures is the way to go: 
they open up to scalability.

That specific PR is just asking a very, very tiny and specific 
question. The answer to that can be generalised, and then can be 
generally applied to similar situation without posing the burden 
to reviewers.

I will try to make this discussion constructive: I've seen Atila 
has commented the PR, and the PR was labeled "Andrej" ... I will 
ping again Atila on that, let's see if we can score a goal here.
Feb 03 2021
prev sibling next sibling parent reply Kagamin <spam here.lot> writes:
 This is how I would do it:

 i"Hello, ${var.withFormat!"%45s"} world";


I have a feeling recent DIPs are too detached from practice. If 
you read feature requests, people wanted interpolated strings in 
order to easily construct strings without tinkering. For ease of 
use. But the DIP's design rationale ended up being "because other 
languages have this". As design thus loses direction you end up 
designing an abstract silver bullet of interpolated strings 
without regard for practical needs. But we already have plenty of 
ways to tinker with string formatting. Who asked for one more way 
to do it? Almost all uses of interpolated strings are trivial 
ones i"I have ${n} apples".
Feb 01 2021
next sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 1 February 2021 at 13:31:48 UTC, Kagamin wrote:

 i"Hello, ${var.withFormat!"%45s"} world";


 I have a feeling recent DIPs are too detached from practice.
 [...]
 Who asked for one more way to do it? Almost all uses of 
 interpolated strings are trivial ones i"I have ${n} apples".


In ruby and php and stuff you can do

"I have ${some_function()} apples".

That's what withFormat is, simply a function you're calling in 
there which does a format.

The DIP itself just allows for

i"I have ${d_code_here} apples."


and d_code_here can be a simple variable or it can be a function 
call or other fancy things. All the same feature.
Feb 01 2021
prev sibling next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 02/02/2021 2:31 AM, Kagamin wrote:

 Almost all uses of interpolated strings are trivial ones i"I have ${n} 
 apples".


SQL is a good example which is not trivial due to anti-injection 
techniques like escaping.
Feb 01 2021
parent Kagamin <spam here.lot> writes:
On Monday, 1 February 2021 at 13:40:22 UTC, rikki cattermole 
wrote:

 On 02/02/2021 2:31 AM, Kagamin wrote:

 Almost all uses of interpolated strings are trivial ones i"I 
 have ${n} apples".


 SQL is a good example which is not trivial due to 
 anti-injection techniques like escaping.


Such queries usually appear in tests, where you work with trusted 
data. In any case %45s means nothing for sql, and you can't use 
anything like that anyway.
Feb 01 2021
prev sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/1/21 8:31 AM, Kagamin wrote:

 This is how I would do it:

 i"Hello, ${var.withFormat!"%45s"} world";


 
 I have a feeling recent DIPs are too detached from practice. If you read 
 feature requests, people wanted interpolated strings in order to easily 
 construct strings without tinkering. For ease of use. But the DIP's 
 design rationale ended up being "because other languages have this". As 
 design thus loses direction you end up designing an abstract silver 
 bullet of interpolated strings without regard for practical needs. But 
 we already have plenty of ways to tinker with string formatting. Who 
 asked for one more way to do it? Almost all uses of interpolated strings 
 are trivial ones i"I have ${n} apples".


I'm not sure how you read this as "cuz other langs do it": 
https://github.com/dlang/DIPs/blob/344e00ee2d6683d61ee019d5ef6c1a0646570093/DIPs/DIP1036.md#rationale

But in any case, the `withFormat` thing has nothing to do with the 
referenced DIP. That's why we're in a separate thread here.

I have wanted in the past to just have a construct that says "format 
this value", but not have to decide where it goes. In other words, a 
lazy formatting.

All of std.format expects a *destination* to put formatted data. I want 
the *source* to just be formatted data.

It's actually a lower level abstraction than what is already in 
std.format, which could be built upon such a thing. Just an idea any way 
(and of course, it's relevant for string interpolation to anyone who 
might want a one-off format difference than the default).

-Steve
Feb 01 2021
prev sibling next sibling parent reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Monday, 1 February 2021 at 00:25:42 UTC, Steven Schveighoffer 
wrote:

 I have often wanted a lazy formatting feature where you aren't 
 allocating but it returns a string-able thing that can be used 
 as an input range. I was surprised that phobos doesn't provide 
 this.

 Would be really awesome to have, but I think possibly it means 
 we need to reengineer the lower level. Probably a phobos 2.0 
 thing.


Um, do you mean something like this?

https://github.com/CyberShadow/ae/blob/904d5b6ab7a15b3ecac499ef41bb40a88fd64e1e/utils/text/package.d#L47

assert(64.formatted!"%x".text == "40");

It's not an input range (I think emitting formatted strings as an 
input range without an intermediary buffer would be impractically 
difficult in general), but it does use the sink toString 
interface.
Feb 01 2021
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/1/21 10:28 AM, Vladimir Panteleev wrote:

 On Monday, 1 February 2021 at 00:25:42 UTC, Steven Schveighoffer wrote:

 I have often wanted a lazy formatting feature where you aren't 
 allocating but it returns a string-able thing that can be used as an 
 input range. I was surprised that phobos doesn't provide this.

 Would be really awesome to have, but I think possibly it means we need 
 to reengineer the lower level. Probably a phobos 2.0 thing.


 
 Um, do you mean something like this?
 
 https://github.com/CyberShadow/ae/blob/904d5b6ab7a15b3ecac499ef41bb40a88fd64e1e/util
/text/package.d#L47 
 
 
 assert(64.formatted!"%x".text == "40");
 
 It's not an input range (I think emitting formatted strings as an input 
 range without an intermediary buffer would be impractically difficult in 
 general), but it does use the sink toString interface.


Oh nice, yes! That would actually work with the DIP as defined.

Yeah, you definitely need a buffer for formatting, it's somewhat 
possible to do with integers, but not as pleasant.

But local buffers can be part of the type anyway (like a static array).

-Steve
Feb 01 2021
prev sibling parent Berni44 <someone somemail.com> writes:
On Monday, 1 February 2021 at 00:25:42 UTC, Steven Schveighoffer 
wrote:

 I have often wanted a lazy formatting feature where you aren't 
 allocating but it returns a string-able thing that can be used 
 as an input range. I was surprised that phobos doesn't provide 
 this.


Most implementations in std.format produce the result from left 
to right. It looks like this has been done on purpose, though I 
do not fully understand why. Aside from a few corner cases it's 
mainly formatting floating point numbers, that does not do so 
yet. Pondering whether it's possible to making the floatingpoint 
algorithms completely  nogc I meanwhile think it might be 
possible to produce the results from left to right here too (with 
some local buffers and/or RYU printf or something similar). With 
that, it shouldn't be too difficult to put everything into a lazy 
range. (But still, it's a long road to go.)

Anyway I wonder a little bit what use cases there are for such a 
range?
Feb 02 2021