www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Efficient outputting of to-string conversions

reply Tomek =?ISO-8859-2?B?U293afFza2k=?= <just ask.me> writes:
Looks like std.conv.to always allocates behind the scenes. It's a shame as the
returned string is immediately processed and discarded in my XML writer. Are
there plans to include a custom output variant, e.g. to!string(7, outputRange)?

-- 
Tomek
Feb 07 2011
next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday 07 February 2011 13:10:09 Tomek Sowi=C5=84ski wrote:
 Looks like std.conv.to always allocates behind the scenes. It's a shame as
 the returned string is immediately processed and discarded in my XML
 writer. Are there plans to include a custom output variant, e.g.
 to!string(7, outputRange)?

http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP9
Feb 07 2011
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/7/11 4:43 PM, Jonathan M Davis wrote:
 On Monday 07 February 2011 13:10:09 Tomek SowiƄski wrote:
 Looks like std.conv.to always allocates behind the scenes. It's a shame as
 the returned string is immediately processed and discarded in my XML
 writer. Are there plans to include a custom output variant, e.g.
 to!string(7, outputRange)?

http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP9

Yah, we should hop on that. The DIP is concerned mainly with toString, not std.conv. For the latter, Tomek's idea of passing an output range as an optional second parameter seems appropriate. Please file as an enhancement to bugzilla. If anyone has time to work on this, please do. If not, I'll work on it as my schedule allows. Andrei
Feb 07 2011
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/7/11 5:38 PM, Tomek Sowiński wrote:
 Jonathan M Davis napisał:

 On Monday 07 February 2011 13:10:09 Tomek Sowiński wrote:
 Looks like std.conv.to always allocates behind the scenes. It's a shame as
 the returned string is immediately processed and discarded in my XML
 writer. Are there plans to include a custom output variant, e.g.
 to!string(7, outputRange)?

http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP9

I know about Steven's proposal but it applies only to user types not primitives. Either way std.conv.to would need a buffered output range as integers are written from the right. Any chance for an abstraction analogous to buffered input ranges discussed recently?

Generally I found it more difficult to define a solid output buffer abstraction. This is a great motivating example though. To my surprise, an API of the same form seems to be what the doctor prescribed. Here's a semi-formal definition: A buffered output range R is defined as such: R.front returns the currently uncommitted buffer of type T[] R.moreFront(n) makes n more elements available for writing R.commitFront(n) writes the first n elements in front() R.flushFront() writes the buffer currently held in front() and makes another buffer available (initially empty). Andrei
Feb 07 2011
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/7/11 7:25 PM, Andrei Alexandrescu wrote:
 On 2/7/11 5:38 PM, Tomek Sowiński wrote:
 Jonathan M Davis napisał:

 On Monday 07 February 2011 13:10:09 Tomek Sowiński wrote:
 Looks like std.conv.to always allocates behind the scenes. It's a
 shame as
 the returned string is immediately processed and discarded in my XML
 writer. Are there plans to include a custom output variant, e.g.
 to!string(7, outputRange)?

http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP9

I know about Steven's proposal but it applies only to user types not primitives. Either way std.conv.to would need a buffered output range as integers are written from the right. Any chance for an abstraction analogous to buffered input ranges discussed recently?

Generally I found it more difficult to define a solid output buffer abstraction. This is a great motivating example though. To my surprise, an API of the same form seems to be what the doctor prescribed.

I meant "of the same form as buffered input ranges". The primitives are similar: expose a buffer, allow it to grow, and discard it progressively. Andrei
Feb 07 2011
prev sibling parent spir <denis.spir gmail.com> writes:
On 02/07/2011 11:38 PM, Tomek SowiƄski wrote:
 Jonathan M Davis napisaƂ:

 On Monday 07 February 2011 13:10:09 Tomek SowiƄski wrote:
 Looks like std.conv.to always allocates behind the scenes. It's a shame as
 the returned string is immediately processed and discarded in my XML
 writer. Are there plans to include a custom output variant, e.g.
 to!string(7, outputRange)?

http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP9

I know about Steven's proposal but it applies only to user types not primitives. Either way std.conv.to would need a buffered output range as integers are written from the right. Any chance for an abstraction analogous to buffered input ranges discussed recently?

I haven't read a solution like the proposal for writeTo would not apply to primitives. Is it really so? (If only for language consistency, I would prefere this big shift to apply to all types.) -- _________________ vita es estrany spir.wikidot.com
Feb 08 2011
prev sibling next sibling parent Tomek =?ISO-8859-2?B?U293afFza2k=?= <just ask.me> writes:
Jonathan M Davis napisa=B3:

 On Monday 07 February 2011 13:10:09 Tomek Sowi=F1ski wrote:
 Looks like std.conv.to always allocates behind the scenes. It's a shame=


 the returned string is immediately processed and discarded in my XML
 writer. Are there plans to include a custom output variant, e.g.
 to!string(7, outputRange)?

http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP9

I know about Steven's proposal but it applies only to user types not primit= ives. Either way std.conv.to would need a buffered output range as integers= are written from the right. Any chance for an abstraction analogous to buf= fered input ranges discussed recently? --=20 Tomek
Feb 07 2011
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday 08 February 2011 07:55:23 spir wrote:
 On 02/07/2011 11:38 PM, Tomek Sowi=C5=84ski wrote:
 Jonathan M Davis napisa=C5=82:
 On Monday 07 February 2011 13:10:09 Tomek Sowi=C5=84ski wrote:
 Looks like std.conv.to always allocates behind the scenes. It's a sha=




 as the returned string is immediately processed and discarded in my
 XML writer. Are there plans to include a custom output variant, e.g.
 to!string(7, outputRange)?

http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP9

I know about Steven's proposal but it applies only to user types not primitives. Either way std.conv.to would need a buffered output range as integers are written from the right. Any chance for an abstraction analogous to buffered input ranges discussed recently?

I haven't read a solution like the proposal for writeTo would not apply to primitives. Is it really so? (If only for language consistency, I would prefere this big shift to apply to all types.)

Umm. _How_ would it apply to primitives. Does _toString_ apply to primitive= s?=20 No. toString and writeTo apply to user-defined types that they're defined o= n. That=20 doesn't mean that we can't or shouldn't find a buffered solution for dealin= g with=20 primitives and I/O similar to writeTo, but primitives can't have writeTo de= fined=20 on them any more than they can have toString defined on them. =2D Jonathan M Davis
Feb 08 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 08 Feb 2011 10:55:23 -0500, spir <denis.spir gmail.com> wrote:

 On 02/07/2011 11:38 PM, Tomek SowiƄski wrote:
 Jonathan M Davis napisaƂ:

 On Monday 07 February 2011 13:10:09 Tomek SowiƄski wrote:
 Looks like std.conv.to always allocates behind the scenes. It's a  
 shame as
 the returned string is immediately processed and discarded in my XML
 writer. Are there plans to include a custom output variant, e.g.
 to!string(7, outputRange)?

http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP9

I know about Steven's proposal but it applies only to user types not primitives. Either way std.conv.to would need a buffered output range as integers are written from the right. Any chance for an abstraction analogous to buffered input ranges discussed recently?

I haven't read a solution like the proposal for writeTo would not apply to primitives. Is it really so? (If only for language consistency, I would prefere this big shift to apply to all types.)

writeTo does not need to apply to primitives because primitives can be written to streams without first converting to char[]. For instance, one does not write: writeln(to!string(5)) The whole point of writeTo is to allow easy output of custom data types that the standard library does not know how to output. It was not meant to extend to primitives. As far as Tomek's request, I would expect std.format.formattedWrite(buffer, "%s", 7) to work, or std.format.formatValue (with associated FormatSpec, which I don't feel like looking up the usage of) -Steve
Feb 08 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 08 Feb 2011 11:09:44 -0500, Steven Schveighoffer  
<schveiguy yahoo.com> wrote:

 On Tue, 08 Feb 2011 10:55:23 -0500, spir <denis.spir gmail.com> wrote:

 On 02/07/2011 11:38 PM, Tomek SowiƄski wrote:
 Jonathan M Davis napisaƂ:

 On Monday 07 February 2011 13:10:09 Tomek SowiƄski wrote:
 Looks like std.conv.to always allocates behind the scenes. It's a  
 shame as
 the returned string is immediately processed and discarded in my XML
 writer. Are there plans to include a custom output variant, e.g.
 to!string(7, outputRange)?

http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP9

I know about Steven's proposal but it applies only to user types not primitives. Either way std.conv.to would need a buffered output range as integers are written from the right. Any chance for an abstraction analogous to buffered input ranges discussed recently?

I haven't read a solution like the proposal for writeTo would not apply to primitives. Is it really so? (If only for language consistency, I would prefere this big shift to apply to all types.)

writeTo does not need to apply to primitives because primitives can be written to streams without first converting to char[]. For instance, one does not write: writeln(to!string(5)) The whole point of writeTo is to allow easy output of custom data types that the standard library does not know how to output. It was not meant to extend to primitives. As far as Tomek's request, I would expect std.format.formattedWrite(buffer, "%s", 7) to work, or std.format.formatValue (with associated FormatSpec, which I don't feel like looking up the usage of)

BTW, I think we probably should have a formatValue override like this void formatValue(Char, Writer, T)(Writer w, T t) { FormatSpec!Char fs; // use default options formatValue(w, t, fs); } Having to construct a FormatSpec when I just want the default seems like overkill. Esp. since FormatSpec arg is always ref. -Steve
Feb 08 2011
prev sibling next sibling parent Tomek =?ISO-8859-2?B?U293afFza2k=?= <just ask.me> writes:
Andrei Alexandrescu napisa=B3:

 I know about Steven's proposal but it applies only to user types not pr=


gers are written from the right. Any chance for an abstraction analogous to= buffered input ranges discussed recently? =20
=20
 Generally I found it more difficult to define a solid output buffer=20
 abstraction. This is a great motivating example though.
=20
 To my surprise, an API of the same form seems to be what the doctor=20
 prescribed. Here's a semi-formal definition:
=20
 A buffered output range R is defined as such:
=20
 R.front returns the currently uncommitted buffer of type T[]
=20
 R.moreFront(n) makes n more elements available for writing
=20
 R.commitFront(n) writes the first n elements in front()
=20
 R.flushFront() writes the buffer currently held in front() and makes=20
 another buffer available (initially empty).

I was thinking along the same lines. There's one missing: R.skipFront(n) skips the first n elements without outputting Why? Look at integral conversions in std.conv.to. It first calculates maxim= um string size, then writes numbers to the char array back to front, then r= eturns result[$ - ndigits .. $] where ndigits is how long the string turned= out. Returning to Steven's DIP, I think writeTo should take the above rather tha= n void delegate(char[]). With the latter you still have to allocate the pie= ces. Our buffered output range is friends with polymorphism too. If you set= T=3Dchar, its API is devoid of generics. Such interface can be placed in o= bject.d with an official blessing. --=20 Tomek
Feb 08 2011
prev sibling parent Tomek =?ISO-8859-2?B?U293afFza2k=?= <just ask.me> writes:
Andrei Alexandrescu napisa=B3:

 For the latter, Tomek's idea of passing an output range as=20
 an optional second parameter seems appropriate. Please file as an=20
 enhancement to bugzilla. If anyone has time to work on this, please do.=20
 If not, I'll work on it as my schedule allows.

http://d.puremagic.com/issues/show_bug.cgi?id=3D5548 --=20 Tomek
Feb 08 2011