digitalmars.D.learn - Why is sformat and formattedWrite (appender) allocating GC mem here?
- cc (45/45) Aug 31 2019 And what, if anything, can I do to avoid it?
- Mike Parker (4/5) Aug 31 2019 import core.stdc.stdio : printf;
- ag0aep6g (12/16) Aug 31 2019 `formatElement` does something like that. It writes the string into a
- Nicholas Wilson (2/4) Aug 31 2019 Merged.
- cc (3/5) Sep 01 2019 Thanks for the responses, very cool seeing these updates happen
- James Blachly (4/6) Sep 03 2019 Wonderful!
- ag0aep6g (4/6) Sep 03 2019 `sformat` uses the GC less now, but it's not @nogc. It can still throw
And what, if anything, can I do to avoid it? string[] arr = ["abcd", "efgh", "ijkl"]; char[4096] buf; char[] res; writeln(GC.stats); res = sformat(buf, "%s", arr); assert(res.ptr == buf.ptr); writeln(res); writeln(GC.stats); res = sformat(buf, "%s", arr); assert(res.ptr == buf.ptr); writeln(res); writeln(GC.stats); // Results: Stats(64, 1048512, 80) ["abcd", "efgh", "ijkl"] Stats(272, 1048304, 272) ["abcd", "efgh", "ijkl"] Stats(464, 1048112, 464) I get similar behavior trying to use formattedWrite with an appender: string[] arr = ["abcd", "efgh", "ijkl"]; auto formatBuffer = appender!(char[])(); formatBuffer.reserve(4096); formatBuffer.clear(); writeln(GC.stats); formatBuffer.formattedWrite!"%s"(arr); writeln(formatBuffer.data); writeln(GC.stats); formatBuffer.clear(); formatBuffer.formattedWrite!"%s"(arr); writeln(formatBuffer.data); writeln(GC.stats); // Results: Stats(12288, 5230592, 4208) ["abcd", "efgh", "ijkl"] Stats(12432, 5230448, 4352) ["abcd", "efgh", "ijkl"] Stats(12576, 5230304, 4496) Same thing if I use a format string like "%(%s,%)" rather than just "%s". I'm guessing it's allocating a string first to write the contents of the array and then inserting that string into the buffer I supplied. Is there no way to have it skip this step and just write each element (plus the joining punctuation, etc) directly into the buffer?
Aug 31 2019
On Saturday, 31 August 2019 at 12:07:56 UTC, cc wrote:And what, if anything, can I do to avoid it?import core.stdc.stdio : printf; There are no nogc versions of the Phobos write* and format functions.
Aug 31 2019
On 31.08.19 14:07, cc wrote:I'm guessing it's allocating a string first to write the contents of the array and then inserting that string into the buffer I supplied. Is there no way to have it skip this step and just write each element (plus the joining punctuation, etc) directly into the buffer?`formatElement` does something like that. It writes the string into a temporary buffer while looking for invalid Unicode. When it finds some, the temporary is discarded and the whole string is formatted differently. When the string is a-ok, the data is copied over to the actual destination. I'm not sure if that's the best approach, though. The temporary buffer and the string copy are costly. There is also a closure being allocated for no reason in `sformat` itself. The compiler isn't smart enough to see that it's not really needed. I've made a pull request to get rid of those allocations: https://github.com/dlang/phobos/pull/7163
Aug 31 2019
On Saturday, 31 August 2019 at 21:12:32 UTC, ag0aep6g wrote:I've made a pull request to get rid of those allocations: https://github.com/dlang/phobos/pull/7163Merged.
Aug 31 2019
On Saturday, 31 August 2019 at 21:12:32 UTC, ag0aep6g wrote:I've made a pull request to get rid of those allocations: https://github.com/dlang/phobos/pull/7163Thanks for the responses, very cool seeing these updates happen so fluidly.
Sep 01 2019
On 8/31/19 5:12 PM, ag0aep6g wrote:I've made a pull request to get rid of those allocations: https://github.com/dlang/phobos/pull/7163Wonderful! For my own learning, why was a unittest to ensure no GC added to sformat instead of a nogc annotation?
Sep 03 2019
On 03.09.19 16:03, James Blachly wrote:For my own learning, why was a unittest to ensure no GC added to sformat instead of a nogc annotation?`sformat` uses the GC less now, but it's not nogc. It can still throw GC-allocated exceptions, e.g. when the arguments don't match the format string.
Sep 03 2019