www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - jai-like CTFE string formating

reply Stefan Koch <uplink.coder googlemail.com> writes:
Hi Guys,

I've just implemented a subset of the std.format functionality.
In the same style that Johnathan Blow uses for JAI.

It's about 10x faster then using std.format and uses much less 
memory :)

the follwing code takes over 250 ms to compile :
{
     import std.format;
     pragma(msg, format("Hello %s %s %s %s %s %s %s %s %s", " I ", 
  "just", " have", " to", " concat",  " a lot", " of", " strings 
...", 9));
}
Whereas the following alternative takes 20 ms :
{
     import ctfe_utils;
     pragma(msg, format_jai("Hello % % % % % % % % %", " I ", " 
just", " have" , " to", " concat", " a lot", " of", " strings 
...", 9));
}

see for yourself:
https://www.youtube.com/watch?v=T0BJxdt61RY
Aug 12
next sibling parent tetyys <tetyys tetyys.com> writes:
On Saturday, 12 August 2017 at 11:47:10 UTC, Stefan Koch wrote:
 [...]
very nice
Aug 12
prev sibling next sibling parent reply ixid <nuaccount gmail.com> writes:
On Saturday, 12 August 2017 at 11:47:10 UTC, Stefan Koch wrote:
 Whereas the following alternative takes 20 ms :
 {
     import ctfe_utils;
     pragma(msg, format_jai("Hello % % % % % % % % %", " I ", " 
 just", " have" , " to", " concat", " a lot", " of", " strings 
 ...", 9));
 }

 see for yourself:
 https://www.youtube.com/watch?v=T0BJxdt61RY
Could we not improve the syntax to something like: pragma(msg, format_jai("Hello %9", " I ", " just", " have" , " to", " concat", " a lot", " of", " strings ...")); } Or whatever is closest that doesn't step on the current print syntax.
Aug 12
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 12 August 2017 at 13:19:12 UTC, ixid wrote:
 On Saturday, 12 August 2017 at 11:47:10 UTC, Stefan Koch wrote:
 Whereas the following alternative takes 20 ms :
 {
     import ctfe_utils;
     pragma(msg, format_jai("Hello % % % % % % % % %", " I ", " 
 just", " have" , " to", " concat", " a lot", " of", " strings 
 ...", 9));
 }

 see for yourself:
 https://www.youtube.com/watch?v=T0BJxdt61RY
Could we not improve the syntax to something like: pragma(msg, format_jai("Hello %9", " I ", " just", " have" , " to", " concat", " a lot", " of", " strings ...")); } Or whatever is closest that doesn't step on the current print syntax.
The point of format is that you want to replace the % my the argument and then write more static text. "The Parameter '%' was null and it should not be" "Rule '%' was not found in catalog '%' is possible the closest match"
Aug 12
parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 13 August 2017 at 00:15, Stefan Koch via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Saturday, 12 August 2017 at 13:19:12 UTC, ixid wrote:

 On Saturday, 12 August 2017 at 11:47:10 UTC, Stefan Koch wrote:

 Whereas the following alternative takes 20 ms :
 {
     import ctfe_utils;
     pragma(msg, format_jai("Hello % % % % % % % % %", " I ", " just", "
 have" , " to", " concat", " a lot", " of", " strings ...", 9));
 }

 see for yourself:
 https://www.youtube.com/watch?v=T0BJxdt61RY
Could we not improve the syntax to something like: pragma(msg, format_jai("Hello %9", " I ", " just", " have" , " to", " concat", " a lot", " of", " strings ...")); } Or whatever is closest that doesn't step on the current print syntax.
The point of format is that you want to replace the % my the argument and then write more static text. "The Parameter '%' was null and it should not be" "Rule '%' was not found in catalog '%' is possible the closest match"
If you're keen to introduce a new function, I'd strongly suggest changing to {1} {2} {3}, or %1 %2 %3, format/printf functions where you don't supply the place index are useless a lot of the time and need to be rewritten, which is a tedious task.
Aug 12
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 13 August 2017 at 00:42:08 UTC, Manu wrote:
 On 13 August 2017 at 00:15, Stefan Koch via Digitalmars-d <
 [ ... ]
If you're keen to introduce a new function, I'd strongly suggest changing to {1} {2} {3}, or %1 %2 %3, format/printf functions where you don't supply the place index are useless a lot of the time and need to be rewritten, which is a tedious task.
I am not too keen introducing a new function, given that it most likely won't be accepted into phobos. However your suggestion is indeed a good suggestion. The check would then see if the number of args are equal to the max index. And it would check that all indices from 0 to maxIdx are used at least once. Thanks, Stefan
Aug 12
parent Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 13 August 2017 at 12:07, Stefan Koch via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Sunday, 13 August 2017 at 00:42:08 UTC, Manu wrote:

 On 13 August 2017 at 00:15, Stefan Koch via Digitalmars-d <

 [ ... ]
If you're keen to introduce a new function, I'd strongly suggest changing to {1} {2} {3}, or %1 %2 %3, format/printf functions where you don't supply the place index are useless a lot of the time and need to be rewritten, which is a tedious task.
I am not too keen introducing a new function, given that it most likely won't be accepted into phobos. However your suggestion is indeed a good suggestion. The check would then see if the number of args are equal to the max index. And it would check that all indices from 0 to maxIdx are used at least once.
No, it's not always the case that a format string uses every argument. Format strings are usually taken from the translation table. Different languages have different word orders (which is why placeholders need to accept an arg index), and occasionally, some translations don't use all the args because the sentence structure doesn't make sense, or excess args are supplied because one language requires a string that others don't use.
Aug 13
prev sibling next sibling parent reply =?UTF-8?Q?S=c3=b6nke_Ludwig?= <sludwig+d outerproduct.org> writes:
Am 12.08.2017 um 13:47 schrieb Stefan Koch:
 Hi Guys,
 
 I've just implemented a subset of the std.format functionality.
 In the same style that Johnathan Blow uses for JAI.
 
 It's about 10x faster then using std.format and uses much less memory :)
 
 the follwing code takes over 250 ms to compile :
 {
      import std.format;
      pragma(msg, format("Hello %s %s %s %s %s %s %s %s %s", " I ", 
   "just", " have", " to", " concat",  " a lot", " of", " strings ...", 9));
 }
 Whereas the following alternative takes 20 ms :
 {
      import ctfe_utils;
      pragma(msg, format_jai("Hello % % % % % % % % %", " I ", " just", " 
 have" , " to", " concat", " a lot", " of", " strings ...", 9));
 }
 
 see for yourself:
 https://www.youtube.com/watch?v=T0BJxdt61RY
I was a bit shocked by this number and performed a little test with multiple calls to format. Fortunately only the first one takes that long to compile. For 500 different calls I got about 2ms per call on Windows (which might get slowed down somewhat by the slow terminal output): import std.format; template T(size_t i) { static if (i > 0) { pragma(msg, format("Bye %s %s %s %s %s %s %s %s %s %s", i, " I ", "just", " have", " to", " concat", " a lot", " of", " strings ...", 9)); enum T = T!(i-1); } else enum T = 0; } enum test = T!500;
Aug 13
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 13 August 2017 at 18:20:15 UTC, Sönke Ludwig wrote:
 Am 12.08.2017 um 13:47 schrieb Stefan Koch:
 Hi Guys,
 
 I've just implemented a subset of the std.format functionality.
 In the same style that Johnathan Blow uses for JAI.
 
 It's about 10x faster then using std.format and uses much less 
 memory :)
 
 the follwing code takes over 250 ms to compile :
 {
      import std.format;
      pragma(msg, format("Hello %s %s %s %s %s %s %s %s %s", " 
 I ",
   "just", " have", " to", " concat",  " a lot", " of", " 
 strings ...", 9));
 }
 Whereas the following alternative takes 20 ms :
 {
      import ctfe_utils;
      pragma(msg, format_jai("Hello % % % % % % % % %", " I ", 
 " just", " have" , " to", " concat", " a lot", " of", " 
 strings ...", 9));
 }
 
 see for yourself:
 https://www.youtube.com/watch?v=T0BJxdt61RY
I was a bit shocked by this number and performed a little test with multiple calls to format. Fortunately only the first one takes that long to compile. For 500 different calls I got about 2ms per call on Windows (which might get slowed down somewhat by the slow terminal output): import std.format; template T(size_t i) { static if (i > 0) { pragma(msg, format("Bye %s %s %s %s %s %s %s %s %s %s", i, " I ", "just", " have", " to", " concat", " a lot", " of", " strings ...", 9)); enum T = T!(i-1); } else enum T = 0; } enum test = T!500;
Wouldn't you say that 2ms is still a pretty long time for a function call ?
Aug 13
parent =?UTF-8?Q?S=c3=b6nke_Ludwig?= <sludwig+d outerproduct.org> writes:
Am 13.08.2017 um 20:25 schrieb Stefan Koch:
 On Sunday, 13 August 2017 at 18:20:15 UTC, Sönke Ludwig wrote:
 I was a bit shocked by this number and performed a little test with 
 multiple calls to format. Fortunately only the first one takes that 
 long to compile. For 500 different calls I got about 2ms per call on 
 Windows (which might get slowed down somewhat by the slow terminal 
 output):

     import std.format;
     template T(size_t i) {
       static if (i > 0) {
         pragma(msg, format("Bye %s %s %s %s %s %s %s %s %s %s",
           i, " I ",  "just", " have", " to", " concat",  " a lot",
           " of", " strings ...", 9));
         enum T = T!(i-1);
       } else enum T = 0;
     }
     enum test = T!500;
Wouldn't you say that 2ms is still a pretty long time for a function call ?
For this particular function call, yes, but still a lot less scary ;) To get some better numbers, can you do a side by side comparison of your two versions both executed a few hundred times? If the simplified version still turns out considerably faster then there may be some low hanging fruits in vibe.d's web/REST modules.
Aug 13
prev sibling parent reply Jerry <hurricane hereiam.com> writes:
Seems like it'd be a good idea to pre compute all of phobos for 
compile time computations, as they should be changing. That would 
drastically reduce using any of phobos for compile time 
computation.
Aug 13
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 13 August 2017 at 19:47:37 UTC, Jerry wrote:
 Seems like it'd be a good idea to pre compute all of phobos for 
 compile time computations, as they should be changing. That 
 would drastically reduce using any of phobos for compile time 
 computation.
You cannot do that. The point of template code is that the user can change it from the outside. There is no way to precompute the reaction to things that are yet to be realized. The way compile time is saved here is by limiting the flexibility.
Aug 13
parent Jerry <hurricane hereiam.com> writes:
On Sunday, 13 August 2017 at 19:51:31 UTC, Stefan Koch wrote:
 On Sunday, 13 August 2017 at 19:47:37 UTC, Jerry wrote:
 Seems like it'd be a good idea to pre compute all of phobos 
 for compile time computations, as they should be changing. 
 That would drastically reduce using any of phobos for compile 
 time computation.
You cannot do that. The point of template code is that the user can change it from the outside. There is no way to precompute the reaction to things that are yet to be realized. The way compile time is saved here is by limiting the flexibility.
There are ways to do it, not having to parse the code would make it faster on its own. Compiling 500 source files individually that starts and stops DMD is drastically slower than putting the 500 source files together and running DMD once. The tests that are run for DMD for example could be made to run much faster.
Aug 19