www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - D std.regex is so slow

reply Daniel Kozak <kozzi11 gmail.com> writes:
https://github.com/mariomka/regex-benchmark#performance

Do you know why?

Here is a code:
https://github.com/mariomka/regex-benchmark/blob/master/d/benchmark.d

I have try it with ldc too, but is still much slower (10x) than 
PHP
Sep 19
next sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
On 19/09/2017 8:53 AM, Daniel Kozak wrote:
 https://github.com/mariomka/regex-benchmark#performance
 
 Do you know why?
 
 Here is a code:
 https://github.com/mariomka/regex-benchmark/blob/master/d/benchmark.d
 
 I have try it with ldc too, but is still much slower (10x) than PHP
Most likely everything below D is being JIT'd.
Sep 19
prev sibling next sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On Tuesday, 19 September 2017 at 07:53:27 UTC, Daniel Kozak wrote:
 https://github.com/mariomka/regex-benchmark#performance

 Do you know why?

 Here is a code:
 https://github.com/mariomka/regex-benchmark/blob/master/d/benchmark.d

 I have try it with ldc too, but is still much slower (10x) than 
 PHP
Well I'd be interested in getting the data. The benchmark looks fine. --- Dmitry Olshansky
Sep 19
parent John Colvin <john.loughran.colvin gmail.com> writes:
On Tuesday, 19 September 2017 at 10:14:05 UTC, Dmitry Olshansky 
wrote:
 On Tuesday, 19 September 2017 at 07:53:27 UTC, Daniel Kozak 
 wrote:
 https://github.com/mariomka/regex-benchmark#performance

 Do you know why?

 Here is a code:
 https://github.com/mariomka/regex-benchmark/blob/master/d/benchmark.d

 I have try it with ldc too, but is still much slower (10x) 
 than PHP
Well I'd be interested in getting the data. The benchmark looks fine. --- Dmitry Olshansky
As mentioned in the REAME: https://raw.githubusercontent.com/mariomka/regex-benchmark/master/input-text.txt
Sep 19
prev sibling next sibling parent reply Jon Degenhardt <jond noreply.com> writes:
On Tuesday, 19 September 2017 at 07:53:27 UTC, Daniel Kozak wrote:
 https://github.com/mariomka/regex-benchmark#performance

 Do you know why?

 Here is a code:
 https://github.com/mariomka/regex-benchmark/blob/master/d/benchmark.d

 I have try it with ldc too, but is still much slower (10x) than 
 PHP
Might get some gain by compiling with '-flto-full'. Was faster on a regex test I ran, though not nearly the deltas shown. --Jon
Sep 19
parent Daniel Kozak <kozzi11 gmail.com> writes:
I have tried it, but does not change anything

On Tue, Sep 19, 2017 at 6:05 PM, Jon Degenhardt via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Tuesday, 19 September 2017 at 07:53:27 UTC, Daniel Kozak wrote:

 https://github.com/mariomka/regex-benchmark#performance

 Do you know why?

 Here is a code:
 https://github.com/mariomka/regex-benchmark/blob/master/d/benchmark.d

 I have try it with ldc too, but is still much slower (10x) than PHP
Might get some gain by compiling with '-flto-full'. Was faster on a regex test I ran, though not nearly the deltas shown. --Jon
Sep 19
prev sibling next sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Tuesday, 19 September 2017 at 07:53:27 UTC, Daniel Kozak wrote:
 https://github.com/mariomka/regex-benchmark#performance

 Do you know why?

 Here is a code:
 https://github.com/mariomka/regex-benchmark/blob/master/d/benchmark.d

 I have try it with ldc too, but is still much slower (10x) than 
 PHP
Looks like they added ldc to it. I'm seeing 3x slower than PHP on Email and URI, but roughly par on IP. It really stands out on the DMD one how much better it does on the IP one than the Email/URI ones.
Sep 19
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On Tuesday, 19 September 2017 at 16:27:51 UTC, jmh530 wrote:
 On Tuesday, 19 September 2017 at 07:53:27 UTC, Daniel Kozak 
 wrote:
 https://github.com/mariomka/regex-benchmark#performance

 Do you know why?

 Here is a code:
 https://github.com/mariomka/regex-benchmark/blob/master/d/benchmark.d

 I have try it with ldc too, but is still much slower (10x) 
 than PHP
Looks like they added ldc to it. I'm seeing 3x slower than PHP on Email and URI, but roughly par on IP. It really stands out on the DMD one how much better it does on the IP one than the Email/URI ones.
IP is detected to be "semi-fixed" thus hitting a nice fast path. Others would have worked with BitNFA, a patch that we had to revert because auto-tester run out of memory.
Sep 19
parent reply Daniel Kozak <kozzi11 gmail.com> writes:
Is there a plan to make BitNFA back? Is possible that newCTFE will improve
problem with memory? Or it is possible to improve those slow cases?

On Tue, Sep 19, 2017 at 8:12 PM, Dmitry Olshansky via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Tuesday, 19 September 2017 at 16:27:51 UTC, jmh530 wrote:

 On Tuesday, 19 September 2017 at 07:53:27 UTC, Daniel Kozak wrote:

 https://github.com/mariomka/regex-benchmark#performance

 Do you know why?

 Here is a code:
 https://github.com/mariomka/regex-benchmark/blob/master/d/benchmark.d

 I have try it with ldc too, but is still much slower (10x) than PHP
Looks like they added ldc to it. I'm seeing 3x slower than PHP on Email and URI, but roughly par on IP. It really stands out on the DMD one how much better it does on the IP one than the Email/URI ones.
IP is detected to be "semi-fixed" thus hitting a nice fast path. Others would have worked with BitNFA, a patch that we had to revert because auto-tester run out of memory.
Sep 19
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On Tuesday, 19 September 2017 at 19:52:57 UTC, Daniel Kozak wrote:
 Is there a plan to make BitNFA back?
Yes, the moment we have CTFE that doesn't leak.
 Is possible that newCTFE will improve problem with memory?
It should but it doesn't support classes and exceptions. I need them.
 Or it is possible to improve those slow cases?
There are many other things but BitNFA is a nobrainer - all the work has been already done and it is super fast where applicable. --- Dmitry Olshansky
Sep 19
parent reply Daniel Kozak <kozzi11 gmail.com> writes:
Hmm. I have tested old version od dmd and ldc with BitNFA, but it does not
help much with those two cases

actual version of ldc:

240.964 - 92
211.433 - 5301
6.76683 - 5

ldc 1.2:
215.158 - 92
190.255 - 5301
6.58155 - 5

So there is some improvments but not as much as I would expected.

On Wed, Sep 20, 2017 at 6:54 AM, Dmitry Olshansky via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Tuesday, 19 September 2017 at 19:52:57 UTC, Daniel Kozak wrote:

 Is there a plan to make BitNFA back?
Yes, the moment we have CTFE that doesn't leak. Is possible that newCTFE will improve problem with memory?

 It should but it doesn't support classes and exceptions. I need them.

 Or it is possible to improve those slow cases?

 There are many other things but BitNFA is a nobrainer - all the work has
 been already done and it is super fast where applicable.

 ---
 Dmitry Olshansky
Sep 20
parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On Wednesday, 20 September 2017 at 10:29:45 UTC, Daniel Kozak 
wrote:
 Hmm. I have tested old version od dmd and ldc with BitNFA, but 
 it does not help much with those two cases

 actual version of ldc:

 240.964 - 92
 211.433 - 5301
 6.76683 - 5

 ldc 1.2:
 215.158 - 92
 190.255 - 5301
 6.58155 - 5

 So there is some improvments but not as much as I would 
 expected.
I'm going to test BitNFA myself but for the curious, the phobos branch is: https://github.com/DmitryOlshansky/phobos/tree/ctfe-bitnfa?files=1
 ---
 Dmitry Olshansky
Sep 20
prev sibling parent 12345swordy <alexanderheistermann gmail.com> writes:
On Tuesday, 19 September 2017 at 07:53:27 UTC, Daniel Kozak wrote:
 https://github.com/mariomka/regex-benchmark#performance

 Do you know why?

 Here is a code:
 https://github.com/mariomka/regex-benchmark/blob/master/d/benchmark.d

 I have try it with ldc too, but is still much slower (10x) than 
 PHP
Irc a rewrite is in the work.
Sep 19