• realhet (12/12) Nov 29 2019 Hi,
• Paul Backus (15/27) Nov 29 2019 It's actually a much simpler reason: filter calls .front twice
• realhet (6/12) Nov 29 2019 This makes it clear.
• Paul Backus (3/15) Nov 29 2019 The "cache" wrapper only caches one element at a time. The cached
• realhet (26/27) Nov 29 2019 Unfortunately function purity is not the answer.

realhet <real_het hotmail.com> writes:

Hi,

I have an input range.
I use the map! on it to transform using a function.
Finally I use filter! on the transformed data.

When I do a foreach on this, i noticed, that the transform 
function is called twice for each element.
If I remove the filter! it does only one transform function call 
per element.

Is this because the function is NOT PURE, so Phobos thinks that 
the function can give different results for the same input?

If I modify the function to be pure, will the transform function 
be called only once?

Paul Backus <snarwin gmail.com> writes:

On Friday, 29 November 2019 at 15:30:22 UTC, realhet wrote:
 Hi,

 I have an input range.
 I use the map! on it to transform using a function.
 Finally I use filter! on the transformed data.

 When I do a foreach on this, i noticed, that the transform 
 function is called twice for each element.
 If I remove the filter! it does only one transform function 
 call per element.

 Is this because the function is NOT PURE, so Phobos thinks that 
 the function can give different results for the same input?

 If I modify the function to be pure, will the transform 
 function be called only once?

It's actually a much simpler reason: filter calls .front twice 
for each element in its input (once to check if the value 
satisfies the predicate, and then again to return the value if it 
does), and the range returned by map doesn't save the value of 
.front between calls, so it has to re-compute it each time by 
calling the transform function.

You can work around this using std.algorithm.iteration.cache, a 
range wrapper that calls .front on the underlying range exactly 
once, and then saves the value and reuses it for subsequent calls:

myRange
     .map!transform
     .cache
     .filter!pred
     .each!doSomething;

realhet <real_het hotmail.com> writes:

On Friday, 29 November 2019 at 15:49:24 UTC, Paul Backus wrote:
 It's actually a much simpler reason: filter calls .front twice 
 for each element in its input (once to check if the value 
 satisfies the predicate, and then again to return the value if 
 it does), and the range returned by map doesn't save the value 
 of .front between calls, so it has to re-compute it each time 
 by calling the transform function.

This makes it clear.

In my case a cache which can access all the previous elements 
would be a too big thing, so I will do the filtering in a later 
stage manually. But most importantly, now I know what's going on, 
Thank You!

Paul Backus <snarwin gmail.com> writes:

On Friday, 29 November 2019 at 15:54:13 UTC, realhet wrote:
 On Friday, 29 November 2019 at 15:49:24 UTC, Paul Backus wrote:
 It's actually a much simpler reason: filter calls .front twice 
 for each element in its input (once to check if the value 
 satisfies the predicate, and then again to return the value if 
 it does), and the range returned by map doesn't save the value 
 of .front between calls, so it has to re-compute it each time 
 by calling the transform function.

 This makes it clear.

 In my case a cache which can access all the previous elements 
 would be a too big thing, so I will do the filtering in a later 
 stage manually. But most importantly, now I know what's going 
 on, Thank You!

The "cache" wrapper only caches one element at a time. The cached 
value is discarded after each call to .popFront.

realhet <real_het hotmail.com> writes:

On Friday, 29 November 2019 at 15:30:22 UTC, realhet wrote:
 ...

Unfortunately function purity is not the answer.

I put a very long calculation into the transform function which 
is called from "map!".
And the "filter!" is making the "map!" call my function 2 times:
First for the "filter!" to decide it allows the element or not.
And second when the map! is passing it to the foreach() where I 
write the elements out.

Is there a solution for this performance issue?

Using functional programming, my program looks beautiful. But I 
need it to be optimal too.

//To be easier to understand, I want to make the following:
foreach(a; input){
   auto temp = slowFunction(a);
   if(check(temp))
     writeln(temp);
}

//But the elegant functional form ->
input.map!slowFunction.filter!(a => check(a)).each!writeln;

//Produces the following unoptimal solution:
foreach(a; input){
   if(check(slowFunction(a)))
     writeln(slowFunction(a));
}

Is there a functional way to fix this?

Thank you in advance!