www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Using chunks with Generator

reply Tetiana <tetiana.khot gmail.com> writes:
I'd like to evaluate values lazily and then process them one by 
one, using e.g. `map`. One of the steps requires processing the 
values in chunks. I was thinking of using a Generator for lazy 
evaluation, but I can't seem to make it work with the `chunks`.

How can I make `std.range.chunks` work with a generator?


import std.stdio;
import std.concurrency: Generator, yield;
import std.range:chunks;
import std.algorithm:map;


void main()
{

     auto gen = new Generator!int(
     {
         foreach (i; 1 .. 10)
             yield(i);
     });


     foreach (n; chunks(gen, 2))
     {
         writeln(n);
     }
}

Build fails with the following error:

source/app.d(18,23): Error: template std.range.chunks cannot 
deduce function from argument types !()(Generator!int, int), 
candidates are:
/usr/include/dmd/phobos/std/range/package.d(6940,15):        
std.range.chunks(Source)(Source source, size_t chunkSize) if 
(isForwardRange!Source)
dmd failed with exit code 1.
Mar 14
parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Tuesday, 14 March 2017 at 10:00:24 UTC, Tetiana wrote:
 Build fails with the following error:
Just looking at the Documentation, Generator is an InputRange and chunks requires a ForwardRange (able to use save functionality). So the API doesn't match up more or less. https://dlang.org/phobos/std_concurrency.html#.Generator https://dlang.org/phobos/std_range.html#chunks
Mar 14
parent "H. S. Teoh via Digitalmars-d-learn" <digitalmars-d-learn puremagic.com> writes:
On Tue, Mar 14, 2017 at 12:30:13PM +0000, Era Scarecrow via Digitalmars-d-learn
wrote:
 On Tuesday, 14 March 2017 at 10:00:24 UTC, Tetiana wrote:
 Build fails with the following error:
Just looking at the Documentation, Generator is an InputRange and chunks requires a ForwardRange (able to use save functionality). So the API doesn't match up more or less. https://dlang.org/phobos/std_concurrency.html#.Generator https://dlang.org/phobos/std_range.html#chunks
You probably need to cache the generated elements so that they can be reused. Or you could try using std.algorithm.iteration.chunkBy instead, which *can* work with mere input ranges (that aren't necessarily forward ranges). Note, though, that you would only be able to evaluate each element once, and calling popFront on the outer range will discard all elements in the current chunk. This is generally OK for single-pass algorithms, but if you ever need to backtrack or process chunk elements out-of-order, it won't work and you'll need to cache. T -- Help a man when he is in trouble and he will remember you when he is in trouble again.
Mar 14