www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - How to use ranges?

reply "Doolan" <Doolan MyCentral.com> writes:
I keep running into areas of my code where it looks like I'd 
benefit from using ranges, and then I try to do some range stuff 
and my compiler tells me I'm using the wrong types, or there's 
this problem, or that problem... so I'm scared off and I just 
figure ways to work around using ranges.

I've tried reading the documentation, but for a language priding 
itself on being readable and nice to look at, it really should 
have less complicated docs:

auto tee(Flag!"pipeOnPop" pipeOnPop = Yes.pipeOnPop, R1, R2)(R1 
inputRange, R2 outputRange) if (isInputRange!R1 && 
isOutputRange!(R2, ElementType!R1));
auto tee(alias fun, Flag!"pipeOnPop" pipeOnPop = Yes.pipeOnPop, 
R1)(R1 inputRange) if (is(typeof(fun) == void) || 
isSomeFunction!fun);

(That's fine as secondary information, but this is the heading 
for this function...)

And the use of auto everywhere makes it really hard to tell what 
types I should be using for anything. My compiler talks about 
RangeT!(whatever) but you try to use RangeT!(whatever) and you 
find out RangeT is private...

I don't mean to complain so hard, I obviously like D enough to 
want to use it, just the current amount/layout of documentation 
can be frustrating at times.

Can someone give me a short summary of how to use ranges? And how 
do they relate to slices? That line is really blurry...
Aug 23 2015
next sibling parent reply Rikki Cattermole <alphaglosined gmail.com> writes:
On 8/23/2015 10:17 PM, Doolan wrote:

 I keep running into areas of my code where it looks like I'd benefit
 from using ranges, and then I try to do some range stuff and my compiler
 tells me I'm using the wrong types, or there's this problem, or that
 problem... so I'm scared off and I just figure ways to work around using
 ranges.

 I've tried reading the documentation, but for a language priding itself
 on being readable and nice to look at, it really should have less
 complicated docs:

 auto tee(Flag!"pipeOnPop" pipeOnPop = Yes.pipeOnPop, R1, R2)(R1
 inputRange, R2 outputRange) if (isInputRange!R1 && isOutputRange!(R2,
 ElementType!R1));
 auto tee(alias fun, Flag!"pipeOnPop" pipeOnPop = Yes.pipeOnPop, R1)(R1
 inputRange) if (is(typeof(fun) == void) || isSomeFunction!fun);

 (That's fine as secondary information, but this is the heading for this
 function...)

 And the use of auto everywhere makes it really hard to tell what types I
 should be using for anything. My compiler talks about RangeT!(whatever)
 but you try to use RangeT!(whatever) and you find out RangeT is private...

 I don't mean to complain so hard, I obviously like D enough to want to
 use it, just the current amount/layout of documentation can be
 frustrating at times.

 Can someone give me a short summary of how to use ranges? And how do
 they relate to slices? That line is really blurry...


Have a read of: 
https://github.com/rikkimax/twp-d/blob/master/manuscript/content/idioms/ranges.md

Let me know what you think :)
Aug 23 2015
parent reply "Doolan" <Doolan MyCentral.com> writes:
On Sunday, 23 August 2015 at 10:38:53 UTC, Rikki Cattermole wrote:

 On 8/23/2015 10:17 PM, Doolan wrote:
 Have a read of: 
 https://github.com/rikkimax/twp-d/blob/master/manuscript/content/idioms/ranges.md

 Let me know what you think :)


I think I'm a little more confused than before... this says there 
are two kinds of ranges and I thought there were five??
Aug 23 2015
parent Rikki Cattermole <alphaglosined gmail.com> writes:
On 8/24/2015 12:52 AM, Doolan wrote:

 On Sunday, 23 August 2015 at 10:38:53 UTC, Rikki Cattermole wrote:

 On 8/23/2015 10:17 PM, Doolan wrote:
 Have a read of:
 https://github.com/rikkimax/twp-d/blob/master/manuscript/content/idioms/ranges.md


 Let me know what you think :)


 I think I'm a little more confused than before... this says there are
 two kinds of ranges and I thought there were five??


There are more, but input/output ranges are the core two.
They underpin all the others.

Generally speaking these are the two that you will be using. The others 
such as ForwardRange are really just optional extras.
Aug 23 2015
prev sibling next sibling parent reply anonymous <anonymous example.com> writes:
On Sunday 23 August 2015 12:17, Doolan wrote:


 And the use of auto everywhere makes it really hard to tell what
 types I should be using for anything. My compiler talks about
 RangeT!(whatever) but you try to use RangeT!(whatever) and you
 find out RangeT is private...


You can use typeof to get the type of a range expression when typing it out 
is impractical/impossible.


 Can someone give me a short summary of how to use ranges?


I'm not sure what exactly you're looking for. The documentation for tee has 
some example code. How about you show something you're having trouble with?


 And how do they relate to slices? That line is really blurry...


"Slice" is a synonym for what the spec calls a "dynamic array", i.e. a 
structure containing a pointer and a length.

"Slicing" a dynamic array, static array, or pointer produces a dynamic 
array, referencing (not copying) the sliced elements:

----
int[] d = [1, 2, 3, 4]; /* dynamic array */
int[] slice = d[1 .. 3];
assert(slice == [2, 3]);

d[1] = 20;
assert(slice[0] == 20);

int[4] s = [1, 2, 3, 4]; /* static array */
slice = s[];
assert(slice == [1, 2, 3, 4]);

int* p = [1, 2, 3, 4].ptr; /* pointer */
slice = p[1 .. 3];
assert(slice == [2, 3]);
----

Note that the source types are different, but slicing them yields the same 
type every time: int[].

Generally, dynamic arrays / slices are random-access ranges. Narrow strings 
(string/wstring/char[]/wchar[]/...) are a notable exception to this. They 
are dynamic arrays of UTF-8/UTF-16 code units. But they're not random-access 
ranges of Unicode code units. Instead, they're _forward_ ranges of Unicode 
code _points_ (dchar). They have special range primitives that to the 
decoding.
Aug 23 2015
parent reply "Doolan" <Doolan MyCentral.com> writes:
On Sunday, 23 August 2015 at 13:46:30 UTC, anonymous wrote:

 On Sunday 23 August 2015 12:17, Doolan wrote:


 And the use of auto everywhere makes it really hard to tell 
 what types I should be using for anything. My compiler talks 
 about RangeT!(whatever) but you try to use RangeT!(whatever) 
 and you find out RangeT is private...


 You can use typeof to get the type of a range expression when 
 typing it out is impractical/impossible.


What if I want to save a range in a struct? Or is a range more of 
a verb than a noun..?

 Can someone give me a short summary of how to use ranges?


 I'm not sure what exactly you're looking for. The documentation 
 for tee has some example code. How about you show something 
 you're having trouble with?


The possibility of using ranges comes up a lot, but here's 
today's example:

I need to compress some data, and luckily it's very suited for 
Running Length Encoding, so I've gone with doing that. 
Occasionally changes need to be made to this data and so rather 
than extracting the data, changing it, and then recompressing it, 
I can just do the equivalent of leaving a post-it note reminding 
the decompression function to sprinkle these changes in after 
decompression. Occasionally, I also need to grab some values out 
of this data without decompressing it, but for every additional 
value I look for I have to search the post-it notes and it gets a 
little fiddly.

So, I vaguely know what ranges are, and I've heard you can chain 
them together, and my code would be much more readable if I could 
cut up access to the data and splice in changes... but I don't 
even know how to define a range of the right type...


 And how do they relate to slices? That line is really blurry...


 "Slice" is a synonym for what the spec calls a "dynamic array", 
 i.e. a structure containing a pointer and a length.

 "Slicing" a dynamic array, static array, or pointer produces a 
 dynamic array, referencing (not copying) the sliced elements:

 ----
 int[] d = [1, 2, 3, 4]; /* dynamic array */
 int[] slice = d[1 .. 3];
 assert(slice == [2, 3]);

 d[1] = 20;
 assert(slice[0] == 20);

 int[4] s = [1, 2, 3, 4]; /* static array */
 slice = s[];
 assert(slice == [1, 2, 3, 4]);

 int* p = [1, 2, 3, 4].ptr; /* pointer */
 slice = p[1 .. 3];
 assert(slice == [2, 3]);
 ----

 Note that the source types are different, but slicing them 
 yields the same type every time: int[].

 Generally, dynamic arrays / slices are random-access ranges. 
 Narrow strings (string/wstring/char[]/wchar[]/...) are a 
 notable exception to this. They are dynamic arrays of 
 UTF-8/UTF-16 code units. But they're not random-access ranges 
 of Unicode code units. Instead, they're _forward_ ranges of 
 Unicode code _points_ (dchar). They have special range 
 primitives that to the decoding.


So slices are random-access ranges... I understand the 
random-access part... but are they inheriting from Range, do they 
just include a Range? Why is int[] an array when I declare, but 
variable[] a Range?? Or isn't it a Range?
Aug 23 2015
next sibling parent "Gary Willoughby" <dev nomad.so> writes:
On Sunday, 23 August 2015 at 17:58:44 UTC, Doolan wrote:

 ...


Read this for a nice introduction: 
http://ddili.org/ders/d.en/ranges.html
Then watch this: https://www.youtube.com/watch?v=A8Btr8TPJ8c
Aug 23 2015
prev sibling next sibling parent "John Colvin" <john.loughran.colvin gmail.com> writes:
 Generally, dynamic arrays / slices are random-access ranges. 
 Narrow strings (string/wstring/char[]/wchar[]/...) are a 
 notable exception to this. They are dynamic arrays of 
 UTF-8/UTF-16 code units. But they're not random-access ranges 
 of Unicode code units. Instead, they're _forward_ ranges of 
 Unicode code _points_ (dchar). They have special range 
 primitives that to the decoding.


 So slices are random-access ranges... I understand the 
 random-access part... but are they inheriting from Range, do 
 they just include a Range? Why is int[] an array when I 
 declare, but variable[] a Range?? Or isn't it a Range?


A range is just any type that satisfies certain properties (e.g. 
has front, empty, popFront in the most basic case). int[] is a 
range because front, empty and popFront are defined for it in 
std.range, which can be called with "uniform function call 
syntax" (UFCS) as if they were members of the type int[] itself.
Aug 23 2015
prev sibling next sibling parent "ZombineDev" <valid_email he.re> writes:
On Sunday, 23 August 2015 at 17:58:44 UTC, Doolan wrote:

 ...


Ali's book has a very nice chapter about ranges:
http://ddili.org/ders/d.en/ranges.html
Aug 23 2015
prev sibling parent anonymous <anonymous example.com> writes:
On Sunday 23 August 2015 19:58, Doolan wrote:


 You can use typeof to get the type of a range expression when
 typing it out is impractical/impossible.


 What if I want to save a range in a struct? Or is a range more of
 a verb than a noun..?


Can still use typeof then:
----
struct S
{
    import std.range: tee;
    import std.stdio: writeln;
    
    typeof((int[]).init.tee!writeln) teeing;
    
    this(int[] a)
    {
        teeing = a.tee!writeln;
    }
}

void main()
{
    auto s = S([1, 2, 3, 4]);
    foreach(x; s.teeing) {}
}
----

Alternatively you can get classy and use std.range.interfaces:

----
struct S
{
    import std.range: tee;
    import std.range.interfaces: InputRange, inputRangeObject;
    import std.stdio: writeln;
    
    InputRange!int teeing;
    
    this(int[] a)
    {
        teeing = inputRangeObject(a.tee!writeln);
    }
}

void main()
{
    auto s = S([1, 2, 3, 4]);
    foreach(x; s.teeing) {}
}
----


 I need to compress some data, and luckily it's very suited for
 Running Length Encoding, so I've gone with doing that.
 Occasionally changes need to be made to this data and so rather
 than extracting the data, changing it, and then recompressing it,
 I can just do the equivalent of leaving a post-it note reminding
 the decompression function to sprinkle these changes in after
 decompression. Occasionally, I also need to grab some values out
 of this data without decompressing it, but for every additional
 value I look for I have to search the post-it notes and it gets a
 little fiddly.


Here's a quick implementation of a run-length decoder range, and some 
example usage.

I'm not sure how to go about the post-it note concept, or how practical it 
is. The decoder is not a random-access range, so when you want to edit the 
data and access individual items, then decode-edit-encode may work better. 
You don't have to shoehorn something into ranges just because ranges are 
cool.

But if you want to apply a bunch of transformations to the whole thing, then 
ranges shine.

----
import std.range: empty, front, popFront, save;
import std.stdio;

struct RunLengthItem(T)
{
    T value;
    size_t length;
}

struct RunLengthDecoder(T)
{
    RunLengthItem!T[] items;
    size_t currentRun = 0;
    
     property bool empty() const {return items.empty;}
     property T front() {return items.front.value;}
    void popFront()
    {
        ++currentRun;
        if(currentRun >= items.front.length)
        {
            items.popFront();
            currentRun = 0;
        }
    }
     property RunLengthDecoder save() {return this;}
}

void main()
{
    /* two 'f's, three 'o's, five 'b's, seven 'a's, eleven 'r's */
    RunLengthItem!dchar[] data = [
        RunLengthItem!dchar('f', 2),
        RunLengthItem!dchar('o', 3),
        RunLengthItem!dchar('b', 5),
        RunLengthItem!dchar('a', 7),
        RunLengthItem!dchar('r', 11)
    ];
    
    auto decoder = RunLengthDecoder!dchar(data);
    
    /* Just print the elements (writeln is aware of ranges): */
    writeln(decoder.save); /* ffooobbbbbaaaaaaarrrrrrrrrrr */
    
    /* Uppercase when printing: */
    import std.uni: asUpperCase;
    writeln(decoder.save.asUpperCase); /* FFOOOBBBBBAAAAAAARRRRRRRRRRR */
    
    /* Filter out the 'b's, uppercase, and put an underscore every five 
characters: */
    import std.algorithm: filter, joiner;
    import std.range: chunks;
    auto r = decoder.save
        .filter!(c => c != 'b')
        .asUpperCase
        .chunks(5).joiner("_");
    writeln(r); /* FFOOO_AAAAA_AARRR_RRRRR_RRR */
}
----



 So, I vaguely know what ranges are, and I've heard you can chain
 them together, and my code would be much more readable if I could
 cut up access to the data and splice in changes... but I don't
 even know how to define a range of the right type...


Sounds like you may have some specific code you could use help with. If so, 
don't be afraid to post that code and ask how it can be range-ified.


 So slices are random-access ranges... I understand the
 random-access part... but are they inheriting from Range, do they
 just include a Range?


There's no inheritance going on. Inheritance is a class thing. Arrays/slices 
are not classes.

std.range defines the necessary range operations for slices: empty, front, 
popFront, etc. That's how slices are ranges.


 Why is int[] an array when I declare, but
 variable[] a Range?? Or isn't it a Range?


Every int[] is a dynamic array, a slice, and a range. Doesn't matter where 
that int[] comes from.
Aug 23 2015
prev sibling parent "welkam" <wwwelkam gmail.com> writes:
There was a talk on ranges in Dconf 2015
https://www.youtube.com/watch?v=A8Btr8TPJ8c&list=PLEDeq48KhndP-mlE-0Bfb_qPIMA4RrrKo&index=10
Aug 23 2015