www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Ranges seem awkward to work with

reply Hasen Judy <hasen.judy gmail.com> writes:
Is this is a common beginner issue? I remember using an earlier 
version of D some long time ago and I don't remember seeing this 
concept.

Now, a lot of library functions seem to expect ranges as inputs 
and return ranges as output.

Even parsing a csv line returns a range. And the funny thing is, 
once you loop over it, it's done. You've basically consumed it.

For example, I was having some trouble with the api of the 
std.csv module, so to help me debug, I printed the result of the 
csv. ok, the result seems good. Now I try to use it, for example:

     auto name = row[1];

And to my surprise there's a runtime error, something about range 
something something. I don't even remember what the error was. 
The thing is, it wasn't clear what was going on.

The line was actually more like:

         auto some_var = 
some_function(row[1].some_other_library_method!template_variable);

So because I was calling several library methods on the same 
line, I thought the problem might have something to do with the 
range not exactly matching what the library was expecting. I 
thought maybe row[1] also returned some range instead of a string 
and that range had something wrong with it.

Well, it turned out that my earlier attempt to print the parsed 
csv row resulted in the row being "consumed" and now the row is 
an empty range(!).

Is there a straight forward way to convert a Range to a list 
other than manually doing a foreach?

     string[] items;
     foreach(item; someRangeThing) {
         items ~= item;
     }

I feel like that is a bit of an overkill.
Sep 11
next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Tuesday, 12 September 2017 at 01:13:29 UTC, Hasen Judy wrote:
 Is this is a common beginner issue? I remember using an earlier 
 version of D some long time ago and I don't remember seeing 
 this concept.

 Now, a lot of library functions seem to expect ranges as inputs 
 and return ranges as output.

 Even parsing a csv line returns a range. And the funny thing 
 is, once you loop over it, it's done. You've basically consumed 
 it.

 For example, I was having some trouble with the api of the 
 std.csv module, so to help me debug, I printed the result of 
 the csv. ok, the result seems good. Now I try to use it, for 
 example:

     auto name = row[1];

 And to my surprise there's a runtime error, something about 
 range something something. I don't even remember what the error 
 was. The thing is, it wasn't clear what was going on.

 The line was actually more like:

         auto some_var = 
 some_function(row[1].some_other_library_method!template_variable);

 So because I was calling several library methods on the same 
 line, I thought the problem might have something to do with the 
 range not exactly matching what the library was expecting. I 
 thought maybe row[1] also returned some range instead of a 
 string and that range had something wrong with it.

 Well, it turned out that my earlier attempt to print the parsed 
 csv row resulted in the row being "consumed" and now the row is 
 an empty range(!).

 Is there a straight forward way to convert a Range to a list 
 other than manually doing a foreach?

     string[] items;
     foreach(item; someRangeThing) {
         items ~= item;
     }

 I feel like that is a bit of an overkill.
if `range.save` works use that, otherwise `range.dup` will duplicate the range or `range.array` (from std.array) will enumerate the range into an array (which is pretty much the same as your solution above.
Sep 11
parent Jesse Phillips <Jesse.K.Phillips+D gmail.com> writes:
On Tuesday, 12 September 2017 at 01:18:21 UTC, Nicholas Wilson 
wrote:
 On Tuesday, 12 September 2017 at 01:13:29 UTC, Hasen Judy wrote:
 Is this is a common beginner issue?
if `range.save` works use that, otherwise
std.csv does not, IIRC.
 `range.dup` will duplicate the range
That isn't a range property.
 or `range.array` (from std.array) will enumerate the range into 
 an array (which is pretty much the same as your solution above.
This. Ranges are great in part because they are often lazy, a sequence of operations can be defined but only executed when used. Some operations need the whole collection, eg sort.
Sep 11
prev sibling next sibling parent Moritz Maxeiner <moritz ucworks.org> writes:
On Tuesday, 12 September 2017 at 01:13:29 UTC, Hasen Judy wrote:
 Is this is a common beginner issue? I remember using an earlier 
 version of D some long time ago and I don't remember seeing 
 this concept.
D's ranges can take getting used to, so if you haven't already, these two articles are worth the read to get familiar with them imho [1][2]. One way to look at it is that input ranges (empty,front,popFront) model iteration of the elements of some data source (another is that they model a monotonic advancing data source). [1] http://www.drdobbs.com/architecture-and-design/component-programming-in-d/240008321 [2] https://wiki.dlang.org/Component_programming_with_ranges
Sep 11
prev sibling next sibling parent Mike Parker <aldacron gmail.com> writes:
On Tuesday, 12 September 2017 at 01:13:29 UTC, Hasen Judy wrote:
 Is this is a common beginner issue? I remember using an earlier 
 version of D some long time ago and I don't remember seeing 
 this concept.

 Now, a lot of library functions seem to expect ranges as inputs 
 and return ranges as output.

 Even parsing a csv line returns a range. And the funny thing 
 is, once you loop over it, it's done. You've basically consumed 
 it.

 For example, I was having some trouble with the api of the 
 std.csv module, so to help me debug, I printed the result of 
 the csv. ok, the result seems good. Now I try to use it, for 
 example:

     auto name = row[1];

 And to my surprise there's a runtime error, something about 
 range something something. I don't even remember what the error 
 was. The thing is, it wasn't clear what was going on.

 The line was actually more like:

         auto some_var = 
 some_function(row[1].some_other_library_method!template_variable);

 So because I was calling several library methods on the same 
 line, I thought the problem might have something to do with the 
 range not exactly matching what the library was expecting. I 
 thought maybe row[1] also returned some range instead of a 
 string and that range had something wrong with it.

 Well, it turned out that my earlier attempt to print the parsed 
 csv row resulted in the row being "consumed" and now the row is 
 an empty range(!).

 Is there a straight forward way to convert a Range to a list 
 other than manually doing a foreach?

     string[] items;
     foreach(item; someRangeThing) {
         items ~= item;
     }

 I feel like that is a bit of an overkill.
Don't think of ranges as persistent containers. They aren't. They are transient entities, temporary views of data. If you have a container and want a consumable view of it that doesn't mutate the container itself, produce a range. If you receive a range and want to transform it from a transient entity into something concrete, use std.array.array or a range-based container API. If it helps, Chapter 6: Understanding Ranges from 'Learning D' is available (in somewhat mutilated form) as an article at the publisher's site: https://www.packtpub.com/books/content/understanding-ranges
Sep 11
prev sibling parent reply Azi Hassan <azi.hassan live.fr> writes:
On Tuesday, 12 September 2017 at 01:13:29 UTC, Hasen Judy wrote:
 Now, a lot of library functions seem to expect ranges as inputs 
 and return ranges as output.
Unless I'm mistaken, it was done on purpose to reduce the amount of memory allocations in the standard library so that it becomes nogc-friendly. But yes, you can use std.array.array to turn it into an array, but keep in mind that it does allocate so you might need to watch out if you're dealing with large CSV files.
Sep 12
parent Jonathan M Davis via Digitalmars-d-learn writes:
On Tuesday, September 12, 2017 13:47:47 Azi Hassan via Digitalmars-d-learn 
wrote:
 On Tuesday, 12 September 2017 at 01:13:29 UTC, Hasen Judy wrote:
 Now, a lot of library functions seem to expect ranges as inputs
 and return ranges as output.
Unless I'm mistaken, it was done on purpose to reduce the amount of memory allocations in the standard library so that it becomes nogc-friendly. But yes, you can use std.array.array to turn it into an array, but keep in mind that it does allocate so you might need to watch out if you're dealing with large CSV files.
Ranges were a big thing in Phobos years before nogc came along. They're used because they're an extremely powerful idiom. I don't even recall much in the way of discussions related to their efficiency vs arrays early on. That did come later, and ranges have significantly helped the memory efficiency of D programs regardless of nogc, but that really wasn't the motivating factor. And even now, a lot of the basic uses of ranges _do_ allocate, because they use lambdas that end up with closures being allocated for them (which uses far less memory than allocating a dynamic array but is not nogc). That can be avoided by doing stuff like using static nested functions or functors, but your average rangd-based code doesn't tend to jump through those hoops. Ranges do take some getting used to, but we have them precisely because they're so flexible and powerful. - Jonathan M Davis
Sep 12