www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Why splitter() doesn't work?

reply "Mehrdad" <wfunction hotmail.com> writes:
Why doesn't code like this work?

chain(repeat("a").take(5), repeat("b").take(5)).splitter("b");
Oct 14 2012
next sibling parent "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Sunday, 14 October 2012 at 09:50:20 UTC, Mehrdad wrote:
 Why doesn't code like this work?

 chain(repeat("a").take(5), repeat("b").take(5)).splitter("b");

There's two versions of splitter, the version that splits a range using an element, and a version that splits a range using another range. The chain you're using creates a range of strings, and you are splitting it using a string, so it appears you are going for the first version. The signature for that version is: auto splitter(Range, Separator)(Range r, Separator s) if (is(typeof(ElementType!Range.init == Separator.init)) && (hasSlicing!Range || isNarrowString!Range)) chain doesn't support slicing (and isn't a narrow string), so you can't split it using this function.
Oct 14 2012
prev sibling next sibling parent "Mehrdad" <wfunction hotmail.com> writes:
On Sunday, 14 October 2012 at 19:26:37 UTC, Peter Alexander wrote:
 On Sunday, 14 October 2012 at 09:50:20 UTC, Mehrdad wrote:
 Why doesn't code like this work?

 chain(repeat("a").take(5), repeat("b").take(5)).splitter("b");

There's two versions of splitter, the version that splits a range using an element, and a version that splits a range using another range. The chain you're using creates a range of strings, and you are splitting it using a string, so it appears you are going for the first version. The signature for that version is: auto splitter(Range, Separator)(Range r, Separator s) if (is(typeof(ElementType!Range.init == Separator.init)) && (hasSlicing!Range || isNarrowString!Range)) chain doesn't support slicing (and isn't a narrow string), so you can't split it using this function.

Yup, but why does it need slicing?
Oct 14 2012
prev sibling next sibling parent "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Sunday, 14 October 2012 at 20:03:52 UTC, Mehrdad wrote:
 On Sunday, 14 October 2012 at 19:26:37 UTC, Peter Alexander 
 wrote:
 On Sunday, 14 October 2012 at 09:50:20 UTC, Mehrdad wrote:
 Why doesn't code like this work?

 chain(repeat("a").take(5), repeat("b").take(5)).splitter("b");

chain doesn't support slicing (and isn't a narrow string), so you can't split it using this function.

Yup, but why does it need slicing?

Well, for example: splitter("hello world", ' ') gives ["hello", "world"] The splits are slices of the original range. splitter could allocate new ranges, but if you want that behaviour then it's better to specify it manually: chain(repeat("a").take(5), repeat("b").take(5)).array().splitter("b"); That works, giving: [["a", "a", "a", "a", "a"], [], [], [], [], []] I wonder if a better design for splitter would automatically allocate an array when the input range doesn't support slicing?
Oct 14 2012
prev sibling next sibling parent "Mehrdad" <wfunction hotmail.com> writes:
On Sunday, 14 October 2012 at 20:09:44 UTC, Peter Alexander wrote:
 Well, for example:

 splitter("hello world", ' ')

 gives

 ["hello", "world"]

 The splits are slices of the original range.

 splitter could allocate new ranges, but if you want that 
 behaviour then it's better to specify it manually:

 chain(repeat("a").take(5), 
 repeat("b").take(5)).array().splitter("b");

 That works, giving:

 [["a", "a", "a", "a", "a"], [], [], [], [], []]

 I wonder if a better design for splitter would automatically 
 allocate an array when the input range doesn't support slicing?

Oh I didn't realize that, it's because it wants to slice the original range... Yeah, I think that might not be such a bad idea. It should be possible to slice infinite ranges too, after all.
Oct 14 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, October 14, 2012 22:13:28 Mehrdad wrote:

 Yeah, I think that might not be such a bad idea. It should be
 possible to slice infinite ranges too, after all.

There's actually some discussion on making it so that you can't. The main problem is that a slice can't be the same type as the original for an infinite range, which tends to screw with things. But if infinite ranges end up losing the ability to be sliced, then other, similar functionality would have to be added, because you _do_ need to be able to take chunks of an infinite range like that. - Jonathan M Davis
Oct 14 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, October 14, 2012 22:09:43 Peter Alexander wrote:
 I wonder if a better design for splitter would automatically
 allocate an array when the input range doesn't support slicing?

We generally try and avoid any kind of allocation like that in std.algorithm. We leave it up to the programmer to do that. Otherwise, you'd end up with all kinds of invisible performance hits. And it's doubly bad for anyone trying to avoid using the GC. - Jonathan M Davis
Oct 14 2012
prev sibling next sibling parent "Mehrdad" <wfunction hotmail.com> writes:
On Sunday, 14 October 2012 at 23:01:02 UTC, Jonathan M Davis 
wrote:
 On Sunday, October 14, 2012 22:09:43 Peter Alexander wrote:
 I wonder if a better design for splitter would automatically
 allocate an array when the input range doesn't support slicing?

We generally try and avoid any kind of allocation like that in std.algorithm. We leave it up to the programmer to do that. Otherwise, you'd end up with all kinds of invisible performance hits. And it's doubly bad for anyone trying to avoid using the GC. - Jonathan M Davis

Well, you don't need to automatically allocate anything. As long as you require the input to be a forward range, you can simply iterate over it over and over again every time you want to return front(). AFAICT no need to store anything anywhere.
Oct 14 2012
prev sibling next sibling parent "Mehrdad" <wfunction hotmail.com> writes:
On Sunday, 14 October 2012 at 22:58:27 UTC, Jonathan M Davis 
wrote:
 On Sunday, October 14, 2012 22:13:28 Mehrdad wrote:

 Yeah, I think that might not be such a bad idea. It should be
 possible to slice infinite ranges too, after all.

There's actually some discussion on making it so that you can't. The main problem is that a slice can't be the same type as the original for an infinite range, which tends to screw with things. But if infinite ranges end up losing the ability to be sliced, then other, similar functionality would have to be added, because you _do_ need to be able to take chunks of an infinite range like that. - Jonathan M Davis

How are you supposed to split a range that doesn't support slicing though? You can't just call array() because it might be too big to be reasonable for fitting it into memory...
Oct 14 2012
prev sibling next sibling parent "Mehrdad" <wfunction hotmail.com> writes:
On Sunday, 14 October 2012 at 23:11:32 UTC, Mehrdad wrote:
 How are you supposed to split a range that doesn't support 
 slicing though?

 You can't just call array() because it might be too big to be 
 reasonable for fitting it into memory...

(Canonical example: splitting a stream by newlines)
Oct 14 2012
prev sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, October 15, 2012 01:11:28 Mehrdad wrote:
 How are you supposed to split a range that doesn't support
 slicing though?
 
 You can't just call array() because it might be too big to be
 reasonable for fitting it into memory...

If you're splitting on an element, slicing isn't necessary. It's only splitting on a range that requires slicing. However, because splitter is lazy, I would think that it would be perfectly possible to define it such that it didn't need slicing. It would still need a forward range (for the same reason that find requires forward ranges if it's not looking for a single element), but I would think that you could define it with just forward ranges. Worst case, the result would have to be a range over Take!Range rather than over slices of Range, and that's basically what infinite ranges have to do when slicing anyway. So, unless I'm missing something, I think that splitter is currently being overly restrictive in its implementation. - Jonathan M Davis
Oct 14 2012