www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - std.string.join & co. do a character-by-character copy because

reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
Why hasLength!string is false:  
https://github.com/D-Programming-Language/phobos/blob/96941d5384a5fee302df/std/range.d#L767

Would it make sense to introduce a hasOpaqueLength oslt. to fix this?

-- 
Best regards,
  Vladimir                            mailto:vladimir thecybershadow.net
May 27 2011
next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On 2011-05-27 11:05, Vladimir Panteleev wrote:
 Why hasLength!string is false:
 https://github.com/D-Programming-Language/phobos/blob/96941d5384a5fee302df/
 std/range.d#L767
 
 Would it make sense to introduce a hasOpaqueLength oslt. to fix this?

Um. Why? What would it give you? hasLength is intended for range-based stuff. and and any arrays of char or wchar are ranges of dchar, so they don't have a length property as far as ranges are concerned. If you want your template to work with any of the string types, then just use isSomeString to check whether it's a string. What would hasOpaqueLength do? What would it get you? Yes, the situation with char and wchar technically having a length property but having hasLength be false for them is a bit odd, but it works, and hasLength does exactly what it's supposed to - check whether the type has a length property for use with ranges. And char and wchar arrays don't. - Jonathan M Davis
May 27 2011
prev sibling next sibling parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Fri, 27 May 2011 21:18:40 +0300, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:

 On 2011-05-27 11:05, Vladimir Panteleev wrote:
 Why hasLength!string is false:
 https://github.com/D-Programming-Language/phobos/blob/96941d5384a5fee302df/
 std/range.d#L767

 Would it make sense to introduce a hasOpaqueLength oslt. to fix this?

Um. Why? What would it give you? hasLength is intended for range-based stuff. and and any arrays of char or wchar are ranges of dchar, so they don't have a length property as far as ranges are concerned. If you want your template to work with any of the string types, then just use isSomeString to check whether it's a string. What would hasOpaqueLength do? What would it get you? Yes, the situation with char and wchar technically having a length property but having hasLength be false for them is a bit odd, but it works, and hasLength does exactly what it's supposed to - check whether the type has a length property for use with ranges. And char and wchar arrays don't.

Wow, you sure took your time to bash my stupid suggestion :) I agree completely, TBH I only added that because I didn't want my post to look like a complaint without even an attempt to suggest a solution. As long as std.array.join & co. don't do a character-by-character copy for strings, I'm happy. Still, putting isSomeString in std.array/std.range checks seems like a hack to me. -- Best regards, Vladimir mailto:vladimir thecybershadow.net
May 27 2011
prev sibling next sibling parent reply KennyTM~ <kennytm gmail.com> writes:
On May 28, 11 02:05, Vladimir Panteleev wrote:
 Why hasLength!string is false:
 https://github.com/D-Programming-Language/phobos/blob/96941d5384a5fee302df/std/range.d#L767


 Would it make sense to introduce a hasOpaqueLength oslt. to fix this?

You mean std.array.join? The 2-argument overload has specialization for strings (https://github.com/D-Programming-Language/phobos/blob/master/std/array.d#L966). static if (isForwardRange!RoR && hasLength!RoR && (hasLength!(ElementType!RoR) || isSomeString!(ElementType!RoR)) // ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ && hasLength!R) It doesn't exist for the 1-argument overload though.
May 27 2011
next sibling parent KennyTM~ <kennytm gmail.com> writes:
On May 28, 11 02:51, Vladimir Panteleev wrote:
 On Fri, 27 May 2011 21:29:10 +0300, KennyTM~ <kennytm gmail.com> wrote:

 On May 28, 11 02:05, Vladimir Panteleev wrote:
 Why hasLength!string is false:
 https://github.com/D-Programming-Language/phobos/blob/96941d5384a5fee302df/std/range.d#L767



 Would it make sense to introduce a hasOpaqueLength oslt. to fix this?

You mean std.array.join? The 2-argument overload has specialization for strings (https://github.com/D-Programming-Language/phobos/blob/master/std/array.d#L966). static if (isForwardRange!RoR && hasLength!RoR && (hasLength!(ElementType!RoR) || isSomeString!(ElementType!RoR)) // ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ && hasLength!R) It doesn't exist for the 1-argument overload though.

Thanks! But it looks like the 2-argument overload is broken too. It doesn't check if the separator is a string, so that branch isn't taken for join(string[], string). In fact, it doesn't work at all when the second argument is an iterator with length: C:\Soft\dmd2\src\phobos\std\array.d(778): Error: template std.algorithm.copy(Range1,Range2) if (isInputRange!(Range1) && isOutputRange!(Range2,ElementType!(Range1))) does not match any function template declaration C:\Soft\dmd2\src\phobos\std\array.d(778): Error: template std.algorithm.copy(Range1,Range2) if (isInputRange!(Range1) && isOutputRange!(Range2,ElementType!(Range1))) cannot deduce template function from argument types !()(Result,string) test26.d(9): Error: template instance std.array.join!(string[],Map!(result,string)) error instantiating Program: import std.array; import std.algorithm; void main() { string[] arr; string sep; join(arr, map!"a"(sep)); }

You're right. File a bug report?
May 27 2011
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/27/11 2:00 PM, Jonathan M Davis wrote:
 On 2011-05-27 11:51, Vladimir Panteleev wrote:

 Create a bug report and I'll look into it.

 - Jonathan M Davis

Thank you, Vladimir and even more so Jonathan! Andrei
May 27 2011
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On 2011-05-27 11:25, Vladimir Panteleev wrote:
 On Fri, 27 May 2011 21:18:40 +0300, Jonathan M Davis <jmdavisProg gmx.com>
 
 wrote:
 On 2011-05-27 11:05, Vladimir Panteleev wrote:
 Why hasLength!string is false:
 https://github.com/D-Programming-Language/phobos/blob/96941d5384a5fee302
 df/ std/range.d#L767
 
 Would it make sense to introduce a hasOpaqueLength oslt. to fix this?

Um. Why? What would it give you? hasLength is intended for range-based stuff. and and any arrays of char or wchar are ranges of dchar, so they don't have a length property as far as ranges are concerned. If you want your template to work with any of the string types, then just use isSomeString to check whether it's a string. What would hasOpaqueLength do? What would it get you? Yes, the situation with char and wchar technically having a length property but having hasLength be false for them is a bit odd, but it works, and hasLength does exactly what it's supposed to - check whether the type has a length property for use with ranges. And char and wchar arrays don't.

Wow, you sure took your time to bash my stupid suggestion :) I agree completely, TBH I only added that because I didn't want my post to look like a complaint without even an attempt to suggest a solution. As long as std.array.join & co. don't do a character-by-character copy for strings, I'm happy. Still, putting isSomeString in std.array/std.range checks seems like a hack to me.

In some cases, where two strings are of the same type, character by character processing can be avoided (it depends on the function), but in many cases, it can't be, and if the string types differ, it definitely can't be. So, it may be that some functions could be improved to handle string processing better, but in the general case, the strings have to be decoded to process them, which means operating on them character by character. But if you notice a function which could and should do a better job with strings and you have an improved implementation, please create a pull request for it. There _are_ quite a few functions, however, which are optimized for string-processing. - Jonathan M Davis
May 27 2011
prev sibling next sibling parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Fri, 27 May 2011 21:29:10 +0300, KennyTM~ <kennytm gmail.com> wrote:

 On May 28, 11 02:05, Vladimir Panteleev wrote:
 Why hasLength!string is false:
 https://github.com/D-Programming-Language/phobos/blob/96941d5384a5fee302df/std/range.d#L767


 Would it make sense to introduce a hasOpaqueLength oslt. to fix this?

You mean std.array.join? The 2-argument overload has specialization for strings (https://github.com/D-Programming-Language/phobos/blob/master/std/array.d#L966). static if (isForwardRange!RoR && hasLength!RoR && (hasLength!(ElementType!RoR) || isSomeString!(ElementType!RoR)) // ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ && hasLength!R) It doesn't exist for the 1-argument overload though.

Thanks! But it looks like the 2-argument overload is broken too. It doesn't check if the separator is a string, so that branch isn't taken for join(string[], string). In fact, it doesn't work at all when the second argument is an iterator with length: C:\Soft\dmd2\src\phobos\std\array.d(778): Error: template std.algorithm.copy(Range1,Range2) if (isInputRange!(Range1) && isOutputRange!(Range2,ElementType!(Range1))) does not match any function template declaration C:\Soft\dmd2\src\phobos\std\array.d(778): Error: template std.algorithm.copy(Range1,Range2) if (isInputRange!(Range1) && isOutputRange!(Range2,ElementType!(Range1))) cannot deduce template function from argument types !()(Result,string) test26.d(9): Error: template instance std.array.join!(string[],Map!(result,string)) error instantiating Program: import std.array; import std.algorithm; void main() { string[] arr; string sep; join(arr, map!"a"(sep)); } -- Best regards, Vladimir mailto:vladimir thecybershadow.net
May 27 2011
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On 2011-05-27 11:51, Vladimir Panteleev wrote:
 On Fri, 27 May 2011 21:29:10 +0300, KennyTM~ <kennytm gmail.com> wrote:
 On May 28, 11 02:05, Vladimir Panteleev wrote:
 Why hasLength!string is false:
 https://github.com/D-Programming-Language/phobos/blob/96941d5384a5fee302
 df/std/range.d#L767
 
 
 Would it make sense to introduce a hasOpaqueLength oslt. to fix this?

You mean std.array.join? The 2-argument overload has specialization for strings (https://github.com/D-Programming-Language/phobos/blob/master/std/array.d #L966). static if (isForwardRange!RoR && hasLength!RoR && (hasLength!(ElementType!RoR) || isSomeString!(ElementType!RoR)) // ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ && hasLength!R) It doesn't exist for the 1-argument overload though.

Thanks! But it looks like the 2-argument overload is broken too. It doesn't check if the separator is a string, so that branch isn't taken for join(string[], string). In fact, it doesn't work at all when the second argument is an iterator with length: C:\Soft\dmd2\src\phobos\std\array.d(778): Error: template std.algorithm.copy(Range1,Range2) if (isInputRange!(Range1) && isOutputRange!(Range2,ElementType!(Range1))) does not match any function template declaration C:\Soft\dmd2\src\phobos\std\array.d(778): Error: template std.algorithm.copy(Range1,Range2) if (isInputRange!(Range1) && isOutputRange!(Range2,ElementType!(Range1))) cannot deduce template function from argument types !()(Result,string) test26.d(9): Error: template instance std.array.join!(string[],Map!(result,string)) error instantiating Program: import std.array; import std.algorithm; void main() { string[] arr; string sep; join(arr, map!"a"(sep)); }

Create a bug report and I'll look into it. - Jonathan M Davis
May 27 2011
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On 2011-05-27 12:33, Andrei Alexandrescu wrote:
 On 5/27/11 2:00 PM, Jonathan M Davis wrote:
 On 2011-05-27 11:51, Vladimir Panteleev wrote:

 Create a bug report and I'll look into it.
 
 - Jonathan M Davis

Thank you, Vladimir and even more so Jonathan!

Well, I'm in the midst of working on revamping some of the string stuff anyway, so it makes perfect sense for me to look at it. Though doing that has definitely reminded me that someone needs to go through Phobos (particularly the pieces that get used by everything else) and make as much safe, pure, and nothrow as currently possible. There's stuff in std.string which should be but can't be (though very little of it could even be nothrow thanks to UtfExceptions). Of course, the lack of conditional attributes mostly make that not work for templates anyway, but we could be doing a lot better. I guess that I'll have to put it on my todo list to go through Phobos and do that, but who knows when I'll get around to it. The two features that I'd _really_ like to see implemented at this point are conditional attributes and deprecation with messages (including soft deprecation). They'd definitely make life easier and fix otherwise unfixable problems. But naturally, Walter's todo list is quite large and presumably quite larger than mine, so who knows when we'll get them. Well, we'll just have to do the best that we can in the interim, but some things just won't be feasible until those features are implemented. - Jonathan M Davis
May 27 2011
prev sibling parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Fri, 27 May 2011 22:00:35 +0300, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:

 Create a bug report and I'll look into it.

http://d.puremagic.com/issues/show_bug.cgi?id=6064 . Thanks! I had a look into fixing it myself, but the changes required are not trivial - joiner always returns a dchar range when given narrow strings. I look forward to seeing your solution. -- Best regards, Vladimir mailto:vladimir thecybershadow.net
May 27 2011