www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - std.string.join & co. do a character-by-character copy because

reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
Why hasLength!string is false:  
https://github.com/D-Programming-Language/phobos/blob/96941d5384a5fee302df/std/range.d#L767

Would it make sense to introduce a hasOpaqueLength oslt. to fix this?

-- 
Best regards,
  Vladimir                            mailto:vladimir thecybershadow.net
May 27 2011
next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On 2011-05-27 11:05, Vladimir Panteleev wrote:

 Why hasLength!string is false:
 https://github.com/D-Programming-Language/phobos/blob/96941d5384a5fee302df/
 std/range.d#L767
 
 Would it make sense to introduce a hasOpaqueLength oslt. to fix this?


Um. Why? What would it give you? hasLength is intended for range-based stuff. 
and and any arrays of char or wchar are ranges of dchar, so they don't have a 
length property as far as ranges are concerned. If you want your template to 
work with any of the string types, then just use isSomeString to check whether 
it's a string. What would hasOpaqueLength do? What would it get you? Yes, the 
situation with char and wchar technically having a length property but having 
hasLength be false for them is a bit odd, but it works, and hasLength does 
exactly what it's supposed to - check whether the type has a length property 
for use with ranges. And char and wchar arrays don't.

- Jonathan M Davis
May 27 2011
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Fri, 27 May 2011 21:18:40 +0300, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:


 On 2011-05-27 11:05, Vladimir Panteleev wrote:

 Why hasLength!string is false:
 https://github.com/D-Programming-Language/phobos/blob/96941d5384a5fee302df/
 std/range.d#L767

 Would it make sense to introduce a hasOpaqueLength oslt. to fix this?


 Um. Why? What would it give you? hasLength is intended for range-based  
 stuff.
 and and any arrays of char or wchar are ranges of dchar, so they don't  
 have a
 length property as far as ranges are concerned. If you want your  
 template to
 work with any of the string types, then just use isSomeString to check  
 whether
 it's a string. What would hasOpaqueLength do? What would it get you?  
 Yes, the
 situation with char and wchar technically having a length property but  
 having
 hasLength be false for them is a bit odd, but it works, and hasLength  
 does
 exactly what it's supposed to - check whether the type has a length  
 property
 for use with ranges. And char and wchar arrays don't.


Wow, you sure took your time to bash my stupid suggestion :) I agree  
completely, TBH I only added that because I didn't want my post to look  
like a complaint without even an attempt to suggest a solution.

As long as std.array.join & co. don't do a character-by-character copy for  
strings, I'm happy. Still, putting isSomeString in std.array/std.range  
checks seems like a hack to me.

-- 
Best regards,
  Vladimir                            mailto:vladimir thecybershadow.net
May 27 2011
parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On 2011-05-27 11:25, Vladimir Panteleev wrote:

 On Fri, 27 May 2011 21:18:40 +0300, Jonathan M Davis <jmdavisProg gmx.com>
 
 wrote:

 On 2011-05-27 11:05, Vladimir Panteleev wrote:

 Why hasLength!string is false:
 https://github.com/D-Programming-Language/phobos/blob/96941d5384a5fee302
 df/ std/range.d#L767
 
 Would it make sense to introduce a hasOpaqueLength oslt. to fix this?


 
 Um. Why? What would it give you? hasLength is intended for range-based
 stuff.
 and and any arrays of char or wchar are ranges of dchar, so they don't
 have a
 length property as far as ranges are concerned. If you want your
 template to
 work with any of the string types, then just use isSomeString to check
 whether
 it's a string. What would hasOpaqueLength do? What would it get you?
 Yes, the
 situation with char and wchar technically having a length property but
 having
 hasLength be false for them is a bit odd, but it works, and hasLength
 does
 exactly what it's supposed to - check whether the type has a length
 property
 for use with ranges. And char and wchar arrays don't.


 
 Wow, you sure took your time to bash my stupid suggestion :) I agree
 completely, TBH I only added that because I didn't want my post to look
 like a complaint without even an attempt to suggest a solution.
 
 As long as std.array.join & co. don't do a character-by-character copy for
 strings, I'm happy. Still, putting isSomeString in std.array/std.range
 checks seems like a hack to me.


In some cases, where two strings are of the same type, character by character 
processing can be avoided (it depends on the function), but in many cases, it 
can't be, and if the string types differ, it definitely can't be. So, it may 
be that some functions could be improved to handle string processing better, 
but in the general case, the strings have to be decoded to process them, which 
means operating on them character by character. But if you notice a function 
which could and should do a better job with strings and you have an improved 
implementation, please create a pull request for it. There _are_ quite a few 
functions, however, which are optimized for string-processing.

- Jonathan M Davis
May 27 2011
prev sibling parent reply KennyTM~ <kennytm gmail.com> writes:
On May 28, 11 02:05, Vladimir Panteleev wrote:

 Why hasLength!string is false:
 https://github.com/D-Programming-Language/phobos/blob/96941d5384a5fee302df/std/range.d#L767


 Would it make sense to introduce a hasOpaqueLength oslt. to fix this?


You mean std.array.join? The 2-argument overload has specialization for 
strings 
(https://github.com/D-Programming-Language/phobos/blob/master/std/array.d#L966).

     static if (isForwardRange!RoR && hasLength!RoR
             && (hasLength!(ElementType!RoR) || 
isSomeString!(ElementType!RoR))
// 
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
             && hasLength!R)

It doesn't exist for the 1-argument overload though.
May 27 2011
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Fri, 27 May 2011 21:29:10 +0300, KennyTM~ <kennytm gmail.com> wrote:


 On May 28, 11 02:05, Vladimir Panteleev wrote:

 Why hasLength!string is false:
 https://github.com/D-Programming-Language/phobos/blob/96941d5384a5fee302df/std/range.d#L767


 Would it make sense to introduce a hasOpaqueLength oslt. to fix this?


 You mean std.array.join? The 2-argument overload has specialization for  
 strings  
 (https://github.com/D-Programming-Language/phobos/blob/master/std/array.d#L966).

      static if (isForwardRange!RoR && hasLength!RoR
              && (hasLength!(ElementType!RoR) ||  
 isSomeString!(ElementType!RoR))
 // ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
              && hasLength!R)

 It doesn't exist for the 1-argument overload though.


Thanks! But it looks like the 2-argument overload is broken too. It  
doesn't check if the separator is a string, so that branch isn't taken for  
join(string[], string). In fact, it doesn't work at all when the second  
argument is an iterator with length:

C:\Soft\dmd2\src\phobos\std\array.d(778): Error: template  
std.algorithm.copy(Range1,Range2) if (isInputRange!(Range1) &&  
isOutputRange!(Range2,ElementType!(Range1))) does not match any function  
template declaration
C:\Soft\dmd2\src\phobos\std\array.d(778): Error: template  
std.algorithm.copy(Range1,Range2) if (isInputRange!(Range1) &&  
isOutputRange!(Range2,ElementType!(Range1))) cannot deduce template  
function from argument types !()(Result,string)
test26.d(9): Error: template instance  
std.array.join!(string[],Map!(result,string)) error instantiating

Program:

import std.array;
import std.algorithm;

void main()
{
	string[] arr;
	string sep;

	join(arr, map!"a"(sep));
}


-- 
Best regards,
  Vladimir                            mailto:vladimir thecybershadow.net
May 27 2011
next sibling parent KennyTM~ <kennytm gmail.com> writes:
On May 28, 11 02:51, Vladimir Panteleev wrote:

 On Fri, 27 May 2011 21:29:10 +0300, KennyTM~ <kennytm gmail.com> wrote:


 On May 28, 11 02:05, Vladimir Panteleev wrote:

 Why hasLength!string is false:
 https://github.com/D-Programming-Language/phobos/blob/96941d5384a5fee302df/std/range.d#L767



 Would it make sense to introduce a hasOpaqueLength oslt. to fix this?


 You mean std.array.join? The 2-argument overload has specialization
 for strings
 (https://github.com/D-Programming-Language/phobos/blob/master/std/array.d#L966).


 static if (isForwardRange!RoR && hasLength!RoR
 && (hasLength!(ElementType!RoR) || isSomeString!(ElementType!RoR))
 // ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 && hasLength!R)

 It doesn't exist for the 1-argument overload though.


 Thanks! But it looks like the 2-argument overload is broken too. It
 doesn't check if the separator is a string, so that branch isn't taken
 for join(string[], string). In fact, it doesn't work at all when the
 second argument is an iterator with length:

 C:\Soft\dmd2\src\phobos\std\array.d(778): Error: template
 std.algorithm.copy(Range1,Range2) if (isInputRange!(Range1) &&
 isOutputRange!(Range2,ElementType!(Range1))) does not match any function
 template declaration
 C:\Soft\dmd2\src\phobos\std\array.d(778): Error: template
 std.algorithm.copy(Range1,Range2) if (isInputRange!(Range1) &&
 isOutputRange!(Range2,ElementType!(Range1))) cannot deduce template
 function from argument types !()(Result,string)
 test26.d(9): Error: template instance
 std.array.join!(string[],Map!(result,string)) error instantiating

 Program:

 import std.array;
 import std.algorithm;

 void main()
 {
 string[] arr;
 string sep;

 join(arr, map!"a"(sep));
 }


You're right. File a bug report?
May 27 2011
prev sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On 2011-05-27 11:51, Vladimir Panteleev wrote:

 On Fri, 27 May 2011 21:29:10 +0300, KennyTM~ <kennytm gmail.com> wrote:

 On May 28, 11 02:05, Vladimir Panteleev wrote:

 Why hasLength!string is false:
 https://github.com/D-Programming-Language/phobos/blob/96941d5384a5fee302
 df/std/range.d#L767
 
 
 Would it make sense to introduce a hasOpaqueLength oslt. to fix this?


 
 You mean std.array.join? The 2-argument overload has specialization for
 strings
 (https://github.com/D-Programming-Language/phobos/blob/master/std/array.d
 #L966).
 
 static if (isForwardRange!RoR && hasLength!RoR
 
 && (hasLength!(ElementType!RoR) ||
 
 isSomeString!(ElementType!RoR))
 // ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 && hasLength!R)
 
 It doesn't exist for the 1-argument overload though.


 
 Thanks! But it looks like the 2-argument overload is broken too. It
 doesn't check if the separator is a string, so that branch isn't taken for
 join(string[], string). In fact, it doesn't work at all when the second
 argument is an iterator with length:
 
 C:\Soft\dmd2\src\phobos\std\array.d(778): Error: template
 std.algorithm.copy(Range1,Range2) if (isInputRange!(Range1) &&
 isOutputRange!(Range2,ElementType!(Range1))) does not match any function
 template declaration
 C:\Soft\dmd2\src\phobos\std\array.d(778): Error: template
 std.algorithm.copy(Range1,Range2) if (isInputRange!(Range1) &&
 isOutputRange!(Range2,ElementType!(Range1))) cannot deduce template
 function from argument types !()(Result,string)
 test26.d(9): Error: template instance
 std.array.join!(string[],Map!(result,string)) error instantiating
 
 Program:
 
 import std.array;
 import std.algorithm;
 
 void main()
 {
 string[] arr;
 string sep;
 
 join(arr, map!"a"(sep));
 }


Create a bug report and I'll look into it.

- Jonathan M Davis
May 27 2011
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/27/11 2:00 PM, Jonathan M Davis wrote:

 On 2011-05-27 11:51, Vladimir Panteleev wrote:


[snip]

 Create a bug report and I'll look into it.

 - Jonathan M Davis


Thank you, Vladimir and even more so Jonathan!

Andrei
May 27 2011
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On 2011-05-27 12:33, Andrei Alexandrescu wrote:

 On 5/27/11 2:00 PM, Jonathan M Davis wrote:

 On 2011-05-27 11:51, Vladimir Panteleev wrote:


 [snip]
 

 Create a bug report and I'll look into it.
 
 - Jonathan M Davis


 
 Thank you, Vladimir and even more so Jonathan!


Well, I'm in the midst of working on revamping some of the string stuff 
anyway, so it makes perfect sense for me to look at it. Though doing that has 
definitely reminded me that someone needs to go through Phobos (particularly 
the pieces that get used by everything else) and make as much  safe, pure, and 
nothrow as currently possible. There's stuff in std.string which should be but 
can't be (though very little of it could even be nothrow thanks to 
UtfExceptions). Of course, the lack of conditional attributes mostly make that 
not work for templates anyway, but we could be doing a lot better. I guess 
that I'll have to put it on my todo list to go through Phobos and do that, but 
who knows when I'll get around to it.

The two features that I'd _really_ like to see implemented at this point are 
conditional attributes and deprecation with messages (including soft 
deprecation). They'd definitely make life easier and fix otherwise unfixable 
problems. But naturally, Walter's todo list is quite large and presumably 
quite larger than mine, so who knows when we'll get them. Well, we'll just 
have to do the best that we can in the interim, but some things just won't be 
feasible until those features are implemented.

- Jonathan M Davis
May 27 2011
prev sibling parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Fri, 27 May 2011 22:00:35 +0300, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:


 Create a bug report and I'll look into it.


http://d.puremagic.com/issues/show_bug.cgi?id=6064 . Thanks!

I had a look into fixing it myself, but the changes required are not  
trivial - joiner always returns a dchar range when given narrow strings. I  
look forward to seeing your solution.

-- 
Best regards,
  Vladimir                            mailto:vladimir thecybershadow.net
May 27 2011