www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Something goes wrong with range and sort?

reply "Andrea Fontana" <nospam example.com> writes:
	string[] test_filter(string[] words)
	{
		static blackList =
		[
			"d", "c", "e", "a", "è", "é", "e"
		].sort();
		
		return words.filter!((a) => 
!blackList.assumeSorted.contains(a)).array;
	}

	test_filter(["a", "b", "test", "hello"]).writeln;


This code crash! It's just a useless trimmed-down version of a 
more complex code, just to show you the bug.

If you remove accented letters from "blacklist" array it works 
fine. Why?
Jun 14 2013
next sibling parent "Aleksandar Ruzicic" <aleksandar ruzicic.info> writes:
On Friday, 14 June 2013 at 10:07:13 UTC, Andrea Fontana wrote:
 	string[] test_filter(string[] words)
 	{
 		static blackList =
 		[
 			"d", "c", "e", "a", "è", "é", "e"
 		].sort();
 		
 		return words.filter!((a) => 
 !blackList.assumeSorted.contains(a)).array;
 	}

 	test_filter(["a", "b", "test", "hello"]).writeln;


 This code crash! It's just a useless trimmed-down version of a 
 more complex code, just to show you the bug.

 If you remove accented letters from "blacklist" array it works 
 fine. Why?

sort() requires UTF32 input (a dstring/dchar[]) or it will fail. UTF8 (i.e. string type in D) is a variable-length encoding and if such input is given to sort it would refuse to compile since it cannot sort it in-place (a guarantee sort() makes). I guess that in case UTF8 input with 1-byte only characters is given to sort() it can detect that there are no variable length characters and that is probably the reason your code works when you remove non-ascii characters.
Jun 14 2013
prev sibling next sibling parent "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Friday, 14 June 2013 at 10:07:13 UTC, Andrea Fontana wrote:
 This code crash! It's just a useless trimmed-down version of a 
 more complex code, just to show you the bug.

 If you remove accented letters from "blacklist" array it works 
 fine. Why?

Works for me... http://dpaste.dzfl.pl/79db5a8b Can you produce a failing case on dpaste?
 sort() requires UTF32 input (a dstring/dchar[]) or it will fail.

 UTF8 (i.e. string type in D) is a variable-length encoding and 
 if such input is given to sort it would refuse to compile since 
 it cannot sort it in-place (a guarantee sort() makes).

 I guess that in case UTF8 input with 1-byte only characters is 
 given to sort() it can detect that there are no variable length 
 characters and that is probably the reason your code works when 
 you remove non-ascii characters.

sort works just fine with string[]. Note that he's not sorting a string, he's sorting an array of strings.
Jun 14 2013
prev sibling next sibling parent "Aleksandar Ruzicic" <aleksandar ruzicic.info> writes:
On Friday, 14 June 2013 at 10:25:04 UTC, Peter Alexander wrote:
 sort works just fine with string[]. Note that he's not sorting 
 a string, he's sorting an array of strings.

Then I must have understood it wrong (or have confused sort() with some other function), gotta go read the docs again :)
Jun 14 2013
prev sibling next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Friday, 14 June 2013 at 10:30:18 UTC, Aleksandar Ruzicic wrote:
 On Friday, 14 June 2013 at 10:25:04 UTC, Peter Alexander wrote:
 sort works just fine with string[]. Note that he's not sorting 
 a string, he's sorting an array of strings.

Then I must have understood it wrong (or have confused sort() with some other function), gotta go read the docs again :)

You are mostly right, but there is a special overload of sort for narrow strings, that does it smart.
Jun 14 2013
prev sibling next sibling parent "Andrea Fontana" <nospam example.com> writes:
Just compile your example with -debug and assert will be hit!

On Friday, 14 June 2013 at 10:25:04 UTC, Peter Alexander wrote:
 On Friday, 14 June 2013 at 10:07:13 UTC, Andrea Fontana wrote:
 This code crash! It's just a useless trimmed-down version of a 
 more complex code, just to show you the bug.

 If you remove accented letters from "blacklist" array it works 
 fine. Why?

Works for me... http://dpaste.dzfl.pl/79db5a8b Can you produce a failing case on dpaste?

Jun 14 2013
prev sibling next sibling parent "Brad Anderson" <eco gnuk.net> writes:
On Friday, 14 June 2013 at 10:07:13 UTC, Andrea Fontana wrote:
 	string[] test_filter(string[] words)
 	{
 		static blackList =
 		[
 			"d", "c", "e", "a", "è", "é", "e"
 		].sort();
 		
 		return words.filter!((a) => 
 !blackList.assumeSorted.contains(a)).array;
 	}

 	test_filter(["a", "b", "test", "hello"]).writeln;


 This code crash! It's just a useless trimmed-down version of a 
 more complex code, just to show you the bug.

 If you remove accented letters from "blacklist" array it works 
 fine. Why?

I'm not seeing a crash here: http://dpaste.dzfl.pl/59d5fb36 What version of the compiler/phobos are you using?
Jun 14 2013
prev sibling next sibling parent "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Friday, 14 June 2013 at 12:40:16 UTC, Andrea Fontana wrote:
 Just compile your example with -debug and assert will be hit!

Tried that, still cannot repro. As Brad said, what compiler version are you using?
Jun 14 2013
prev sibling parent "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Friday, 14 June 2013 at 17:58:36 UTC, Peter Alexander wrote:
 On Friday, 14 June 2013 at 12:40:16 UTC, Andrea Fontana wrote:
 Just compile your example with -debug and assert will be hit!

Tried that, still cannot repro. As Brad said, what compiler version are you using?

I tried with version 2.062 and I get an assert in -debug. If you are on that version or a previous version, you might want to try downloading the latest version.
Jun 14 2013