www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - String Slice/Concatenate bug

reply Russ Lewis <spamhole-2001-07-16 deming-os.org> writes:
DMD 0.129, Linux (Fedora Core 3)

Notice that in the output, the 2nd line starts with a 4 instead of a 5. 
  As it says in the comment, it will work if you .dup the slice.

 [russ russ dmd_bugs]$ cat slice.d
 import std.stdio;
 
 void main() {
   char[] foo = "1234567890".dup;
   char[] bar;
   bar = foo[0..1];  // works if you append .dup here
   foo = foo[1..$];
   writefln(foo," ",bar);
   bar ~= ","~foo[0..3];
   foo  = foo[3..$];
   writefln(foo," ",bar);
 }
 
 [russ russ dmd_bugs]$ dmd slice.d
 gcc slice.o -o slice -lphobos -lpthread -lm
 [russ russ dmd_bugs]$ ./slice
 234567890 1
 467890 1,234
 [russ russ dmd_bugs]$

Aug 18 2005
next sibling parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
"Russ Lewis" <spamhole-2001-07-16 deming-os.org> wrote in message 
news:de376q$lmr$1 digitaldaemon.com...
 DMD 0.129, Linux (Fedora Core 3)

 Notice that in the output, the 2nd line starts with a 4 instead of a 5. As 
 it says in the comment, it will work if you .dup the slice.

 [russ russ dmd_bugs]$ cat slice.d
 import std.stdio;

 void main() {
   char[] foo = "1234567890".dup;
   char[] bar;
   bar = foo[0..1];  // works if you append .dup here
   foo = foo[1..$];
   writefln(foo," ",bar);
   bar ~= ","~foo[0..3];
   foo  = foo[3..$];
   writefln(foo," ",bar);
 }

 [russ russ dmd_bugs]$ dmd slice.d
 gcc slice.o -o slice -lphobos -lpthread -lm
 [russ russ dmd_bugs]$ ./slice
 234567890 1
 467890 1,234
 [russ russ dmd_bugs]$


I don't think that's a bug. ~= is behaving as expected (though I don't know if it's actually documented when ~= dups and when it doesn't). Are you suggesting ~= always dup? I'm not sure what you are expecting.
Aug 18 2005
parent reply Russ Lewis <spamhole-2001-07-16 deming-os.org> writes:
Ben Hinkle wrote:
 "Russ Lewis" <spamhole-2001-07-16 deming-os.org> wrote in message 
 news:de376q$lmr$1 digitaldaemon.com...
 
DMD 0.129, Linux (Fedora Core 3)

Notice that in the output, the 2nd line starts with a 4 instead of a 5. As 
it says in the comment, it will work if you .dup the slice.


[russ russ dmd_bugs]$ cat slice.d
import std.stdio;

void main() {
  char[] foo = "1234567890".dup;
  char[] bar;
  bar = foo[0..1];  // works if you append .dup here
  foo = foo[1..$];
  writefln(foo," ",bar);
  bar ~= ","~foo[0..3];
  foo  = foo[3..$];
  writefln(foo," ",bar);
}

[russ russ dmd_bugs]$ dmd slice.d
gcc slice.o -o slice -lphobos -lpthread -lm
[russ russ dmd_bugs]$ ./slice
234567890 1
467890 1,234
[russ russ dmd_bugs]$


I don't think that's a bug. ~= is behaving as expected (though I don't know if it's actually documented when ~= dups and when it doesn't). Are you suggesting ~= always dup? I'm not sure what you are expecting.

I don't remember if this is documented or not, either. My asumption was that ~= would do something analogous to realloc(); if the memory immediately following the buffer is already unallocated, then just extend the buffer; otherwise, duplicate it to a new location where there is space.
Aug 18 2005
parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
"Russ Lewis" <spamhole-2001-07-16 deming-os.org> wrote in message 
news:de38os$nba$1 digitaldaemon.com...
 Ben Hinkle wrote:
 "Russ Lewis" <spamhole-2001-07-16 deming-os.org> wrote in message 
 news:de376q$lmr$1 digitaldaemon.com...

DMD 0.129, Linux (Fedora Core 3)

Notice that in the output, the 2nd line starts with a 4 instead of a 5. 
As it says in the comment, it will work if you .dup the slice.


[russ russ dmd_bugs]$ cat slice.d
import std.stdio;

void main() {
  char[] foo = "1234567890".dup;
  char[] bar;
  bar = foo[0..1];  // works if you append .dup here
  foo = foo[1..$];
  writefln(foo," ",bar);
  bar ~= ","~foo[0..3];
  foo  = foo[3..$];
  writefln(foo," ",bar);
}

[russ russ dmd_bugs]$ dmd slice.d
gcc slice.o -o slice -lphobos -lpthread -lm
[russ russ dmd_bugs]$ ./slice
234567890 1
467890 1,234
[russ russ dmd_bugs]$


I don't think that's a bug. ~= is behaving as expected (though I don't know if it's actually documented when ~= dups and when it doesn't). Are you suggesting ~= always dup? I'm not sure what you are expecting.

I don't remember if this is documented or not, either. My asumption was that ~= would do something analogous to realloc(); if the memory immediately following the buffer is already unallocated, then just extend the buffer; otherwise, duplicate it to a new location where there is space.

I think the problem is it can't tell that foo is using the memory following bar. All bar knows is that it's a pointer to the start of an allocation block that can hold the requested addition. Thinking about it some more, it seems like users of ~= must know if the memory following the array is "in use". If it could be (or is) then the user must dup explicitly. That means the +1 that Walter had to add to all memory allocations can go away because slicing off the end of an array shouldn't be extended using ~=. That would mean your bug actually has a silver lining since I've never liked that +1. Put the burden on the user to know if they can extend safely - just like COW the rule is "don't ~= in memory you don't own".
Aug 18 2005
parent reply Russ Lewis <spamhole-2001-07-16 deming-os.org> writes:
Ben Hinkle wrote:
 "Russ Lewis" <spamhole-2001-07-16 deming-os.org> wrote in message 
 news:de38os$nba$1 digitaldaemon.com...
 
Ben Hinkle wrote:

"Russ Lewis" <spamhole-2001-07-16 deming-os.org> wrote in message 
news:de376q$lmr$1 digitaldaemon.com...


DMD 0.129, Linux (Fedora Core 3)

Notice that in the output, the 2nd line starts with a 4 instead of a 5. 
As it says in the comment, it will work if you .dup the slice.



[russ russ dmd_bugs]$ cat slice.d
import std.stdio;

void main() {
 char[] foo = "1234567890".dup;
 char[] bar;
 bar = foo[0..1];  // works if you append .dup here
 foo = foo[1..$];
 writefln(foo," ",bar);
 bar ~= ","~foo[0..3];
 foo  = foo[3..$];
 writefln(foo," ",bar);
}

[russ russ dmd_bugs]$ dmd slice.d
gcc slice.o -o slice -lphobos -lpthread -lm
[russ russ dmd_bugs]$ ./slice
234567890 1
467890 1,234
[russ russ dmd_bugs]$


I don't think that's a bug. ~= is behaving as expected (though I don't know if it's actually documented when ~= dups and when it doesn't). Are you suggesting ~= always dup? I'm not sure what you are expecting.

I don't remember if this is documented or not, either. My asumption was that ~= would do something analogous to realloc(); if the memory immediately following the buffer is already unallocated, then just extend the buffer; otherwise, duplicate it to a new location where there is space.

I think the problem is it can't tell that foo is using the memory following bar. All bar knows is that it's a pointer to the start of an allocation block that can hold the requested addition. Thinking about it some more, it seems like users of ~= must know if the memory following the array is "in use". If it could be (or is) then the user must dup explicitly. That means the +1 that Walter had to add to all memory allocations can go away because slicing off the end of an array shouldn't be extended using ~=. That would mean your bug actually has a silver lining since I've never liked that +1. Put the burden on the user to know if they can extend safely - just like COW the rule is "don't ~= in memory you don't own".

I hear you, but it seems to me that if you are in the middle of an allocation, and you're only using part of it, then you should *assume* that the rest of the string is being used by somebody else. It's not always true, but it often will be. Just my opinion, though. I'd love to hear what the official word is.
Aug 18 2005
parent "Ben Hinkle" <ben.hinkle gmail.com> writes:
"Russ Lewis" <spamhole-2001-07-16 deming-os.org> wrote in message
news:de3ino$vld$1 digitaldaemon.com...
 Ben Hinkle wrote:
 "Russ Lewis" <spamhole-2001-07-16 deming-os.org> wrote in message
 news:de38os$nba$1 digitaldaemon.com...

Ben Hinkle wrote:

"Russ Lewis" <spamhole-2001-07-16 deming-os.org> wrote in message
news:de376q$lmr$1 digitaldaemon.com...


DMD 0.129, Linux (Fedora Core 3)

Notice that in the output, the 2nd line starts with a 4 instead of a 5.
As it says in the comment, it will work if you .dup the slice.

[russ russ dmd_bugs]$ cat slice.d
import std.stdio;

void main() {
 char[] foo = "1234567890".dup;
 char[] bar;
 bar = foo[0..1];  // works if you append .dup here
 foo = foo[1..$];
 writefln(foo," ",bar);
 bar ~= ","~foo[0..3];
 foo  = foo[3..$];
 writefln(foo," ",bar);
}

[russ russ dmd_bugs]$ dmd slice.d
gcc slice.o -o slice -lphobos -lpthread -lm
[russ russ dmd_bugs]$ ./slice
234567890 1
467890 1,234
[russ russ dmd_bugs]$


I don't think that's a bug. ~= is behaving as expected (though I don't know if it's actually documented when ~= dups and when it doesn't). Are you suggesting ~= always dup? I'm not sure what you are expecting.

I don't remember if this is documented or not, either. My asumption was that ~= would do something analogous to realloc(); if the memory immediately following the buffer is already unallocated, then just extend the buffer; otherwise, duplicate it to a new location where there is space.

I think the problem is it can't tell that foo is using the memory following bar. All bar knows is that it's a pointer to the start of an allocation block that can hold the requested addition. Thinking about it some more, it seems like users of ~= must know if the memory following the array is "in use". If it could be (or is) then the user must dup explicitly. That means the +1 that Walter had to add to all memory allocations can go away because slicing off the end of an array shouldn't be extended using ~=. That would mean your bug actually has a silver lining since I've never liked that +1. Put the burden on the user to know if they can extend safely - just like COW the rule is "don't ~= in memory you don't own".

I hear you, but it seems to me that if you are in the middle of an allocation, and you're only using part of it, then you should *assume* that the rest of the string is being used by somebody else. It's not always true, but it often will be. Just my opinion, though. I'd love to hear what the official word is.

I don't follow who "you" is. Are you saying the compiler should do something different for your example than what it is doing now? Or by "you" do you mean the programmer? Just to be clear, what I'm saying is that 1) ~= do what it does today but document the duping behavior 2) functions that use ~= on inputs should document it so callers can take action to avoid extending into live memory 3) the +1 that is added to gc allocations should be removed since it is the user's responsibility to manage safe extensions. The alternative of having ~= dup every time will mean building arrays in a loop using ~= will waste lots of memory (and time spent duping). The doc http://www.digitalmars.com/d/arrays.html#resize does say that you should avoid setting length or cat'ing with slices.
Aug 19 2005
prev sibling parent Derek Parnell <derek psych.ward> writes:
On Thu, 18 Aug 2005 16:56:09 -0700, Russ Lewis wrote:

 DMD 0.129, Linux (Fedora Core 3)
 
 Notice that in the output, the 2nd line starts with a 4 instead of a 5. 
   As it says in the comment, it will work if you .dup the slice.
 
 [russ russ dmd_bugs]$ cat slice.d
 import std.stdio;
 
 void main() {
   char[] foo = "1234567890".dup;
   char[] bar;
   bar = foo[0..1];  // works if you append .dup here
   foo = foo[1..$];
   writefln(foo," ",bar);
   bar ~= ","~foo[0..3];
   foo  = foo[3..$];
   writefln(foo," ",bar);
 }
 
 [russ russ dmd_bugs]$ dmd slice.d
 gcc slice.o -o slice -lphobos -lpthread -lm
 [russ russ dmd_bugs]$ ./slice
 234567890 1
 467890 1,234
 [russ russ dmd_bugs]$


This is a surprise. I was under the impression that *all* concatenations caused an automatic dup operation. It will also work if you have ... bar = bar~","~foo[0..3]; Damn ... now I have to go back and check my existing code to make sure I didn't use this construct anywhere. -- Derek (skype: derek.j.parnell) Melbourne, Australia 19/08/2005 1:48:42 PM
Aug 18 2005