www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Setting array length to 0 discards reserved allocation?

reply "Andrew Godfrey" <X y.com> writes:
Is this a bug? Or am I misunderstanding
something?


import std.stdio, std.string;
import std.array;

unittest {
     int[] a;
     a.reserve(10);
     a.length++;
     int *p = &(a[0]);

     a.length++; // Increase size. Shouldn't reallocate.
     int *p2 = &(a[0]);
     assert(p == p2); // And indeed it didn't.

     // So sometime later, we want to reuse the allocation. We 
'clear' it:
     a.length = 0;

     // ... and later start using it...
     a.length++;
     int *q = &a[0];
     assert(p == q, format("p: %s; q: %s", p, q)); // FAILS! (on 
dmd 2.065.0) Why? Apparently, setting length to 0 caused it to 
free the
                                                   // reserved 
allocation?
}
Jul 24 2014
next sibling parent "safety0ff" <safety0ff.dev gmail.com> writes:
On Thursday, 24 July 2014 at 23:30:13 UTC, Andrew Godfrey wrote:
 Is this a bug? Or am I misunderstanding
 something?
You need to use assumeSafeAppend before increasing the length again to avoid allocating new storage. The runtime doesn't know that `a` is the only reference to that array, so increasing the length causes an allocation to avoid stomping on data.
Jul 24 2014
prev sibling next sibling parent "Chris Cain" <zshazz gmail.com> writes:
See this post:
http://forum.dlang.org/thread/shpcpmwvphbyaumxghsd forum.dlang.org

My response in that thread:
----
The reason is simple, observe:

auto arr = [1,2,3]

It is safe if you append 4 to that.

auto arr = [1,2,3,4]
auto other = arr[];
arr.length = 2;

It is *not* safe to append 5 to arr here, because doing so would
change other to [1,2,5,4] which is (probably) not what you want
(or, at least, you haven't made it explicit that you're okay with
this behavior). Since slices are unaware of whether other views
on memory exist, they must always reallocate if they've been
shrunk to avoid the possibility of overwriting existing elements
in other views of the array.

When you use "assumeSafeAppend", you are guaranteeing that either
no other views exist, or if they do exist that you don't care if
they are overwritten. Take note that the second case is *really
bad* if your element type is immutable, for obvious reasons.
----
Jul 24 2014
prev sibling parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Thu, 24 Jul 2014 23:30:11 +0000
Andrew Godfrey via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 Is this a bug? Or am I misunderstanding
 something?
It's not a bug. If it did it any other way, you'd have problems with arrays stomping on each others memory. The underlying data structure keeps track of the farthest point in the block which an array has been expanded to, but it doesn't keep track of what arrays exist that point to it, so it can't know which ones point where. So, it has no way of knowing that the slice whose length you're setting to 0 is the only slice. assumeSafeAppend is used to fix that problem, but you have to be sure that it _is_ safe to append to that array without stomping on other arrays, or you're going to have problems. You really should read this article: http://dlang.org/d-array-article.html - Jonathan M Davis
Jul 24 2014
parent reply "Andrew Godfrey" <X y.com> writes:
On Friday, 25 July 2014 at 04:38:40 UTC, Jonathan M Davis via 
Digitalmars-d wrote:
 You really should read this article:

 http://dlang.org/d-array-article.html
Thank you, that was educational. What I needed to read was that "A slice does not own the array, it references the array." - which I knew, but I hadn't internalized that even when a dynamic array is a private member, it is accessed via a slice and the runtime is not aware when there's a 1-to-1 correspondence between the two. The documentation could help a bit more. I'm game to try to make a pull request for this, but I'm wondering if the library docs can actually point to the article? More feasibly, I'm thinking: Both the documentation for length() (http://dlang.org/arrays) and reserve() (http://dlang.org/phobos/std_array.html) should at least mention assumeSafeAppend. An example similar to what I posted may also be worthwhile. (As I understand it from the article, the problem is that reducing 'length' also reduces 'used', whereas in this case I really want that slice to continue to own the entire reserved amount.)
Jul 25 2014
next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Fri, Jul 25, 2014 at 03:07:53PM +0000, Andrew Godfrey via Digitalmars-d
wrote:
 On Friday, 25 July 2014 at 04:38:40 UTC, Jonathan M Davis via Digitalmars-d
 wrote:
You really should read this article:

http://dlang.org/d-array-article.html
Thank you, that was educational. What I needed to read was that "A slice does not own the array, it references the array." - which I knew, but I hadn't internalized that even when a dynamic array is a private member, it is accessed via a slice and the runtime is not aware when there's a 1-to-1 correspondence between the two. The documentation could help a bit more. I'm game to try to make a pull request for this, but I'm wondering if the library docs can actually point to the article?
That's a very good idea. I think that would dispel a lot of misconceptions that newcomers might have.
 More feasibly, I'm thinking:
 Both the documentation for length() (http://dlang.org/arrays)
 and reserve() (http://dlang.org/phobos/std_array.html)
 should at least mention assumeSafeAppend.
 An example similar to what I posted may also be worthwhile.
[...] Yes, please submit a PR for this. T -- I see that you JS got Bach.
Jul 25 2014
parent reply "Andrew Godfrey" <X y.com> writes:
On Friday, 25 July 2014 at 15:55:50 UTC, H. S. Teoh via 
Digitalmars-d wrote:
 On Fri, Jul 25, 2014 at 03:07:53PM +0000, Andrew Godfrey via 
 Digitalmars-d wrote:
 On Friday, 25 July 2014 at 04:38:40 UTC, Jonathan M Davis via 
 Digitalmars-d
 wrote:
You really should read this article:

http://dlang.org/d-array-article.html
Thank you, that was educational. What I needed to read was that "A slice does not own the array, it references the array." - which I knew, but I hadn't internalized that even when a dynamic array is a private member, it is accessed via a slice and the runtime is not aware when there's a 1-to-1 correspondence between the two. The documentation could help a bit more. I'm game to try to make a pull request for this, but I'm wondering if the library docs can actually point to the article?
That's a very good idea. I think that would dispel a lot of misconceptions that newcomers might have.
 More feasibly, I'm thinking:
 Both the documentation for length() (http://dlang.org/arrays)
 and reserve() (http://dlang.org/phobos/std_array.html)
 should at least mention assumeSafeAppend.
 An example similar to what I posted may also be worthwhile.
[...] Yes, please submit a PR for this.
So I've been looking into this, and I've realized the 'arrays' page is actually inconsistent. (http://dlang.org/arrays). Instead of spending time on a big update, I'll outline a proposal here. This page generally uses the term "array" to mean the slice, and "array data" to mean the allocated data that a slice currently refers to. (In contrast, the handy article calls these "slice" and "dynamic array" respectively). For example:
 Dynamic Arrays
 int[] a;
 Dynamic arrays consist of a length and a pointer to the array 
 data. Multiple > dynamic arrays can share all or parts of the 
 array data.
Problems start early on:
 Slicing
 Slicing an array means to specify a subarray of it. An array 
 slice does not
 copy the data, it is only another reference to it.
By this page's terminology, this should actually say something like this: "slicing an array means to specify a subarray of that array's current array data". e.g. Causing the first 'array' (slice) to grow would dissociate the slices from each other. Because the term "slice" is so foreign, I sympathize with how the term was avoided to start with. But the fact is that a dynamic array declaration creates two separate things, and so I'd propose to give them reliable names at the earliest. So: Immediately after the section that introduces dynamic arrays, I would then put the section on slices, and rather than describing a slice as a "subarray", I would introduce it as an important part of the workings of a dynamic array. So I'm thinking the first example of a slice should be a simplification of the article's example: int[] a = new int[5]; // A new dynamic array, with slice 'a' pointing to it. int[] b = a[]; // We now have two slices pointing to one dynamic array. a[1] = 42; assert(b[1] == 42); // Demonstrating the point Thereafter can come sub-slice examples and so on. Does this make sense?
Jul 26 2014
parent reply "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Saturday, 26 July 2014 at 23:06:02 UTC, Andrew Godfrey wrote:
 Thereafter can come sub-slice examples and so on.
 Does this make sense?
Yes, the reference documentation is pretty terrible with naming of various array concepts. IIRC, when this was discussed in the past, a majority seemed to be in favour of using "slice" and "dynamic array" for their respective concepts instead of the current situation, but I also remember there was some opposition (for reason I can't remember). A pull request updating the documentation to use slice/dynamic array might weed them out ;)
Jul 26 2014
next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Sunday, 27 July 2014 at 05:51:46 UTC, Jakob Ovrum wrote:
 On Saturday, 26 July 2014 at 23:06:02 UTC, Andrew Godfrey wrote:
 Thereafter can come sub-slice examples and so on.
 Does this make sense?
Yes, the reference documentation is pretty terrible with naming of various array concepts. IIRC, when this was discussed in the past, a majority seemed to be in favour of using "slice" and "dynamic array" for their respective concepts instead of the current situation, but I also remember there was some opposition (for reason I can't remember). A pull request updating the documentation to use slice/dynamic array might weed them out ;)
As defined by the language, T[] is a dynamic array. As great as the article is, it was wrong in its use of the terminology, and that's what's caused a lot of the confusion and resulted in arguments over the difference between a dynamic array and a slice (there really isn't any). IIRC, both Walter and Andrei stated in that discussion that T[] is a dynamic array as far as D is concerned and that's not going to change. The article really should be updated to reflect the correct terminology. As far as D is concerned a slice and a dynamic array are the same thing when it comes to arrays. They're just different names for T[], and trying to treat them as different just causes confusion. - Jonathan M Davis
Jul 27 2014
next sibling parent reply "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Sunday, 27 July 2014 at 08:49:43 UTC, Jonathan M Davis wrote:
 As defined by the language, T[] is a dynamic array. As great as 
 the article is, it was wrong in its use of the terminology, and 
 that's what's caused a lot of the confusion and resulted in 
 arguments over the difference between a dynamic array and a 
 slice (there really isn't any). IIRC, both Walter and Andrei 
 stated in that discussion that T[] is a dynamic array as far as 
 D is concerned and that's not going to change. The article 
 really should be updated to reflect the correct terminology. As 
 far as D is concerned a slice and a dynamic array are the same 
 thing when it comes to arrays. They're just different names for 
 T[], and trying to treat them as different just causes 
 confusion.
Conflating both concepts with the same name is why the article was so dearly needed in the first place.
Jul 27 2014
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Sunday, 27 July 2014 at 09:10:26 UTC, Jakob Ovrum wrote:
 On Sunday, 27 July 2014 at 08:49:43 UTC, Jonathan M Davis wrote:
 As defined by the language, T[] is a dynamic array. As great 
 as the article is, it was wrong in its use of the terminology, 
 and that's what's caused a lot of the confusion and resulted 
 in arguments over the difference between a dynamic array and a 
 slice (there really isn't any). IIRC, both Walter and Andrei 
 stated in that discussion that T[] is a dynamic array as far 
 as D is concerned and that's not going to change. The article 
 really should be updated to reflect the correct terminology. 
 As far as D is concerned a slice and a dynamic array are the 
 same thing when it comes to arrays. They're just different 
 names for T[], and trying to treat them as different just 
 causes confusion.
Conflating both concepts with the same name is why the article was so dearly needed in the first place.
Except that no concepts are being conflated. T[] is the dynamic array. There's a block of memory managed by the GC underneath, but it's completely hidden from the programmer. It is _not_ the dynamic array. It's just a block of memory managed by the GC which is used to manage the memory for dynamic arrays, and it has a completely different type from T[]. It's not even an array in the D sense. The block of memory is referred to by a pointer, not a D array. It's talk of "which array" owns the memory and the like which causes confusion, and talking about the GC-managed block of memory as being the dynamic array and the T[] that the programmer sees as being the slice is just plain wrong. The T[] is both a slice and a dynamic array (because they're the same thing), whereas the block of memory is neither. I think that the fact that the article tried to call the underlying block of memory a dynamic array has caused a lot of unnecessary confusion. - Jonathan M Davis
Jul 27 2014
next sibling parent "Andrew Godfrey" <X y.com> writes:
 Conflating both concepts with the same name is why the article 
 was so dearly needed in the first place.
Except that no concepts are being conflated.
Yes, they are. Consider again the sentence I quoted: "Slicing an array means to specify a subarray of it." The word "it" is wrong; for it to be correct, it needs to refer to the underlying memory block, not the "array". This was the original cause of my confusion, I believe - it led me to think that "T[] is a dynamic array" (by which I mean a concept I understand well from other languages) but in fact, D has redefined the term "dynamic array" to mean something quite different. So, not only is "T[]" NOT the concept I thought it was, what it IS apparently goes by the same name as the concept I had confused it with! For the sake of increased adoption of D, I beg you to reconsider. I am not exaggerating - for a while there I thought I was looking at a huge red flag, and if I didn't already appreciate other strengths of D, I would have stopped there.
Jul 27 2014
prev sibling parent reply "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Sunday, 27 July 2014 at 12:03:05 UTC, Jonathan M Davis wrote:
 Except that no concepts are being conflated. T[] is the dynamic 
 array. There's a block of memory managed by the GC underneath, 
 but it's completely hidden from the programmer. It is _not_ the 
 dynamic array. It's just a block of memory managed by the GC 
 which is used to manage the memory for dynamic arrays, and it 
 has a completely different type from T[]. It's not even an 
 array in the D sense. The block of memory is referred to by a 
 pointer, not a D array.

 It's talk of "which array" owns the memory and the like which 
 causes confusion, and talking about the GC-managed block of 
 memory as being the dynamic array and the T[] that the 
 programmer sees as being the slice is just plain wrong. The T[] 
 is both a slice and a dynamic array (because they're the same 
 thing), whereas the block of memory is neither. I think that 
 the fact that the article tried to call the underlying block of 
 memory a dynamic array has caused a lot of unnecessary 
 confusion.

 - Jonathan M Davis
This very thread is a testament to the fact that no, it's not completely hidden, and it is the conflation that causes confusion, and clarity only comes when the two concepts are explained separately. This matches my experience when I teach about D's arrays on IRC too - people generally feel enlightened after reading the array article. It's debatable whether hamfisting dynamic array capabilities onto a simple slice type was a good idea or not (T[new], anyone?), but we should be as clear with the current situation as possible. Pretending there isn't a problem is ignoring the cries of D newbies everywhere. Please stop.
Jul 27 2014
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Sunday, 27 July 2014 at 13:03:13 UTC, Jakob Ovrum wrote:
 On Sunday, 27 July 2014 at 12:03:05 UTC, Jonathan M Davis wrote:
 Except that no concepts are being conflated. T[] is the 
 dynamic array. There's a block of memory managed by the GC 
 underneath, but it's completely hidden from the programmer. It 
 is _not_ the dynamic array. It's just a block of memory 
 managed by the GC which is used to manage the memory for 
 dynamic arrays, and it has a completely different type from 
 T[]. It's not even an array in the D sense. The block of 
 memory is referred to by a pointer, not a D array.

 It's talk of "which array" owns the memory and the like which 
 causes confusion, and talking about the GC-managed block of 
 memory as being the dynamic array and the T[] that the 
 programmer sees as being the slice is just plain wrong. The 
 T[] is both a slice and a dynamic array (because they're the 
 same thing), whereas the block of memory is neither. I think 
 that the fact that the article tried to call the underlying 
 block of memory a dynamic array has caused a lot of 
 unnecessary confusion.

 - Jonathan M Davis
This very thread is a testament to the fact that no, it's not completely hidden, and it is the conflation that causes confusion, and clarity only comes when the two concepts are explained separately. This matches my experience when I teach about D's arrays on IRC too - people generally feel enlightened after reading the array article. It's debatable whether hamfisting dynamic array capabilities onto a simple slice type was a good idea or not (T[new], anyone?), but we should be as clear with the current situation as possible. Pretending there isn't a problem is ignoring the cries of D newbies everywhere. Please stop.
The array article does a great job explaining a lot about D arrays, and it definitely has helped people understand them. But it uses the wrong terminology, and that's my point. In D, T[] is a dynamic array and a slice. As far as D is concerned, there is no difference. T[] normally refers to a block of memory that the GC manages, but that block of memory is not typed as a dynamic array. And that's where the article gets it wrong. - Jonathan M Davis
Jul 27 2014
parent reply "Andrew Godfrey" <X y.com> writes:
 The array article does a great job explaining a lot about D 
 arrays, and it definitely has helped people understand them. 
 But it uses the wrong terminology, and that's my point. In D, 
 T[] is a dynamic array and a slice. As far as D is concerned, 
 there is no difference. T[] normally refers to a block of 
 memory that the GC manages, but that block of memory is not 
 typed as a dynamic array. And that's where the article gets it 
 wrong.
Look: auto c = new MyClass(); The D spec calls c "a reference to a class instance". This is good terminology because there can be a many-to-one relationship between references and classes. In contrast, early writing on Java or C# (I forget which) used to call c "an object". This was a misguided attempt to hide the fact that there's a reference involved - a fact that cannot and should not be hidden. And the thing it refers to needs a name more specific than "a block of memory". Now: int[] i = new int[5]; For now let's call 'i' a "dynamic array" and the thing it refers to a "foobar". The point is still the same: As with object references, there can be a many-to-one relationship between "dynamic arrays" and "foobars", and attempts to hide that relationship (e.g. by not giving "foobar" a name more specific than "block of memory") are misguided. Furthermore, the concept we're calling a "foobar" here is usually called a "dynamic array" in other languages. So IMO D is making the same mistake as when people call c above "an object".
Jul 27 2014
parent "Jesse Phillips" <Jesse.K.Phillips+D gmail.com> writes:
On Sunday, 27 July 2014 at 18:52:55 UTC, Andrew Godfrey wrote:
 Now:

 int[] i = new int[5];

 For now let's call 'i' a "dynamic array" and the thing it 
 refers to
 a "foobar".
'i' is a "dynamic array" and contains a reference to a "block of memory." Frankly I don't see how making the distinction at 'dynamic array' and 'slice' will help with the particular confusion. For example, in Go a slice stomps memory: http://he-the-great.livejournal.com/48672.html
Jul 28 2014
prev sibling parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 07/27/2014 01:49 AM, Jonathan M Davis wrote:

 As defined by the language, T[] is a dynamic array.
I bet that terminology is from before the design of slices was finalized about four years ago.
 difference between a dynamic array and a slice (there really isn't any).
At the risk of beating this never-dying horse, when you say that, I don't think you can explain the following code: int[] a = [ 1, 2, 3 ]; int[] b = a; According to your terminology, 'a' is a dynamic array and 'b' is another dynamic array. Knowing how good you know D, I can't understand how you think that way. In fact, both 'a' and 'b' are slices into the single dynamic array. The dynamic array is managed by the GC. So, there are three entities in that code: One dynamic array and two slices.
 IIRC, both Walter and Andrei stated in that discussion that T[]
 is a dynamic array as far as D is concerned and that's not
 going to change.
I would like to hear Andrei's and Walter's current opinions on the matter.
 The article really should be updated to reflect the correct
 terminology.
It should stay as is to maintain sanity. :)
 As far as D is concerned a slice and a dynamic array are the
 same thing when it comes to arrays. They're just different
 names for T[], and trying to treat them as different just
 causes confusion.
To me, treating them differently removes all the confusion.
 - Jonathan M Davis
Ali
Jul 28 2014
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Monday, 28 July 2014 at 22:29:04 UTC, Ali Çehreli wrote:
 On 07/27/2014 01:49 AM, Jonathan M Davis wrote:
 difference between a dynamic array and a slice (there really
isn't any). At the risk of beating this never-dying horse, when you say that, I don't think you can explain the following code: int[] a = [ 1, 2, 3 ]; int[] b = a; According to your terminology, 'a' is a dynamic array and 'b' is another dynamic array. Knowing how good you know D, I can't understand how you think that way. In fact, both 'a' and 'b' are slices into the single dynamic array. The dynamic array is managed by the GC. So, there are three entities in that code: One dynamic array and two slices.
It's simple, D's dynamic arrays _never_ own their memory. It's owned by the GC, or it's a slice of a static array, or it points to a buffer that was malloc-ed, etc. The type T[] says _nothing_ about who or what owns the memory or what backs the array. Understanding how it works when they're generated by the GC is very helpful in determining when a dynamic array / slice is likely to be reallocated, but it's pretty irrelevant to the type itself. In your example, both a and b are dynamic arrays which are slices of the same underlying block of memory. It's a and b which are the dynamic arrays, not the buffer. From the type system's perspective, this would be the same int[3] z = [1, 2, 3]; int[] a = z; int[] b = a; a and by are still exactly the same type - dynamic arrays - and they will behave exactly the same save for the fact that due to the fact that they're now backed by a static array rather than the GC, their capacity is guaranteed to be 0, and it's unsafe for them to escape from the scope of z. The underlying buffer for a GC-backed dynamic array isn't even a D array. It's a druntime-specific data structure which holds a buffer via pointer. The array article does a great job of explaining a lot of the details about how D arrays work (especially those backed by the GC), but it got the terminology wrong, and I definitely think that it should be fixed. - Jonathan M Davis
Jul 28 2014
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
You don't say anything below that does not work when I replace "dynamic 
array"s with "slice"s. Let's see... (I mark every such 
replaced-by-me-"slice" with double stars.)

On 07/28/2014 06:35 PM, Jonathan M Davis wrote:

 On Monday, 28 July 2014 at 22:29:04 UTC, Ali Çehreli wrote:
 On 07/27/2014 01:49 AM, Jonathan M Davis wrote:
 difference between a dynamic array and a slice (there really
isn't any). At the risk of beating this never-dying horse, when you say that, I don't think you can explain the following code: int[] a = [ 1, 2, 3 ]; int[] b = a; According to your terminology, 'a' is a dynamic array and 'b' is another dynamic array. Knowing how good you know D, I can't understand how you think that way. In fact, both 'a' and 'b' are slices into the single dynamic array. The dynamic array is managed by the GC. So, there are three entities in that code: One dynamic array and two slices.
It's simple, D's **slices** _never_ own their memory. It's owned by the GC, or it's a slice of a static array, or it points to a buffer that was malloc-ed, etc. The type T[] says _nothing_ about who or what owns the memory or what backs the array. Understanding how it works when they're generated by the GC is very helpful in determining when a **slice** / slice is likely to be reallocated, but it's pretty irrelevant to the type itself. In your example, both a and b are **slices** which are slices of the same underlying block of memory. It's a and b which are the **slices**, not the buffer. From the type system's perspective, this would be the same int[3] z = [1, 2, 3]; int[] a = z; int[] b = a; a and by are still exactly the same type - **slices** - and they will behave exactly the same save for the fact that due to the fact that they're now backed by a static array rather than the GC, their capacity is guaranteed to be 0, and it's unsafe for them to escape from the scope of z. The underlying buffer for a GC-backed **slice** isn't even a D array. It's a druntime-specific data structure which holds a buffer via pointer. The array article does a great job of explaining a lot of the details about how D arrays work (especially those backed by the GC), but it got the terminology wrong, and I definitely think that it should be fixed. - Jonathan M Davis
The array article uses terminology that removes confusion. Ali
Jul 29 2014
parent Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, 29 Jul 2014 02:54:37 -0700
Ali Çehreli via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 You don't say anything below that does not work when I replace
 "dynamic array"s with "slice"s. Let's see... (I mark every such
 replaced-by-me-"slice" with double stars.)
Because they're the same thing as far as D is concerned.
 On 07/28/2014 06:35 PM, Jonathan M Davis wrote:

  > On Monday, 28 July 2014 at 22:29:04 UTC, Ali Çehreli wrote:
  >> On 07/27/2014 01:49 AM, Jonathan M Davis wrote:
  >> > difference between a dynamic array and a slice (there really
  >> isn't any).
  >>
  >> At the risk of beating this never-dying horse, when you say that,
  >> I don't think you can explain the following code:
  >>
  >>     int[] a = [ 1, 2, 3 ];
  >>     int[] b = a;
  >>
  >> According to your terminology, 'a' is a dynamic array and 'b' is
  >> another dynamic array. Knowing how good you know D, I can't
  >> understand how you think that way. In fact, both 'a' and 'b' are
  >> slices into the single dynamic array. The dynamic array is
  >> managed by the GC. So, there are three entities in that code: One
  >> dynamic array and two slices.
  >
  > It's simple, D's **slices** _never_ own their memory. It's owned by
  > the GC, or it's a slice of a static array, or it points to a
  > buffer that was malloc-ed, etc. The type T[] says _nothing_ about
  > who or what owns the memory or what backs the array. Understanding
  > how it works when they're generated by the GC is very helpful in
  > determining when a **slice** / slice is likely to be reallocated,
  > but it's pretty irrelevant to the type itself.
  >
  > In your example, both a and b are **slices** which are slices of
  > the same underlying block of memory. It's a and b which are the
  > **slices**, not the buffer. From the type system's perspective,
  > this would be the same
  >
  > int[3] z = [1, 2, 3];
  > int[] a = z;
  > int[] b = a;
  >
  > a and by are still exactly the same type - **slices** - and they
  > will behave exactly the same save for the fact that due to the
  > fact that they're now backed by a static array rather than the GC,
  > their capacity is guaranteed to be 0, and it's unsafe for them to
  > escape from the scope of z.
  >
  > The underlying buffer for a GC-backed **slice** isn't even a D
  > array. It's a druntime-specific data structure which holds a
  > buffer via pointer.
  >
  > The array article does a great job of explaining a lot of the
  > details about how D arrays work (especially those backed by the
  > GC), but it got the terminology wrong, and I definitely think that
  > it should be fixed.
  >
  > - Jonathan M Davis

 The array article uses terminology that removes confusion.
It uses terminology that does not match the language spec. It could use the correct terminology and still remove confusion. - Jonathan M Davis
Jul 29 2014
prev sibling parent reply "Andrew Godfrey" <X y.com> writes:
On Sunday, 27 July 2014 at 05:51:46 UTC, Jakob Ovrum wrote:
 On Saturday, 26 July 2014 at 23:06:02 UTC, Andrew Godfrey wrote:
 Thereafter can come sub-slice examples and so on.
 Does this make sense?
Yes, the reference documentation is pretty terrible with naming of various array concepts. IIRC, when this was discussed in the past, a majority seemed to be in favour of using "slice" and "dynamic array" for their respective concepts instead of the current situation, but I also remember there was some opposition (for reason I can't remember). A pull request updating the documentation to use slice/dynamic array might weed them out ;)
I gave this a try, and overall it looks like an improvement, but I think we need another name than "slice". The reason is that the slice operator is a distinct thing and interacts with the "slice" in strange ways. When I next get time I'll try updating it to use the term "array reference". That is: int[] a; // defines an array reference, a int[3] b; a = b[1..3]; // updates the array reference a to refer to a slice of b
Jul 29 2014
next sibling parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Tuesday, 29 July 2014 at 07:46:34 UTC, Andrew Godfrey wrote:
 On Sunday, 27 July 2014 at 05:51:46 UTC, Jakob Ovrum wrote:
 On Saturday, 26 July 2014 at 23:06:02 UTC, Andrew Godfrey 
 wrote:
 Thereafter can come sub-slice examples and so on.
 Does this make sense?
Yes, the reference documentation is pretty terrible with naming of various array concepts. IIRC, when this was discussed in the past, a majority seemed to be in favour of using "slice" and "dynamic array" for their respective concepts instead of the current situation, but I also remember there was some opposition (for reason I can't remember). A pull request updating the documentation to use slice/dynamic array might weed them out ;)
I gave this a try, and overall it looks like an improvement, but I think we need another name than "slice". The reason is that the slice operator is a distinct thing and interacts with the "slice" in strange ways. When I next get time I'll try updating it to use the term "array reference". That is: int[] a; // defines an array reference, a int[3] b; a = b[1..3]; // updates the array reference a to refer to a slice of b
IMO slice fits quite well for both. `b[1..3]` is a slice (or refers to one?), and `a` is, too. After the assignment, both slices are equal. But I see that there is an ambiguity when we talk about "copying a slice", which could also be interpreted as "copying what the slice refers to".
Jul 29 2014
parent reply "Andrew Godfrey" <X y.com> writes:
On Tuesday, 29 July 2014 at 08:41:48 UTC, Marc Schütz wrote:
 On Tuesday, 29 July 2014 at 07:46:34 UTC, Andrew Godfrey wrote:
 I gave this a try, and overall it looks like an improvement, 
 but I think we need another name than "slice". The reason is 
 that the slice operator is a distinct thing and interacts with 
 the "slice" in strange ways. When I next get time I'll try 
 updating it to use the term "array reference". That is:

 int[] a; // defines an array reference, a
 int[3] b;
 a = b[1..3]; // updates the array reference a to refer to a 
 slice of b
IMO slice fits quite well for both. `b[1..3]` is a slice (or refers to one?), and `a` is, too. After the assignment, both slices are equal. But I see that there is an ambiguity when we talk about "copying a slice", which could also be interpreted as "copying what the slice refers to".
That wasn't meant to be a motivating example. The totality of what I have so far is really what makes me think this - it's too big to share here, but I suppose I could submit a PR for rejection? Here's an attempt at a short motivating example: int[3] a = [1, 0, -1]; int[] b = a; int[] c = new int[4]; b[] = c[1..4]; assert(a[2] == 0); // Using the proposed 'slice' term, an exhaustive description of the above operation is: // This copies a slice of the slice c onto the slice b (which refers to a). // // If we instead call b and c "array references", the description reads: // This copies a slice of the array referenced by c, onto the array referenced by b (which is a).
Jul 29 2014
parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Tuesday, 29 July 2014 at 16:54:48 UTC, Andrew Godfrey wrote:
 On Tuesday, 29 July 2014 at 08:41:48 UTC, Marc Schütz wrote:
 On Tuesday, 29 July 2014 at 07:46:34 UTC, Andrew Godfrey wrote:
 I gave this a try, and overall it looks like an improvement, 
 but I think we need another name than "slice". The reason is 
 that the slice operator is a distinct thing and interacts 
 with the "slice" in strange ways. When I next get time I'll 
 try updating it to use the term "array reference". That is:

 int[] a; // defines an array reference, a
 int[3] b;
 a = b[1..3]; // updates the array reference a to refer to a 
 slice of b
IMO slice fits quite well for both. `b[1..3]` is a slice (or refers to one?), and `a` is, too. After the assignment, both slices are equal. But I see that there is an ambiguity when we talk about "copying a slice", which could also be interpreted as "copying what the slice refers to".
That wasn't meant to be a motivating example. The totality of what I have so far is really what makes me think this - it's too big to share here, but I suppose I could submit a PR for rejection? Here's an attempt at a short motivating example: int[3] a = [1, 0, -1]; int[] b = a; int[] c = new int[4]; b[] = c[1..4]; assert(a[2] == 0); // Using the proposed 'slice' term, an exhaustive description of the above operation is: // This copies a slice of the slice c onto the slice b (which refers to a). // // If we instead call b and c "array references", the description reads: // This copies a slice of the array referenced by c, onto the array referenced by b (which is a).
Playing the devil's advocate: Yes, it does so by assigning a slice from `c` to a slice from `b`.
Jul 29 2014
parent reply "Andrew Godfrey" <X y.com> writes:
On Tuesday, 29 July 2014 at 20:06:28 UTC, Marc Schütz wrote:
    int[3] a = [1, 0, -1];
    int[] b = a;
    int[] c = new int[4];

    b[] = c[1..4];
    assert(a[2] == 0);

    // This copies a slice of the slice c onto the slice b 
 (which refers to a).
Playing the devil's advocate: Yes, it does so by assigning a slice from `c` to a slice from `b`.
Let's try to explain why this: b = c[1..4]; changes what b refers to, while this: b[] = c[1..4]; copies. Well, if you assign a slice to a slice, that means 'copy elements'. But if you assign a slice to <what b is>, that means change the reference.
Jul 29 2014
parent reply "Andrew Godfrey" <X y.com> writes:
fyi, here's what I have so far.
I haven't yet added the cross-references we talked about at the
start of the thread. I'll be away for a few weeks soon, so
won't have much more time until after that.

I'm hoping this link is public. It seemed public:

https://github.com/AndrewGodfrey/dlang.org/commit/463b5656151616e2a164ad414cf2bd61d1b84beb
Jul 29 2014
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, 30 July 2014 at 02:08:30 UTC, Andrew Godfrey wrote:
 fyi, here's what I have so far.
 I haven't yet added the cross-references we talked about at the
 start of the thread. I'll be away for a few weeks soon, so
 won't have much more time until after that.

 I'm hoping this link is public. It seemed public:

 https://github.com/AndrewGodfrey/dlang.org/commit/463b5656151616e2a164ad414cf2bd61d1b84beb
I'm completely opposed to changing the official terminology. Per both the spec and TDPL T[] is both a dynamic array and a slice, and I see no reason to change that. And as far as I can tell it's in line with how dynamic arrays are described on Wikipedia: https://en.wikipedia.org/wiki/Dynamic_array It's just that the dynamic arrays in D do not own their own memory, and it's the GC which manages dealing with expanding the memory with amortized cost. So, as far as I can tell, T[] follows the expected semantics of dynamic arrays with a few extra capabilities tacked on. What does trying to change the terminology buy you? Having it be more in line with the array article? If anything, the article should be updated to match the official terminology, not change the terminology to match the article. It's correct to say that T[] is a slice, because it's always a slice of some chunk of memory - be it from the GC, a static array, malloc, etc. And it's correct to say that T[] is a dynamic array, because it supports all of the operations that a dynamic array is supposed to support. It just comes with the added benefit of being able to have multiple arrays refer to the same memory until one of them has to reallocate due to a lack of available capacity (which is exactly how dynamic arrays work in other languages when you append to them and they don't have enough available space to grow in place). I do not understand how any of you can claim that T[] is not a dynamic array. Just because it doesn't manage its own memory? The GC manages it in a manner that gives T[] all of the expected semantics of a dynamic array with the added benefit that we can even append onto a dynamic array that's a slice of non-GC-allocated memory (though that makes it so that it is then GC-allocated when it gets reallocated). What about T[] is _not_ a dynamic array? - Jonathan M Davis
Jul 29 2014
next sibling parent reply "Dominikus Dittes Scherkl" writes:
On Wednesday, 30 July 2014 at 05:55:58 UTC, Jonathan M Davis 
wrote:
 I'm completely opposed to changing the official terminology.
Why? What buys it, to have two terms "slice" and "dynamic array" if they mean exactly the same thing? Especially if we have two different things, the memory and the reference to the memory, but both terms are only adressin the second thing. I would prefer to have one term meaning the one thing and the other term meaning the other thing. Words are changing meaning over the years, and I think it is very instructive to use "dynamic array" to mean the memory, and "slice" to mean the reference to the memory - as done in the article. May be that was not the original meaning of those terms but it is useful, instructive and unambigous, so it should go in the official terminology. (otherwise we would need a new, third term to describe the "other thing" that is not directly accessible in the current language spec wording - but we have none and it would only increase the confusion).
Jul 29 2014
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, 30 July 2014 at 06:11:58 UTC, Dominikus Dittes 
Scherkl wrote:
 On Wednesday, 30 July 2014 at 05:55:58 UTC, Jonathan M Davis 
 wrote:
 I'm completely opposed to changing the official terminology.
Why? What buys it, to have two terms "slice" and "dynamic array" if they mean exactly the same thing? Especially if we have two different things, the memory and the reference to the memory, but both terms are only adressin the second thing. I would prefer to have one term meaning the one thing and the other term meaning the other thing. Words are changing meaning over the years, and I think it is very instructive to use "dynamic array" to mean the memory, and "slice" to mean the reference to the memory - as done in the article. May be that was not the original meaning of those terms but it is useful, instructive and unambigous, so it should go in the official terminology. (otherwise we would need a new, third term to describe the "other thing" that is not directly accessible in the current language spec wording - but we have none and it would only increase the confusion).
The "other thing" is just a block of memory managed by the GC and which keeps track of the farthest into the block a particular slice / dynamic array has grown. It doesn't actually act like a dynamic array at all, because it doesn't ever grow. It's just used by the GC to provide memory for dynamic arrays and is only associated with a particular dynamic array until that array is reallocated. Calling that block of memory a dynamic array would be like calling the guts of a std::vector a dynamic array instead calling std::vector a dynamic array. It's just the memory and bookkeeping that backs the dynamic array. And dynamic array and slice are already not entirely the same, because of more general slicing operations. A dynamic array is always a slice, but if you're talking about the language in general and not arrays specifically, a slice could be something else - e.g. you can slice a DList to get a range over it. It's less common that those are referred to as slices, but they essentially are, and you use opSlice to get them. I'm increasingly tempted to just write up a new article that uses the correct terminology. And if I can do that properly, then maybe it'll help clear up the confusion. - Jonathan M Davis
Jul 31 2014
parent "Dominikus Dittes Scherkl" writes:
On Friday, 1 August 2014 at 06:18:06 UTC, Jonathan M Davis wrote:
 On Wednesday, 30 July 2014 at 06:11:58 UTC, Dominikus Dittes 
 Scherkl wrote:
 [...]
 should go in the official terminology. (otherwise we would 
 need a new, third term to describe the "other thing" that is 
 not directly accessible in the current language spec wording - 
 but we have none and it would only increase the confusion).
The "other thing" is just a block of memory managed by the GC and which keeps track of the farthest into the block a particular slice / dynamic array has grown. It doesn't actually act like a dynamic array at all, because it doesn't ever grow.
Really? If I append to my slice, a new "just block of memory" gets allocated, data copied there and my reference changed to point to this new block. Isn't that what everybody describes as that what a dynamic array does? Ok, I'm not absolutely sure which part of the runtime performs these changes - if it is the GC or the opAppend of arrays or whatever, but the block of memory together with these operations is really what I would call a "dynamic array".
 It's just used by the GC to provide memory for dynamic arrays 
 and is only associated with a particular dynamic array until 
 that array is reallocated. Calling that block of memory a 
 dynamic array would be like calling the guts of a std::vector a 
 dynamic array instead calling std::vector a dynamic array. It's 
 just the memory and bookkeeping that backs the dynamic array.
If a distinction between vector and its backing would be useful, yes I would say it is reasonable to call the backing "dynamic array".
 And dynamic array and slice are already not entirely the same, 
 because of more general slicing operations. A dynamic array is 
 always a slice, but if you're talking about the language in 
 general and not arrays specifically, a slice could be something 
 else - e.g. you can slice a DList to get a range over it. It's 
 less common that those are referred to as slices, but they 
 essentially are, and you use opSlice to get them.
Nice. So we have slices with dynamic array as background and we have slices with different background - even more reason to distinguish slice and dynamic array cleanly.
 I'm increasingly tempted to just write up a new article that 
 uses the correct terminology. And if I can do that properly, 
 then maybe it'll help clear up the confusion.
Yes, do that. I'm eager to read it and see, if it is still as enlightening as the old article was.
Aug 01 2014
prev sibling parent reply "Andrew Godfrey" <X y.com> writes:
 What about T[] is _not_ a dynamic array?
Now that I've done this exercise I can answer more crisply: When T[] is an lvalue, it behaves like a reference, not a dynamic array.
Jul 30 2014
next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, 30 July 2014 at 09:52:15 UTC, Andrew Godfrey wrote:
 What about T[] is _not_ a dynamic array?
Now that I've done this exercise I can answer more crisply: When T[] is an lvalue, it behaves like a reference, not a dynamic array.
So the fact that it slices rather than copies is enough to make it not a dynamic array. As far as I can tell, the copying behavior of a dynamic array has nothing to do with it being a dynamic array. It's the facts that you can append to, it maintains extra capacity to make appending efficient, and it reallocates when it runs out of extra capacity (making appending an amortized O(1)) that make an array a dynamic array. And T[] does that. - Jonathan M Davis
Jul 31 2014
prev sibling parent reply "Andrew Godfrey" <X y.com> writes:
Going through other .dd files, I found an error in expression.dd.
It says "For static or dynamic arrays, identity is defined as 
referring
to the same array elements and the same number of elements."

Well, in fact:

unittest {
     // expression.dd says that equality AND IDENTITY compare 
elements and sizes, for both "static and dynamic arrays". Gaah!

     int[] a = new int[3];
     int[] b = new int[3];
     assert(a == b);
     assert(a !is b); // Nope! The doc is wrong!

     // So then:

     b = a;
     assert(b is a); // Now b points to a, and 'is' does what I'd 
expect.
     // But evidently it's because it compared the array 
references - not
     // the elements and sizes!
}

I would NOT recommend updating the compiler to match what the doc 
says.
The current behavior is consistent with how assignment to an 
array reference behaves.
Aug 01 2014
next sibling parent "Andrew Godfrey" <X y.com> writes:
Pull request:
https://github.com/D-Programming-Language/dlang.org/pull/623

fyi, I will be away for 3 weeks, mostly unavailable but
may be able to respond occasionally.
Aug 01 2014
prev sibling next sibling parent "Andrew Godfrey" <X y.com> writes:
On Friday, 1 August 2014 at 07:51:32 UTC, Andrew Godfrey wrote:
 Going through other .dd files, I found an error in 
 expression.dd.
 It says "For static or dynamic arrays, identity is defined as 
 referring
 to the same array elements and the same number of elements."

 Well, in fact:

 unittest {
     // expression.dd says that equality AND IDENTITY compare 
 elements and sizes, for both "static and dynamic arrays". Gaah!

     int[] a = new int[3];
     int[] b = new int[3];
     assert(a == b);
     assert(a !is b); // Nope! The doc is wrong!

     // So then:

     b = a;
     assert(b is a); // Now b points to a, and 'is' does what 
 I'd expect.
     // But evidently it's because it compared the array 
 references - not
     // the elements and sizes!
 }

 I would NOT recommend updating the compiler to match what the 
 doc says.
 The current behavior is consistent with how assignment to an 
 array reference behaves.
Rereading this, the original text did say "refers to". So calling that an "error" may be a bit strong.
Aug 01 2014
prev sibling parent reply "Chris Cain" <zshazz gmail.com> writes:
On Friday, 1 August 2014 at 07:51:32 UTC, Andrew Godfrey wrote:
 Going through other .dd files, I found an error in 
 expression.dd.
 It says "For static or dynamic arrays, identity is defined as 
 referring
 to the same array elements and the same number of elements."

 Well, in fact:

 unittest {
     // expression.dd says that equality AND IDENTITY compare 
 elements and sizes, for both "static and dynamic arrays". Gaah!

     int[] a = new int[3];
     int[] b = new int[3];
     assert(a == b);
     assert(a !is b); // Nope! The doc is wrong!

     // So then:

     b = a;
     assert(b is a); // Now b points to a, and 'is' does what 
 I'd expect.
     // But evidently it's because it compared the array 
 references - not
     // the elements and sizes!
 }

 I would NOT recommend updating the compiler to match what the 
 doc says.
 The current behavior is consistent with how assignment to an 
 array reference behaves.
I think the docs might can be cleared up in this regard, but from my reading, the docs are correct on this. int[] a = new int[3]; and int[] b = new int[3]; Do not refer to the same elements. They are two different allocations that happen to look alike and compare equal, but they aren't referring to the same point in memory. To prove this, you can simply do `a[0] = 1;` ... did `b[0]` "change" as well? (No). Then they couldn't possibly have been referring to the same thing. What the docs are saying by "referring to the same array elements and the same number of elements": basically, an array in D is simply a pointer + length struct. The `is` operator essentially does a bitwise compare, so naturally `a is b` only returns true when they are bitwise identical structs, which must mean they are the same in both pointer and in length, which naturally means that `a == b` as well. There's no need to compare elements since there's no way something doesn't equal itself unless some weird trickery is going on (concurrent modification?). You can imagine `is` to mean `a.ptr == b.ptr && a.length == b.length` (or that they refer to the same array and they have the same number of elements), but implemented in what is probably a more efficient manner.
Aug 01 2014
parent reply "Andrew Godfrey" <X y.com> writes:
On Friday, 1 August 2014 at 21:36:14 UTC, Chris Cain wrote:
 On Friday, 1 August 2014 at 07:51:32 UTC, Andrew Godfrey wrote:
 Going through other .dd files, I found an error in 
 expression.dd.
 It says "For static or dynamic arrays, identity is defined as 
 referring
 to the same array elements and the same number of elements."

 Well, in fact:

 unittest {
    // expression.dd says that equality AND IDENTITY compare 
 elements and sizes, for both "static and dynamic arrays". Gaah!

    int[] a = new int[3];
    int[] b = new int[3];
    assert(a == b);
    assert(a !is b); // Nope! The doc is wrong!

    // So then:

    b = a;
    assert(b is a); // Now b points to a, and 'is' does what 
 I'd expect.
    // But evidently it's because it compared the array 
 references - not
    // the elements and sizes!
 }

 I would NOT recommend updating the compiler to match what the 
 doc says.
 The current behavior is consistent with how assignment to an 
 array reference behaves.
I think the docs might can be cleared up in this regard, but from my reading, the docs are correct on this. int[] a = new int[3]; and int[] b = new int[3]; Do not refer to the same elements. They are two different allocations that happen to look alike and compare equal, but they aren't referring to the same point in memory. To prove this, you can simply do `a[0] = 1;` ... did `b[0]` "change" as well? (No). Then they couldn't possibly have been referring to the same thing. What the docs are saying by "referring to the same array elements and the same number of elements": basically, an array in D is simply a pointer + length struct. The `is` operator essentially does a bitwise compare, so naturally `a is b` only returns true when they are bitwise identical structs, which must mean they are the same in both pointer and in length, which naturally means that `a == b` as well. There's no need to compare elements since there's no way something doesn't equal itself unless some weird trickery is going on (concurrent modification?). You can imagine `is` to mean `a.ptr == b.ptr && a.length == b.length` (or that they refer to the same array and they have the same number of elements), but implemented in what is probably a more efficient manner.
Yes, I was a bit terse in my previous retraction (was low on time). I realized this is what the doc was actually saying. The edit in the PR stands, but my description of it as fixing an "error" is incorrect. The language was just a bit confusing, perhaps. I am unable to fix the PR description at the moment but that shouldn't prevent review of the PR itself.
Aug 01 2014
parent reply "Andrew Godfrey" <x y.com> writes:
Reminder: The PR is ready for review:

https://github.com/D-Programming-Language/dlang.org/pull/623

Jonathan has summarized his position in the commments.
What do the rest of you think?
H. S. Teoh, Jakob, Ali, Marc, Dominikus, Chris -
your impression of whether this clears up the confusion would
help round out the feedback.
Aug 11 2014
next sibling parent "Dominikus Dittes Scherkl" writes:
On Monday, 11 August 2014 at 07:04:42 UTC, Andrew Godfrey wrote:
 Reminder: The PR is ready for review:

 https://github.com/D-Programming-Language/dlang.org/pull/623

 Jonathan has summarized his position in the commments.
 What do the rest of you think?
 H. S. Teoh, Jakob, Ali, Marc, Dominikus, Chris -
 your impression of whether this clears up the confusion would
 help round out the feedback.
Very good. Most of the documentation becomes imediately much more intuitive and instructive. Beside some typos (at one place I saw an uppercase C where a valiable "c" was menat, in another place a variable c is refered but only b was declared), I'm very much in favor of those changes.
Aug 11 2014
prev sibling next sibling parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Mon, 11 Aug 2014 07:04:41 +0000
Andrew Godfrey via Digitalmars-d <digitalmars-d puremagic.com> wrote:

Jonathan is right. what this PR does is changing one (somewhat
confusing) terminology to another, even more confusing one.
Aug 11 2014
parent reply "Andrew Godfrey" <X y.com> writes:
On Monday, 11 August 2014 at 19:43:26 UTC, ketmar via 
Digitalmars-d wrote:
 On Mon, 11 Aug 2014 07:04:41 +0000
 Andrew Godfrey via Digitalmars-d <digitalmars-d puremagic.com> 
 wrote:

 Jonathan is right. what this PR does is changing one (somewhat
 confusing) terminology to another, even more confusing one.
Thanks for adding your opinion - it's good to see more support for Jonathan's side. Don't think I'm being flippant, but I have trouble interpreting such feedback, because D's dynamic array semantics ARE complicated. So if I happened to succeed in making a clear, accurate description of how it works, I would expect it to take many readers from an inaccurate, overly-simplistic understanding to an accurate (but unfortunately more complex) one. I have cause to question this - the docs themselves had some important misunderstandings - clear errors. It's possible that this was cruft and the authors never had those misunderstandings themselves - but some of their readers certainly did.
Aug 13 2014
next sibling parent "Andrew agodfrey" <X y.com> writes:
I've broken out the less controversial fixes into a separate PR:
https://github.com/D-Programming-Language/dlang.org/pull/629
Aug 17 2014
prev sibling next sibling parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Thu, 14 Aug 2014 06:46:40 +0000
Andrew Godfrey via Digitalmars-d <digitalmars-d puremagic.com> wrote:

sorry for the late answer.

 Don't think I'm being flippant, but I have trouble interpreting=20
 such feedback, because D's dynamic array semantics ARE=20
 complicated.
and it will be even more complicated if we will rename 'em to 'array references'. i completely agree that D dynarray docs can be made clearer and i appreciate any efforts which makes docs better. i'm just against 'reference' term -- seems that i'm from that minority that becomes immediately confused when reading 'array reference'. what i expect from 'reference' is that this code works: void foo (int[] a) { a ~=3D 42; } ... int[] arr; arr ~=3D 666; foo(arr); assert(arr.length =3D=3D 1 && arr[0] =3D=3D 42); but it doesn't. so arrays clearly aren't 'references' (as 'reference', to my opinion, should keep 'pass by reference' semantics in this case). so maybe we should coin some new word to describe dynarrays in D.
Aug 17 2014
parent reply "Andrew Godfrey" <x y.com> writes:
On Sunday, 17 August 2014 at 07:33:01 UTC, ketmar via 
Digitalmars-d wrote:
 On Thu, 14 Aug 2014 06:46:40 +0000
 Andrew Godfrey via Digitalmars-d <digitalmars-d puremagic.com> 
 wrote:

 sorry for the late answer.

 Don't think I'm being flippant, but I have trouble 
 interpreting such feedback, because D's dynamic array 
 semantics ARE complicated.
and it will be even more complicated if we will rename 'em to 'array references'. i completely agree that D dynarray docs can be made clearer and i appreciate any efforts which makes docs better. i'm just against 'reference' term -- seems that i'm from that minority that becomes immediately confused when reading 'array reference'. what i expect from 'reference' is that this code works: void foo (int[] a) { a ~= 42; } ... int[] arr; arr ~= 666; foo(arr); assert(arr.length == 2 && arr[1] == 42); but it doesn't. so arrays clearly aren't 'references' (as 'reference', to my opinion, should keep 'pass by reference' semantics in this case). so maybe we should coin some new word to describe dynarrays in D.
Maybe. But consider that if "a" was a class and "~=" was instead "=", you would have the same confusion. As in: void foo(MyClass a) { a = new MyClass; } In either case, we are "passing a reference by value". So operations on the referent affect the passed object/array, but operations on the reference do not. The further surprise is caused because concatenation affects the _reference_. In fact it doesn't affect the referent at all! Good example. Consider that this works: void foo (int[] a) { a[0] = 42; } int[] arr; arr ~= 666; foo(arr); assert(arr.length == 1 && arr[0] == 42);
Aug 19 2014
parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, 19 Aug 2014 23:58:57 +0000
Andrew Godfrey via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 In either case, we are "passing a reference by value".
yes. but passing null class will not allow to call it's methods, and passing null array will. i.e. auto foo (MyClass a) { return a.length; } will crash if length is not a static method (and it can't be static) and foo is called like this: foo(null). but the same call will work for 'int[]'. so user can assume that 'int[]' is something similar to class, and it's perfectly ok to use int[] methods, and '~=3D' is just the overloaded method, which should change contents of passed dynarray. again, overloaded '~=3D' will work for class (more or less), but will inevitably fail (i.e. will not change passed arg) for dynarray. and it's clear that 'a =3D new MyClass()' creates new object. and it's not clear at all that 'a ~=3D 42' creates new object too. it's counterintuitive. that's why i think that we need new term.
Aug 19 2014
parent reply "Andrew Godfrey" <x y.com> writes:
On Wednesday, 20 August 2014 at 00:13:32 UTC, ketmar via 
Digitalmars-d wrote:
 On Tue, 19 Aug 2014 23:58:57 +0000
 Andrew Godfrey via Digitalmars-d <digitalmars-d puremagic.com> 
 wrote:

 In either case, we are "passing a reference by value".
yes. but passing null class will not allow to call it's methods, and passing null array will. i.e. auto foo (MyClass a) { return a.length; } will crash if length is not a static method (and it can't be static) and foo is called like this: foo(null). but the same call will work for 'int[]'. so user can assume that 'int[]' is something similar to class, and it's perfectly ok to use int[] methods, and '~=' is just the overloaded method, which should change contents of passed dynarray. again, overloaded '~=' will work for class (more or less), but will inevitably fail (i.e. will not change passed arg) for dynarray. and it's clear that 'a = new MyClass()' creates new object. and it's not clear at all that 'a ~= 42' creates new object too. it's counterintuitive. that's why i think that we need new term.
Well, this is just one case of a general confusion that classes and dynamic arrays already share, by virtue of having hidden reference semantics (i.e. the "." operator dereferences in some cases and not in others, sometimes surprising novices). You can see this in the existing array doc (http://dlang.org/arrays) where it describes the "sizeof" property for dynamic arrays. sizeof returns the size of the "dynamic array reference". A novice might expect it to return the size of the pointed-to array, to act more like the static-array case.
Aug 20 2014
next sibling parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Thu, 21 Aug 2014 03:24:35 +0000
Andrew Godfrey via Digitalmars-d <digitalmars-d puremagic.com> wrote:

maybe just call that "slice views"? ;-)

really, uncommon term will (at least it should ;-) make user to read
about that "slice views", and "reference" is something like "ah, i know
what references is, let's start coding now!"
Aug 20 2014
parent reply "Andrew Godfrey" <x y.com> writes:
On Thursday, 21 August 2014 at 03:53:42 UTC, ketmar via 
Digitalmars-d wrote:
 On Thu, 21 Aug 2014 03:24:35 +0000
 Andrew Godfrey via Digitalmars-d <digitalmars-d puremagic.com> 
 wrote:

 maybe just call that "slice views"? ;-)

 really, uncommon term will (at least it should ;-) make user to 
 read
 about that "slice views", and "reference" is something like 
 "ah, i know
 what references is, let's start coding now!"
Yeah, you have a point. "slice view" sounds odd, and I suppose all this is the background to why some parts of the docs use the term "slice" for this. Maybe I should have another look at that - maybe introducing "slice" first and the "slice operator" later, can avoid confusion sufficiently.
Aug 21 2014
parent ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Thu, 21 Aug 2014 07:23:34 +0000
Andrew Godfrey via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 "slice view" sounds odd
i agree, i just took the first words that came into my head. ;-)
 Maybe I should have another look at that - maybe introducing=20
 "slice" first and the "slice operator" later, can avoid confusion=20
 sufficiently.
sound reasonable for me.
Aug 21 2014
prev sibling parent ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Thu, 21 Aug 2014 06:53:32 +0300
ketmar via Digitalmars-d <digitalmars-d puremagic.com> wrote:

offtopic: damn it! i will read my messages before posting. i will read
my messages before posting. i will read my messages before posting. i
will... who i'm trying to cheat?! too bad that we can't edit messages
after posting.
Aug 20 2014
prev sibling parent ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sun, 17 Aug 2014 10:32:51 +0300
ketmar via Digitalmars-d <digitalmars-d puremagic.com> wrote:

sorry, i meant THIS code:
=20
   void foo (int[] a) {
     a ~=3D 42;
   }
   ...
   int[] arr;
   arr ~=3D 666;
   foo(arr);
   assert(arr.length =3D=3D 2 && arr[1] =3D=3D 42);
Aug 17 2014
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 11 Aug 2014 03:04:41 -0400, Andrew Godfrey <x y.com> wrote:

 Reminder: The PR is ready for review:

 https://github.com/D-Programming-Language/dlang.org/pull/623

 Jonathan has summarized his position in the commments.
 What do the rest of you think?
 H. S. Teoh, Jakob, Ali, Marc, Dominikus, Chris -
 your impression of whether this clears up the confusion would
 help round out the feedback.
Sorry to be chiming in late on this. As the author of the article, I stand by my terminology, even if it's not "official." In fact, when I was writing the article, I was struggling to describe how arrays worked until I stumbled into that description, and then it all clicked pretty well. This issue has come up before, and I can't remember what the result was, but I am not severely attached to the terminology if it hurts the documentation of D (In other words, I'm willing to allow a pull request to the article on dlang's site if it is what people want). As I recall, I was in favor of changing the official definition (though not the advocate of it), but most were not. I think the definition of dynamic array as most languages define it, and dynamic array as D defines it, are not exactly at odds with each other, but the differences are significant. What is needed is a nice term for "reference to a chunk of data." I thought slice fit that bill nicely, but I can see how it would be confusing. The fact that slices act sort of like dynamic arrays, even though they may not even point at arrays at all, is a very awkward definition. It would be like calling all C pointers arrays. -Steve
Sep 15 2014
parent reply "deadalnix" <deadalnix gmail.com> writes:
We already are suing slices. We are creating the confusion by
pretending there is 2 different concepts when there is only one.
Sep 15 2014
parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Tuesday, 16 September 2014 at 01:32:55 UTC, deadalnix wrote:
 We already are suing slices. We are creating the confusion by
 pretending there is 2 different concepts when there is only one.
IMO, the term "slice" is not necessarily different, but rather, a *refinement* of the "dynamic array" term. Without changing any formal definition, just use the term slice whenever you can, and 95% of the ambiguity goes away.
Sep 16 2014
prev sibling parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Tuesday, 29 July 2014 at 07:46:34 UTC, Andrew Godfrey wrote:
 On Sunday, 27 July 2014 at 05:51:46 UTC, Jakob Ovrum wrote:
 On Saturday, 26 July 2014 at 23:06:02 UTC, Andrew Godfrey 
 wrote:
 Thereafter can come sub-slice examples and so on.
 Does this make sense?
Yes, the reference documentation is pretty terrible with naming of various array concepts. IIRC, when this was discussed in the past, a majority seemed to be in favour of using "slice" and "dynamic array" for their respective concepts instead of the current situation, but I also remember there was some opposition (for reason I can't remember). A pull request updating the documentation to use slice/dynamic array might weed them out ;)
I gave this a try, and overall it looks like an improvement, but I think we need another name than "slice". The reason is that the slice operator is a distinct thing and interacts with the "slice" in strange ways. When I next get time I'll try updating it to use the term "array reference". That is: int[] a; // defines an array reference, a int[3] b; a = b[1..3]; // updates the array reference a to refer to a slice of b
You can look at it like this: There are no builtin dynamic arrays in D, there are only slices (windows on to arbitrary memory) and static arrays (a block of stack memory, treated a value type). What you choose your slice to be a window on is your own business. int[] a = new int[10]; //a window on to some new gc memory int[3] b; //a static array a = b[1 .. 3]; //a window on to part of of b a = (cast(int*)core.stdc.stdlib.malloc(10*int.sizeof))[0 .. 10]; //a window on to some new memory on the C heap.
Jul 29 2014
prev sibling parent reply "Gary Willoughby" <dev nomad.so> writes:
On Friday, 25 July 2014 at 15:07:55 UTC, Andrew Godfrey wrote:
 The documentation could help a bit more. I'm game to
 try to make a pull request for this, but I'm wondering
 if the library docs can actually point to the article?

 More feasibly, I'm thinking:
 Both the documentation for length() (http://dlang.org/arrays)
 and reserve() (http://dlang.org/phobos/std_array.html)
 should at least mention assumeSafeAppend.
 An example similar to what I posted may also be worthwhile.
Instead of using the old library link (http://dlang.org/phobos/) use http://dlang.org/library/ instead. This way you can go to http://dlang.org/library/std/array.html and in the discuss section at the bottom make comments. The D community is hoping those discussion sections start being used in the same way the PHP documentation is used.
Jul 25 2014
parent reply "Wyatt" <wyatt.epp gmail.com> writes:
On Friday, 25 July 2014 at 15:59:18 UTC, Gary Willoughby wrote:
 The D community is hoping those discussion sections start being
 used in the same way the PHP documentation is used.
We are? Please, no. Holding the PHP doc comments up as an example of a positive thing is alternately alternately hilarious and terrifying. -Wyatt
Jul 25 2014
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdan.org> writes:
"Wyatt" <wyatt.epp gmail.com> wrote:
 On Friday, 25 July 2014 at 15:59:18 UTC, Gary Willoughby wrote:
 
 The D community is hoping those discussion sections start being
 used in the same way the PHP documentation is used.
We are? Please, no. Holding the PHP doc comments up as an example of a positive thing is alternately alternately hilarious and terrifying. -Wyatt
Why? I don't have much context but a few people I talked to were positive about the community contribution to php docs. -- Andrei
Jul 25 2014
next sibling parent reply "Brad Anderson" <eco gnuk.net> writes:
On Friday, 25 July 2014 at 17:32:38 UTC, Andrei Alexandrescu 
wrote:
 "Wyatt" <wyatt.epp gmail.com> wrote:
 On Friday, 25 July 2014 at 15:59:18 UTC, Gary Willoughby wrote:
 
 The D community is hoping those discussion sections start 
 being
 used in the same way the PHP documentation is used.
We are? Please, no. Holding the PHP doc comments up as an example of a positive thing is alternately alternately hilarious and terrifying. -Wyatt
Why? I don't have much context but a few people I talked to were positive about the community contribution to php docs. -- Andrei
They are hit or miss. Sometimes you'll find exactly what you need and sometimes you'll find terrible misinformation. I suspect this our community will fair better than PHP has here but there is still that risk. I think it's worth a shot and if it becomes a problem the user comments can just be removed.
Jul 25 2014
parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Fri, Jul 25, 2014 at 05:37:24PM +0000, Brad Anderson via Digitalmars-d wrote:
 On Friday, 25 July 2014 at 17:32:38 UTC, Andrei Alexandrescu wrote:
"Wyatt" <wyatt.epp gmail.com> wrote:
On Friday, 25 July 2014 at 15:59:18 UTC, Gary Willoughby wrote:
The D community is hoping those discussion sections start being
used in the same way the PHP documentation is used.
We are? Please, no. Holding the PHP doc comments up as an example of a positive thing is alternately alternately hilarious and terrifying. -Wyatt
Why? I don't have much context but a few people I talked to were positive about the community contribution to php docs. -- Andrei
They are hit or miss. Sometimes you'll find exactly what you need and sometimes you'll find terrible misinformation. I suspect this our community will fair better than PHP has here but there is still that risk. I think it's worth a shot and if it becomes a problem the user comments can just be removed.
I'd suggest vetting user comments before posting them on the docs page, but that would mean requiring forumites to allocate yet more of their copious free time to the task. ;-) T -- Жил-был король когда-то, при нём блоха жила.
Jul 25 2014
prev sibling next sibling parent reply "Wyatt" <wyatt.epp gmail.com> writes:
On Friday, 25 July 2014 at 17:32:38 UTC, Andrei Alexandrescu 
wrote:
 "Wyatt" <wyatt.epp gmail.com> wrote:
 On Friday, 25 July 2014 at 15:59:18 UTC, Gary Willoughby wrote:
 
 The D community is hoping those discussion sections start 
 being
 used in the same way the PHP documentation is used.
We are? Please, no. Holding the PHP doc comments up as an example of a positive thing is alternately alternately hilarious and terrifying. -Wyatt
Why? I don't have much context but a few people I talked to were positive about the community contribution to php docs. -- Andrei
I've written about this before, but here's the summary: There will be a comment thread on every single function in the standard library where people will say whatever they want and we can't stop them ahead of time. - Anything they could possibly post there that's legitimately worth having would be better off canonised in the docs in the first place. - Any questions they ask would be better asked on D.learn. - Any other post may be subtly incorrect, previously correct, incredibly unsafe, correct with caveats, or a multitude of other shades of wrong. - Everything else is noise. We'd be better off wikifying the whole lot of it, and I don't say that lightly. At least then there's the possibility of it being clear what's changing and who is changing it. -Wyatt
Jul 25 2014
parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, 25 July 2014 at 19:08:07 UTC, Wyatt wrote:
 I've written about this before, but here's the summary:
 There will be a comment thread on every single function in the 
 standard library where people will say whatever they want and 
 we can't stop them ahead of time.

 - Anything they could possibly post there that's legitimately 
 worth having would be better off canonised in the docs in the 
 first place.
 - Any questions they ask would be better asked on D.learn.
 - Any other post may be subtly incorrect, previously correct, 
 incredibly unsafe, correct with caveats, or a multitude of 
 other shades of wrong.
 - Everything else is noise.
Agreed. I find the idea of having the official docs cluttered with comments from literally anyone who could say anything (correct or not) to be a terrible idea. - Jonathan M Davis
Jul 25 2014
prev sibling next sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Friday, 25 July 2014 at 17:32:38 UTC, Andrei Alexandrescu 
wrote:
 Why? I don't have much context but a few people I talked to 
 were positive
 about the community contribution to php docs. -- Andrei
You have not been reading latest discussions about it carefully then ;) (I also think it is bad idea)
Jul 25 2014
parent "Gary Willoughby" <dev nomad.so> writes:
On Saturday, 26 July 2014 at 01:48:11 UTC, Dicebot wrote:
 On Friday, 25 July 2014 at 17:32:38 UTC, Andrei Alexandrescu 
 wrote:
 Why? I don't have much context but a few people I talked to 
 were positive
 about the community contribution to php docs. -- Andrei
You have not been reading latest discussions about it carefully then ;) (I also think it is bad idea)
So it's pretty much a consensus that it's a bad idea. I kinda agree that *any* comment shouldn't be allowed. However there is great knowledge in the community that would be nice to capture somehow.
Jul 26 2014
prev sibling parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Friday, 25 July 2014 at 17:32:38 UTC, Andrei Alexandrescu 
wrote:
 Why? I don't have much context but a few people I talked to 
 were positive
 about the community contribution to php docs. -- Andrei
I find the php comments useful when I have to use the docs. It isn't about correctness, it is about gotchas and potential work-arounds. No worse than stack overflow. There is noise, but it makes no difference at the bottom of the page. People are also more likely to comment there than submitting a formal complaint about errors in the docs.
Jul 26 2014
parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Sunday, 27 July 2014 at 06:06:03 UTC, Ola Fosheim Grøstad 
wrote:
 On Friday, 25 July 2014 at 17:32:38 UTC, Andrei Alexandrescu 
 wrote:
 Why? I don't have much context but a few people I talked to 
 were positive
 about the community contribution to php docs. -- Andrei
I find the php comments useful when I have to use the docs. It isn't about correctness, it is about gotchas and potential work-arounds. No worse than stack overflow. There is noise, but it makes no difference at the bottom of the page. People are also more likely to comment there than submitting a formal complaint about errors in the docs.
Sure, it could be used as a lightweight bugtracker. But them, someone would have to watch the comments, and actually turn them into PRs, and remove the comments when the PRs are merged.
Jul 27 2014
parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Sunday, 27 July 2014 at 08:23:05 UTC, Marc Schütz wrote:
 Sure, it could be used as a lightweight bugtracker. But them, 
 someone would have to watch the comments, and actually turn 
 them into PRs, and remove the comments when the PRs are merged.
I also find example code useful sometimes when a function/feature is under-documented or just plain buggy. If people added the examples they provide in the forums to the doc pages then it would be easier to learn the language? If people added links to stackoverflow/articles when they find them, and put it in the correct section, then it would be easier to dig deeper into the knowledge base? You could have a "sign of authority" on usernames that are official "D experts" and let them "approve" correct comments. When you have received N approvals you can win a D T-shirt? :) A lot of noise is a sign of a documentation page that needs work or that the page need to be broken up into several pages.
Jul 27 2014