www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Passing dynamic arrays

reply Jens Mueller <jens.k.mueller gmx.de> writes:
Hi,

I do not understand what's going on behind the scene with this code. Or
better said I have some idea but maybe I do not see the whole point.

void foo(int[] array) {
	array.length += 1000; // may copy the array
	array[0] = 1;
}

auto a = new int[1];
foo(a);
assert(a[0] == 1); // fails if a needs to copied inside foo

I do understand that array.length += 1000 may copy the array. Page 98 of
The D Programming Language and
http://www.digitalmars.com/d/2.0/arrays.html#resize shed some light on
this matter. Passing a to foo is achieved by copying let's say a begin
and an end pointer. Now due to array.length += 1000 new memory might be
needed and that's why the begin and end pointer change and array[0]
works now on different data. That's why the assert fails. Right?

I find this behavior rather strange. Arrays are neither passed by value
(copying the whole array) nor by reference. I see reasons for doing it
like this, e.g. doing array = array[1..$] inside should not affect the
outside.
But I wonder whether these semantics are well enough documented?
I think I should use ref int[] in the example above, shouldn't I?

Jens
Nov 08 2010
next sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Mon, 08 Nov 2010 20:30:03 +0300, Jens Mueller <jens.k.mueller gmx.de>  
wrote:

 Hi,

 I do not understand what's going on behind the scene with this code. Or
 better said I have some idea but maybe I do not see the whole point.

 void foo(int[] array) {
 	array.length += 1000; // may copy the array
 	array[0] = 1;
 }

 auto a = new int[1];
 foo(a);
 assert(a[0] == 1); // fails if a needs to copied inside foo

 I do understand that array.length += 1000 may copy the array. Page 98 of
 The D Programming Language and
 http://www.digitalmars.com/d/2.0/arrays.html#resize shed some light on
 this matter. Passing a to foo is achieved by copying let's say a begin
 and an end pointer. Now due to array.length += 1000 new memory might be
 needed and that's why the begin and end pointer change and array[0]
 works now on different data. That's why the assert fails. Right?

 I find this behavior rather strange. Arrays are neither passed by value
 (copying the whole array) nor by reference. I see reasons for doing it
 like this, e.g. doing array = array[1..$] inside should not affect the
 outside.
 But I wonder whether these semantics are well enough documented?
 I think I should use ref int[] in the example above, shouldn't I?

 Jens
Yes, you understood it correctly. The changes to array structure (i.e. size and pointer to contents) aren't visible to outer scope, but changes to the contents are. int[] is merely a tuple of a T* ptr and size_t length.
Nov 08 2010
prev sibling next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Jens Mueller:

 I find this behavior rather strange.
I don't know if it's strange, but surely it is a little bug-prone corner of D. I have had two or three bugs in my code because of that.
 Arrays are neither passed by value
 (copying the whole array) nor by reference.
They are passed by "fat reference" :-)
 I think I should use ref int[] in the example above, shouldn't I?
Right. Bye, bearophile
Nov 08 2010
next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, November 08, 2010 09:40:47 bearophile wrote:
 Jens Mueller:
 I find this behavior rather strange.
I don't know if it's strange, but surely it is a little bug-prone corner of D. I have had two or three bugs in my code because of that.
I don't know. I find it to be pretty straightforward, though I can understand why people would find it to be confusing at first. As long as you don't alter a dynamic array's size in any way, then it and any references to it or any part of it will continue to point to the same data. If you dup an array, then it's guaranteed to point to different data (albeit a copy of that data). If you alter the size of an array but don't explicitly dup it, then it _might_ point to the same data and it might not (hence the potential confusion). So, if you want to guarantee that an array continues to point to the same data, then just don't alter its size. If you want to guarantee that its copied and points to different data, then dup or idup it. If you don't care whether it continues to point to the same data or not, then feel free to resize it through setting its length or appending to it. Granted, it's easier to understand what's going on when you understand that a dynamic array is essentially struct array(T) { T* ptr; size_t length; } but you don't really need to. Truth be told, I don't think that I've _ever_ had a bug due to how arrays reallocate. I think that as long as you understand that resizing could mean reallocation, it's quite easy to avoid bugs with it. That doesn't mean that they'll never happen, but I don't find this particular corner of the language to be particularly bug-prone. - Jonathan M Davis
Nov 08 2010
prev sibling parent reply Daniel Gibson <metalcaedes gmail.com> writes:
bearophile schrieb:
 Jens Mueller:
 
 I find this behavior rather strange.
I don't know if it's strange, but surely it is a little bug-prone corner of D. I have had two or three bugs in my code because of that.
If you pass a dynamic array to a function and chance it's size within the function, you have undefined behaviour - you never know if it will affect the original array (from the calling function) or not. So IMHO a compiler warning would be appropriate in that case. (It would be even better to have more consistent array handling throughout the different kinds of arrays, as I wrote in another branch of this thread, but if that is no option, for example because it contradicts TDPL, a compiler warning is a good compromise) Cheers, - Daniel
Nov 08 2010
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 08 Nov 2010 13:35:38 -0500, Daniel Gibson <metalcaedes gmail.com>  
wrote:

 bearophile schrieb:
 Jens Mueller:

 I find this behavior rather strange.
I don't know if it's strange, but surely it is a little bug-prone corner of D. I have had two or three bugs in my code because of that.
If you pass a dynamic array to a function and chance it's size within the function, you have undefined behaviour - you never know if it will affect the original array (from the calling function) or not.
Not exactly. If you happen to change its size *and* change the original data afterwards, then it's somewhat undefined (I'd call it confusing, since the behavior is perfectly defined, just hard to describe). Such cases are very rare. You are usually changing data on the array in place, or appending to the array, but not usually both.
 So IMHO a compiler warning would  be appropriate in that case.

 (It would be even better to have more consistent array handling  
 throughout the different kinds of arrays, as I wrote in another branch  
 of this thread, but if that is no option, for example because it  
 contradicts TDPL, a compiler warning is a good compromise)
First, D doesn't have compiler warnings. Either something is an error, or it is not. You can use the -w switch to turn on extra checks that become errors, but that's it. Second, if you made that a compiler warning, then 90% of D functions would exhibit a warning. This may appear to be a surprising issue, the one time it happens to you, but when it does, you learn how arrays work and move on. In practice, it's not a huge killer such as to warrant making it a compiler error or warning. -Steve
Nov 08 2010
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
Steven Schveighoffer wrote:
 On Mon, 08 Nov 2010 13:35:38 -0500, Daniel Gibson
 If you pass a dynamic array to a function and chance it's size within
 the function, you have undefined behaviour - you never know if it will
 affect the original array (from the calling function) or not.
Not exactly. If you happen to change its size *and* change the original data afterwards, then it's somewhat undefined
Let's also note that appending to the array qualifies as "change its size *and* change the original data afterwards." We cannot be sure whether appending affects the passed-in array.
 (I'd call it confusing,
 since the behavior is perfectly defined, just hard to describe).
I like the term "discretionary sharing semantics" where any slice can leave the sharing contract at their discretion regardless of whether they modified the shared elements so far. Ali
Nov 08 2010
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 08 Nov 2010 14:22:36 -0500, Ali Çehreli <acehreli yahoo.com> wrote:

 Steven Schveighoffer wrote:
  > On Mon, 08 Nov 2010 13:35:38 -0500, Daniel Gibson
  >> If you pass a dynamic array to a function and chance it's size within
  >> the function, you have undefined behaviour - you never know if it  
 will
  >> affect the original array (from the calling function) or not.
  >
  > Not exactly.  If you happen to change its size *and* change the  
 original
  > data afterwards, then it's somewhat undefined

 Let's also note that appending to the array qualifies as "change its  
 size *and* change the original data afterwards." We cannot be sure  
 whether appending affects the passed-in array.
No, it doesn't. If you are appending to data that was passed in, you are not changing the *original data* passed in. You are only appending to it. for example: char[] s = "foo".dup; s ~= "bar"; does not change the first 3 characters at all. So any aliases to s would not be affected. However, any data aliased to the original s may or may not be aliased to the new s. Once you start changing that original data (either via s or via an alias to the original s), this is where the confusing behavior occurs. In my experience, this does not cause a problem in the vast majority of cases. -Steve
Nov 08 2010
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
Steven Schveighoffer wrote:

 No, it doesn't. If you are appending to data that was passed in, you are
 not changing the *original data* passed in.  You are only appending 
to it. I must be remembering an old behavior. I think appending could affect the original if it had enough capacity. Ali
Nov 08 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 08 Nov 2010 18:29:27 -0500, Ali Çehreli <acehreli yahoo.com> wrote:

 Steven Schveighoffer wrote:

  > No, it doesn't. If you are appending to data that was passed in, you  
 are
  > not changing the *original data* passed in.  You are only appending  
 to it.

 I must be remembering an old behavior. I think appending could affect  
 the original if it had enough capacity.
Before the array append changes were introduced (somewhere around 2.040 I think?), appending to a slice that started at the beginning of the memory block could affect the other data in the array. But that was a memory corruption issue, somewhat different than what we are talking about. -Steve
Nov 09 2010
parent reply Pillsy <pillsbury gmail.com> writes:
Steven Schveighoffer Wrote:

 On Mon, 08 Nov 2010 18:29:27 -0500, Ali Çehreli <acehreli yahoo.com> wrote:
[...]
 I must be remembering an old behavior. I think appending could
 affect the original if it had enough capacity.
 Before the array append changes were introduced (somewhere around
 2.040 I think?), appending to a slice that started at the beginning of
 the memory block could affect the other data in the array.  But that
 was a memory corruption issue, somewhat different than what we 
 are talking about.
Ah! This is a lot of what was confusing me about arrays; I still thought they had this behavior. The fact that they don't makes me a good deal more comfortable with them, though I still don't like the non-deterministic way that they may copy their elements or they may share structure after you append stuff to them. Cheers, Pillsy
Nov 09 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 09 Nov 2010 08:14:40 -0500, Pillsy <pillsbury gmail.com> wrote:

 Steven Schveighoffer Wrote:

 On Mon, 08 Nov 2010 18:29:27 -0500, Ali Çehreli <acehreli yahoo.com>  
 wrote:
[...]
 I must be remembering an old behavior. I think appending could
 affect the original if it had enough capacity.
 Before the array append changes were introduced (somewhere around
 2.040 I think?), appending to a slice that started at the beginning of
 the memory block could affect the other data in the array.  But that
 was a memory corruption issue, somewhat different than what we
 are talking about.
Ah! This is a lot of what was confusing me about arrays; I still thought they had this behavior. The fact that they don't makes me a good deal more comfortable with them, though I still don't like the non-deterministic way that they may copy their elements or they may share structure after you append stuff to them.
As I said before, this rarely affects code. The common cases I've seen: 1. You append to an array and return it. 2. You modify data in the array. 3. You use a passed in array as a buffer, which means you overwrite the array, and then start appending when it runs out of space. I don't ever remember seeing: You append to an array, then go back and modify the first few bytes of the array. Let's assume this is a very common thing and absolutely needs to be addressed. What would you like the behavior to be? How can anything different than the current behavior be reasonable? IMO, the benefits of just being able to append to an array any time you want without having to set up some special type far outweighs this little quirk that almost nobody encounters. You can append to *any* array, no matter where the data is located, or whether the data is a slice, and it just works. I can't see how anyone would prefer another solution! -Steve
Nov 09 2010
parent reply Pillsy <pillsbury gmail.com> writes:
Steven Schveighoffer Wrote:

 On Tue, 09 Nov 2010 08:14:40 -0500, Pillsy <pillsbury gmail.com> wrote:
[...]
 Ah! This is a lot of what was confusing me about arrays; I still thought  
 they had this behavior. The fact that they don't makes me a good deal  
 more comfortable with them, though I still don't like the  
 non-deterministic way that they may copy their elements or they may  
 share structure after you append stuff to them.
 As I said before, this rarely affects code.  The common cases I've seen:
 1. You append to an array and return it.
 2. You modify data in the array.
 3. You use a passed in array as a buffer, which means you overwrite the  
 array, and then start appending when it runs out of space.
 I don't ever remember seeing:
 You append to an array, then go back and modify the first few bytes of the  
 array.
I've certainly encountered situations in at least one other language where standard library functions will return mutable arrays which may or may not share structure with their inputs. This has been such a frequent source of pain when using that language that I tend to react very negatively to the possibility in any context.
 Let's assume this is a very common thing and absolutely needs to be  
 addressed.  What would you like the behavior to be? 
Using a different, library type for a buffer you can append to. I think of "a buffer or abstract list you can cheaply append to" as a different sort of type from a fixed size buffer anyway, since it so often is a different type. Arrays/slices are a very basic type in D, and I'm generally thinking that giving your basic types simpler, easier to understand semantics is worth paying a modest cost. [...]
 IMO, the benefits of just being able to append to an array any time you  
 want without having to set up some special type far outweighs this little  
 quirk that almost nobody encounters.  You can append to *any* array, no  
 matter where the data is located, or whether the data is a slice, and it  
 just works.  I can't see how anyone would prefer another solution!
There's a difference between appending and appending in place. The problem with not appending in place (and arrays not having the possibility of a reserve that's larger than the actual amount, of course) is one of efficiency. Having auto s = "foo"; s ~= "bar"; result in a new array being allocated that is of length 6 and contains "foobar", and assigning that array to `s`, is obviously useful and desirable behavior. If the expansion can happen in place, that's a perfectly reasonable performance optimization to have in the case of strings or other immutable arrays. Indeed, one of the reasons that functional programming and GC go together like peanut butter and jelly is that together they let you get all sorts of wins in terms of efficiency from shared structure. However, I've found working with languages that mix a lot of imperative and functional constructs (Lisp is one, but not the only one) that if you're going to do this, it's really very important that there not be any doubt about when mutable state is shared and when it isn't. D is trying to be that same kind of multi-paradigm language. This means that, for mutable arrays, having int[] x = [1, 2, 3]; x ~= [4, 5, 6]; maybe reallocate and maybe not seems like it's only really there to protect people from doing inefficient things by accident when they append onto the back of an array repeatedly (or to make that admittedly common case more convenient). This really doesn't strike me as worth the trouble. Like I said elsewhere, the uncertainty gives me the screaming willies. Cheers, Pillsy
Nov 09 2010
next sibling parent spir <denis.spir gmail.com> writes:
On Tue, 09 Nov 2010 15:13:55 -0500
Pillsy <pillsbury gmail.com> wrote:

 There's a difference between appending and appending in place. The proble=
m with not appending in place (and arrays not having the possibility of a r= eserve that's larger than the actual amount, of course) is one of efficienc= y. Having
=20
 auto s =3D "foo";
 s ~=3D "bar";
=20
 result in a new array being allocated that is of length 6 and contains "f=
oobar", and assigning that array to `s`, is obviously useful and desirable = behavior. If the expansion can happen in place, that's a perfectly reasonab= le performance optimization to have in the case of strings or other immutab= le arrays. Indeed, one of the reasons that functional programming and GC go= together like peanut butter and jelly is that together they let you get al= l sorts of wins in terms of efficiency from shared structure.=20
=20
 However, I've found working with languages that mix a lot of imperative a=
nd functional constructs (Lisp is one, but not the only one) that if you're= going to do this, it's really very important that there not be any doubt a= bout when mutable state is shared and when it isn't. D is trying to be that= same kind of multi-paradigm language. +++
 This means that, for mutable arrays, having
=20
 int[] x =3D [1, 2, 3];
 x ~=3D [4, 5, 6];
=20
 maybe reallocate and maybe not seems like it's only really there to prote=
ct people from doing inefficient things by accident when they append onto t= he back of an array repeatedly (or to make that admittedly common case more= convenient). This really doesn't strike me as worth the trouble. Like I sa= id elsewhere, the uncertainty gives me the screaming willies.=20 There is some trouble in there; but it's hard to point it clearly. After int[] ints1; ... ints2 =3D ints1; ... depending of what happens to each array, especially when passed to funcs th= at could manipulate them, the relation initially established between variab= les may or may not be maintained. Also, it may be broken only in some cases= , depending on what operations are performed during a given run. Possibly, in practice, things are much easier to do right than seems at fir= st sight. But my impression is I will be bitten more than once, and badly. = (*) Denis (*) Reminds of python's famous gotcha which instead _establishes_ an unexpe= cted relation: def f(i, l=3D[]): l.append(i) return l print f(1) # [1] print f(1) # [1, 1] print f(1) # [1, 1, 1] -- -- -- -- -- -- -- vit esse estrany =E2=98=A3 spir.wikidot.com
Nov 09 2010
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 09 Nov 2010 15:13:55 -0500, Pillsy <pillsbury gmail.com> wrote:

 Steven Schveighoffer Wrote:

 On Tue, 09 Nov 2010 08:14:40 -0500, Pillsy <pillsbury gmail.com> wrote:
[...]
 Ah! This is a lot of what was confusing me about arrays; I still  
thought
 they had this behavior. The fact that they don't makes me a good deal
 more comfortable with them, though I still don't like the
 non-deterministic way that they may copy their elements or they may
 share structure after you append stuff to them.
 As I said before, this rarely affects code.  The common cases I've seen:
 1. You append to an array and return it.
 2. You modify data in the array.
 3. You use a passed in array as a buffer, which means you overwrite the
 array, and then start appending when it runs out of space.
 I don't ever remember seeing:
 You append to an array, then go back and modify the first few bytes of  
 the
 array.
I've certainly encountered situations in at least one other language where standard library functions will return mutable arrays which may or may not share structure with their inputs. This has been such a frequent source of pain when using that language that I tend to react very negatively to the possibility in any context.
Care to name names? I want to understand this dislike of D arrays, because out of all the languages I've ever used, D arrays are by far the easiest and most intuitive to use. I don't expect to be convinced, but at least we can have some debate on this, and maybe we can avoid mistakes made by other languages.
 Let's assume this is a very common thing and absolutely needs to be
 addressed.  What would you like the behavior to be?
Using a different, library type for a buffer you can append to. I think of "a buffer or abstract list you can cheaply append to" as a different sort of type from a fixed size buffer anyway, since it so often is a different type. Arrays/slices are a very basic type in D, and I'm generally thinking that giving your basic types simpler, easier to understand semantics is worth paying a modest cost.
There was a time when the T[new] idea was expected to be part of the language. Both Andrei and Walter were behind it, and seldom does something not make it into the language when that happens. It turns out, that after all the academic and theoretical discussions were finished, and it came time to implement, it was a clunky and confusing feature. Andrei said that for TDPL he had a whole table dedicated to what type to use in which cases (T[] or T[new]) and he didn't even know how to fill out the table. The beauty of D's arrays are that the slice and the array are both the same type, so you only need to define one function to handle both, and appending "just works". I feel like this is simply a case of 'not well enough understood.' BTW, you can allocate a fixed buffer by doing: T[BUFSIZE] buffer; This cannot be appended to. It is still difficult to allocate one of these on the heap, which is a language shortcoming, but it can be fixed.
 [...]
 IMO, the benefits of just being able to append to an array any time you
 want without having to set up some special type far outweighs this  
 little
 quirk that almost nobody encounters.  You can append to *any* array, no
 matter where the data is located, or whether the data is a slice, and it
 just works.  I can't see how anyone would prefer another solution!
There's a difference between appending and appending in place. The problem with not appending in place (and arrays not having the possibility of a reserve that's larger than the actual amount, of course) is one of efficiency. Having auto s = "foo"; s ~= "bar"; result in a new array being allocated that is of length 6 and contains "foobar", and assigning that array to `s`, is obviously useful and desirable behavior. If the expansion can happen in place, that's a perfectly reasonable performance optimization to have in the case of strings or other immutable arrays. Indeed, one of the reasons that functional programming and GC go together like peanut butter and jelly is that together they let you get all sorts of wins in terms of efficiency from shared structure. However, I've found working with languages that mix a lot of imperative and functional constructs (Lisp is one, but not the only one) that if you're going to do this, it's really very important that there not be any doubt about when mutable state is shared and when it isn't. D is trying to be that same kind of multi-paradigm language. This means that, for mutable arrays, having int[] x = [1, 2, 3]; x ~= [4, 5, 6];
To leave no doubt about whether this reallocates or not try: bool willReallocate = x.length + 3 > x.capacity; But I still don't understand this concept. If you find out it's not going to reallocate, what are you going to do? I mean, you have three cases here: 1. You *don't* want it to reallocate -- well, you can't enforce this, but you can use ref to ensure the original is always affected 2. You *want* it to reallocate -- use dup or ~ 3. You don't care -- just use the array directly I don't see how these three options aren't enough.
 maybe reallocate and maybe not seems like it's only really there to  
 protect people from doing inefficient things by accident when they  
 append onto the back of an array repeatedly (or to make that admittedly  
 common case more convenient). This really doesn't strike me as worth the  
 trouble. Like I said elsewhere, the uncertainty gives me the screaming  
 willies.
I hear you, but at the same time, we are talking about common and uncommon cases here. D (at least in my mind) tries to be a practical language -- make the common things easy as long as they are safe. And the cases where D's arrays may surprise you are pretty uncommon IMO. -Steve
Nov 10 2010
prev sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, November 08, 2010 11:22:36 Ali =C3=87ehreli wrote:
 Steven Schveighoffer wrote:
  > On Mon, 08 Nov 2010 13:35:38 -0500, Daniel Gibson
  >=20
  >> If you pass a dynamic array to a function and chance it's size within
  >> the function, you have undefined behaviour - you never know if it will
  >> affect the original array (from the calling function) or not.
  >=20
  > Not exactly.  If you happen to change its size *and* change the origin=
al
  > data afterwards, then it's somewhat undefined
=20
 Let's also note that appending to the array qualifies as "change its
 size *and* change the original data afterwards." We cannot be sure
 whether appending affects the passed-in array.
Yes you can. It _never_ alters the array which was passed in. Sure, it _cou= ld_=20 alter the memory just off the end of the passed in array if no arrays refer= to=20 that memory, but that doesn't cause any problems. It doesn't alter the orig= inal=20 array at all. It just means that when you resize that array, it may have to= =20 reallocate whereas before it might have been able to resize in place. And i= f the=20 array in the called function reallocates instead of resizing in place, then= the=20 original array would either have been forced to reallocate anyway or it may= be=20 able to resize in place depending on how much you try and resize it and whe= ther=20 any other arrays refer to the memory passed its end. In _no_ case does appending or concatenating to an array alter any other ar= rays,=20 even if they point to the same memory. They may or may not end up pointing = to=20 the same memory afterwards (depending on whether a reallocation takes place= ),=20 but you never alter any other arrays. =2D Jonathan M Davis
Nov 08 2010
prev sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, November 08, 2010 10:35:38 Daniel Gibson wrote:
 bearophile schrieb:
 Jens Mueller:
 I find this behavior rather strange.
I don't know if it's strange, but surely it is a little bug-prone corner of D. I have had two or three bugs in my code because of that.
If you pass a dynamic array to a function and chance it's size within the function, you have undefined behaviour - you never know if it will affect the original array (from the calling function) or not.
Implementation-defined behavior would be more accurate. Undefined would mean that it's something dangerous which is not defined by the language, and you shouldn't be doing. It's perfectly defined by the language in this case. It's just that how much extra memory is allocated for a dynamic array is implementation-defined, so in cases where a reallocation would be necessary because the array ran out of memory, it's implementation-dependent as to whether or not a reallocation will be necessary or not. In every other case, it's completely language-defined and deterministic. The algorithm for additional capacity is the only part that isn't.
 So IMHO a compiler warning would  be appropriate in that case.
 
 (It would be even better to have more consistent array handling throughout
 the different kinds of arrays, as I wrote in another branch of this
 thread, but if that is no option, for example because it contradicts TDPL,
 a compiler warning is a good compromise)
Honestly, if you want an array that you're passing in to be altered by the function that you're passing it to, it really should be passed by ref anyway. And if you want to guarantee that it isn't going to be altered, make it const, dup it, or have its individual elements be const or immutable. Problem solved. I really don't see this as an issue. I can understand why there might be some confusion - particularly since the online docs aren't very clear on the matter, but it really isn't complicated, and I'd hate to have to dup an array just to be able to append to it, which is what the warning that you're suggesting would require. A compiler warning is as good as an error really, since you're going to have to fix it anyway, so I'd definitely be against making this either an error or a warning. I see no problem with being allowed to resize arrays that are passed to functions. - Jonathan M Davis
Nov 08 2010
parent reply Daniel Gibson <metalcaedes gmail.com> writes:
Jonathan M Davis schrieb:
 On Monday, November 08, 2010 10:35:38 Daniel Gibson wrote:
 bearophile schrieb:
 Jens Mueller:
 I find this behavior rather strange.
I don't know if it's strange, but surely it is a little bug-prone corner of D. I have had two or three bugs in my code because of that.
If you pass a dynamic array to a function and chance it's size within the function, you have undefined behaviour - you never know if it will affect the original array (from the calling function) or not.
Implementation-defined behavior would be more accurate. Undefined would mean that it's something dangerous which is not defined by the language, and you shouldn't be doing. It's perfectly defined by the language in this case. It's just that how much extra memory is allocated for a dynamic array is implementation-defined, so in cases where a reallocation would be necessary because the array ran out of memory, it's implementation-dependent as to whether or not a reallocation will be necessary or not. In every other case, it's completely language-defined and deterministic. The algorithm for additional capacity is the only part that isn't.
Ok, undefined may have been the wrong word.. non-deterministic may be better. Anyway, you can't know if there's space left behind the array that can be obtained by realloc or if increasing the length will cause the array to be copied to a new block of memory, so the array in the calling function points to other memory than the array in the called function.
 So IMHO a compiler warning would  be appropriate in that case.

 (It would be even better to have more consistent array handling throughout
 the different kinds of arrays, as I wrote in another branch of this
 thread, but if that is no option, for example because it contradicts TDPL,
 a compiler warning is a good compromise)
Honestly, if you want an array that you're passing in to be altered by the function that you're passing it to, it really should be passed by ref anyway.
The documentation[1] says: "For dynamic array and object parameters, which are passed by reference, in/out/ref apply only to the reference and not the contents." So, by reading the documentation one would assume that dynamic arrays are passed by reference - *real* reference, implying that any change to the array within the called function will be visible outside of the function. The truth however is that dynamic arrays are not passed by reference and any changes to the length will be lost (even if the arrays data won't be copied).
 And if you want to guarantee that it isn't going to be altered, make it const, 
 dup it, or have its individual elements be const or immutable. Problem solved.
I 
 really don't see this as an issue. I can understand why there might be some 
 confusion - particularly since the online docs aren't very clear on the
matter, 
 but it really isn't complicated, and I'd hate to have to dup an array just to
be 
 able to append to it, which is what the warning that you're suggesting would 
 require. A compiler warning is as good as an error really, since you're going
to 
 have to fix it anyway, so I'd definitely be against making this either an
error or 
 a warning. I see no problem with being allowed to resize arrays that are
passed 
 to functions.
So maybe yet another solution would be to *really* pass dynamic arrays by reference (like the doc pretends it's already done)?
 
 - Jonathan M Davis
Cheers, - Daniel [1] http://www.digitalmars.com/d/2.0/function.html
Nov 08 2010
next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, November 08, 2010 11:30:33 Daniel Gibson wrote:
 So IMHO a compiler warning would  be appropriate in that case.
 
 (It would be even better to have more consistent array handling
 throughout the different kinds of arrays, as I wrote in another branch
 of this thread, but if that is no option, for example because it
 contradicts TDPL, a compiler warning is a good compromise)
Honestly, if you want an array that you're passing in to be altered by the function that you're passing it to, it really should be passed by ref anyway.
The documentation[1] says: "For dynamic array and object parameters, which are passed by reference, in/out/ref apply only to the reference and not the contents." So, by reading the documentation one would assume that dynamic arrays are passed by reference - *real* reference, implying that any change to the array within the called function will be visible outside of the function. The truth however is that dynamic arrays are not passed by reference and any changes to the length will be lost (even if the arrays data won't be copied).
Then the documentation should be updated to be more clear.
 And if you want to guarantee that it isn't going to be altered, make it
 const, dup it, or have its individual elements be const or immutable.
 Problem solved. I really don't see this as an issue. I can understand
 why there might be some confusion - particularly since the online docs
 aren't very clear on the matter, but it really isn't complicated, and
 I'd hate to have to dup an array just to be able to append to it, which
 is what the warning that you're suggesting would require. A compiler
 warning is as good as an error really, since you're going to have to fix
 it anyway, so I'd definitely be against making this either an error or a
 warning. I see no problem with being allowed to resize arrays that are
 passed to functions.
So maybe yet another solution would be to *really* pass dynamic arrays by reference (like the doc pretends it's already done)?
That would be a devastating change. It would cause bugs all over the place - especially when dealing with std.algorithm. Remember that an array is a range. Really, when an array is passed to a function, you're passing a slice of that array which happen to slice the whole array. - Jonathan M Davis
Nov 08 2010
parent bearophile <bearophileHUGS lycos.com> writes:
Jonathan M Davis:

 Daniel Gibson:
 So maybe yet another solution would be to *really* pass dynamic arrays by
 reference (like the doc pretends it's already done)?
That would be a devastating change.
That's a solution that I have proposed lot of time ago, I did receive no answers :-) A disadvantage of passing on default dynamic arrays by reference is a decrease in performance. Bye, bearophile
Nov 08 2010
prev sibling next sibling parent spir <denis.spir gmail.com> writes:
On Mon, 08 Nov 2010 20:30:33 +0100
Daniel Gibson <metalcaedes gmail.com> wrote:

 The documentation[1] says: "For dynamic array and object parameters, whic=
h are=20
 passed by reference, in/out/ref apply only to the reference and not the c=
ontents."
 So, by reading the documentation one would assume that dynamic arrays are=
passed=20
 by reference - *real* reference, implying that any change to the array wi=
thin=20
 the called function will be visible outside of the function.
 The truth however is that dynamic arrays are not passed by reference and =
any=20
 changes to the length will be lost (even if the arrays data won't be copi=
ed). Exactly. Pass (and assign) by reference mean different semantics from what = D does with dyn arrays: that changes are shared. Denis -- -- -- -- -- -- -- vit esse estrany =E2=98=A3 spir.wikidot.com
Nov 08 2010
prev sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday 08 November 2010 12:07:39 spir wrote:
 On Mon, 08 Nov 2010 20:30:33 +0100
 
 Daniel Gibson <metalcaedes gmail.com> wrote:
 The documentation[1] says: "For dynamic array and object parameters,
 which are passed by reference, in/out/ref apply only to the reference
 and not the contents." So, by reading the documentation one would assume
 that dynamic arrays are passed by reference - *real* reference, implying
 that any change to the array within the called function will be visible
 outside of the function.
 The truth however is that dynamic arrays are not passed by reference and
 any changes to the length will be lost (even if the arrays data won't be
 copied).
Exactly. Pass (and assign) by reference mean different semantics from what D does with dyn arrays: that changes are shared.
The reference semantics for Objects and dynamic arrays are identical. They are passed by reference, but their reference is passed by value. So, any changes to the contents will affect the object or array which was passed in. However, changes to the reference (such as assigning a new Object or array or doing any operation on an array which would result in reallocation) do not change the object or array which was referred to by that reference. "For dynamic array and object parameters, which are passed by reference, in/out/ref apply only to the reference and not the contents." is perfectly correct. The problem is whether you read the "which..." as applying to both dynamic arrays and object parameters or just object parameters. It's perfectly correct as is but apparently ambiguous enough to cause confusion. The docs should be updated to be clearer, but the quoted documentation, at least, is correct. And there's nothing funny about how arrays work in comparison to objects except that arrays happen to have operations which can cause reallocation (in addition to new) whereas objects don't. - Jonathan M Davis
Nov 08 2010
prev sibling next sibling parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Mon, 08 Nov 2010 19:30:03 +0200, Jens Mueller <jens.k.mueller gmx.de>  
wrote:

 I find this behavior rather strange. Arrays are neither passed by value
 (copying the whole array) nor by reference. I see reasons for doing it
 like this, e.g. doing array = array[1..$] inside should not affect the
 outside.
Compare to this C-ish program: void foo(int *array, int length) { array = (int*)realloc(array, (length += 1000) * sizeof(int)); // may copy the array array[0] = 1; } int* a = new int[1]; foo(a, 1); assert(a[0] == 1); // fails if a needs to copied inside foo C's realloc *may* copy the memory area being reallocated. Just like when using realloc, it's something you must be aware of when using D arrays. -- Best regards, Vladimir mailto:vladimir thecybershadow.net
Nov 08 2010
parent reply Daniel Gibson <metalcaedes gmail.com> writes:
Vladimir Panteleev schrieb:
 On Mon, 08 Nov 2010 19:30:03 +0200, Jens Mueller <jens.k.mueller gmx.de> 
 wrote:
 
 I find this behavior rather strange. Arrays are neither passed by value
 (copying the whole array) nor by reference. I see reasons for doing it
 like this, e.g. doing array = array[1..$] inside should not affect the
 outside.
Compare to this C-ish program: void foo(int *array, int length) { array = (int*)realloc(array, (length += 1000) * sizeof(int)); // may copy the array array[0] = 1; } int* a = new int[1]; foo(a, 1); assert(a[0] == 1); // fails if a needs to copied inside foo C's realloc *may* copy the memory area being reallocated. Just like when using realloc, it's something you must be aware of when using D arrays.
This might technically be the same thing, but D's nice syntax hides this. IMHO passing arrays to functions are really inconsistent in D2 anyway: static arrays are passed by value but dynamic arrays are passed by reference, but then again, as this thread shows, not really.. And what about associative arrays? (I don't know, haven't tried yet, afaik it isn't documented). Certainly D's behaviour regarding the dynamic arrays has technical reasons and makes sense when you know how they're implemented (a fat pointer that is passed by value), but it *feels* inconsistent. IMHO either all kinds of arrays should *either* be passed by reference (real reference, not coincidental reference like dynamic arrays are now) *or* by "logical" value, i.e. dynamic arrays would be dup'ed and associative arrays would be cloned. BTW: What were the reasons to pass static arrays by value in D2 (while in D1 they're passed by reference)? Cheers, - Daniel
Nov 08 2010
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
Daniel Gibson wrote:
 BTW: What were the reasons to pass static arrays by value in D2 (while 
 in D1 they're passed by reference)?
It makes things like vectors (i.e. float[3]) natural to manipulate. It also segues nicely into hopeful future support for the CPU's vector instructions.
Nov 08 2010
parent reply Daniel Gibson <metalcaedes gmail.com> writes:
Walter Bright schrieb:
 Daniel Gibson wrote:
 BTW: What were the reasons to pass static arrays by value in D2 (while 
 in D1 they're passed by reference)?
It makes things like vectors (i.e. float[3]) natural to manipulate. It also segues nicely into hopeful future support for the CPU's vector instructions.
Why can't that be done when the static arrays are passed by reference?
Nov 08 2010
parent reply Daniel Gibson <metalcaedes gmail.com> writes:
Daniel Gibson schrieb:
 Walter Bright schrieb:
 Daniel Gibson wrote:
 BTW: What were the reasons to pass static arrays by value in D2 
 (while in D1 they're passed by reference)?
It makes things like vectors (i.e. float[3]) natural to manipulate. It also segues nicely into hopeful future support for the CPU's vector instructions.
Why can't that be done when the static arrays are passed by reference?
Ah I guess you mean something like "alias float[3] vec3", so one may expect vec3 to behave like a value type (like when you define it in a struct) so it's passed by value. That does make sense, even though I'm not sure what's more important: consistency between different kinds of arrays or expectations towards typed defined from static arrays. ;-) I still don't get the part with the CPU's vector instructions though. I don't have any assembly knowledge an no experience with directly using CPU's vector instructions, but the example of a C function wrapping SSE instructions from the wikipedia article[1], which multiplies two arrays of floats, loads the arrays by reference and even stores the result in one of them.
Nov 08 2010
parent reply Daniel Gibson <metalcaedes gmail.com> writes:
Daniel Gibson schrieb:
 Daniel Gibson schrieb:
 Walter Bright schrieb:
 Daniel Gibson wrote:
 BTW: What were the reasons to pass static arrays by value in D2 
 (while in D1 they're passed by reference)?
It makes things like vectors (i.e. float[3]) natural to manipulate. It also segues nicely into hopeful future support for the CPU's vector instructions.
Why can't that be done when the static arrays are passed by reference?
Ah I guess you mean something like "alias float[3] vec3", so one may expect vec3 to behave like a value type (like when you define it in a struct) so it's passed by value. That does make sense, even though I'm not sure what's more important: consistency between different kinds of arrays or expectations towards typed defined from static arrays. ;-) I still don't get the part with the CPU's vector instructions though. I don't have any assembly knowledge an no experience with directly using CPU's vector instructions, but the example of a C function wrapping SSE instructions from the wikipedia article[1], which multiplies two arrays of floats, loads the arrays by reference and even stores the result in one of them.
Hrm forgot the link: [1] http://en.wikipedia.org/wiki/Vector_processor
Nov 08 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
Daniel Gibson wrote:
 Daniel Gibson schrieb:
 Daniel Gibson schrieb:
 Walter Bright schrieb:
 Daniel Gibson wrote:
 BTW: What were the reasons to pass static arrays by value in D2 
 (while in D1 they're passed by reference)?
It makes things like vectors (i.e. float[3]) natural to manipulate. It also segues nicely into hopeful future support for the CPU's vector instructions.
Why can't that be done when the static arrays are passed by reference?
Ah I guess you mean something like "alias float[3] vec3", so one may expect vec3 to behave like a value type (like when you define it in a struct) so it's passed by value. That does make sense, even though I'm not sure what's more important: consistency between different kinds of arrays or expectations towards typed defined from static arrays. ;-) I still don't get the part with the CPU's vector instructions though. I don't have any assembly knowledge an no experience with directly using CPU's vector instructions, but the example of a C function wrapping SSE instructions from the wikipedia article[1], which multiplies two arrays of floats, loads the arrays by reference and even stores the result in one of them.
Hrm forgot the link: [1] http://en.wikipedia.org/wiki/Vector_processor
You don't write an add function by using references to ints. The vector instructions treat them like values, so a value type should correspond to it.
Nov 08 2010
parent Daniel Gibson <metalcaedes gmail.com> writes:
Walter Bright schrieb:
 Daniel Gibson wrote:
 Daniel Gibson schrieb:
 Daniel Gibson schrieb:
 Walter Bright schrieb:
 Daniel Gibson wrote:
 BTW: What were the reasons to pass static arrays by value in D2 
 (while in D1 they're passed by reference)?
It makes things like vectors (i.e. float[3]) natural to manipulate. It also segues nicely into hopeful future support for the CPU's vector instructions.
Why can't that be done when the static arrays are passed by reference?
Ah I guess you mean something like "alias float[3] vec3", so one may expect vec3 to behave like a value type (like when you define it in a struct) so it's passed by value. That does make sense, even though I'm not sure what's more important: consistency between different kinds of arrays or expectations towards typed defined from static arrays. ;-) I still don't get the part with the CPU's vector instructions though. I don't have any assembly knowledge an no experience with directly using CPU's vector instructions, but the example of a C function wrapping SSE instructions from the wikipedia article[1], which multiplies two arrays of floats, loads the arrays by reference and even stores the result in one of them.
Hrm forgot the link: [1] http://en.wikipedia.org/wiki/Vector_processor
You don't write an add function by using references to ints. The vector instructions treat them like values, so a value type should correspond to it.
Ok, thanks for explaining :-)
Nov 08 2010
prev sibling parent spir <denis.spir gmail.com> writes:
On Mon, 08 Nov 2010 19:04:40 +0100
Daniel Gibson <metalcaedes gmail.com> wrote:

 IMHO passing arrays to functions are really inconsistent in D2 anyway:
 static arrays are passed by value but dynamic arrays are passed by refere=
nce,
 but then again, as this thread shows, not really..
It may be better to have 2 kinds of "sequences", one having true value sema= ntics (assignment & parameter passing also protect the content), the other = true reference semantics (say, an array-list). Static arrays may just be co= nsidered as an additional hint to the compiler for possible optimization. Or, have arrays implement true value semantics, but pass them as ref when n= eeded. But then, we may sometimes need assignment not to copy... What do you think? Denis -- -- -- -- -- -- -- vit esse estrany =E2=98=A3 spir.wikidot.com
Nov 08 2010
prev sibling next sibling parent reply Kagamin <spam here.lot> writes:
Jens Mueller Wrote:

 I find this behavior rather strange. Arrays are neither passed by value
 (copying the whole array) nor by reference. I see reasons for doing it
 like this, e.g. doing array = array[1..$] inside should not affect the
 outside.
 But I wonder whether these semantics are well enough documented?
It may take some effort to explain it. Does it help, if you think about T[] as not an array but a slice?
Nov 08 2010
parent Jens Mueller <jens.k.mueller gmx.de> writes:
Kagamin wrote:
 Jens Mueller Wrote:
 
 I find this behavior rather strange. Arrays are neither passed by value
 (copying the whole array) nor by reference. I see reasons for doing it
 like this, e.g. doing array = array[1..$] inside should not affect the
 outside.
 But I wonder whether these semantics are well enough documented?
It may take some effort to explain it. Does it help, if you think about T[] as not an array but a slice?
Yes. This behavior is well explained when one thinks about passing a slice of data instead of an dynamic array. With a dynamic array I think about the std::vector. But passing a reference to a std::vector is not the same as passing a slice. Jens
Nov 08 2010
prev sibling next sibling parent reply Jesse Phillips <jessekphillips+D gmail.com> writes:
Jens Mueller Wrote:

 Hi,
 
 I do not understand what's going on behind the scene with this code. Or
 better said I have some idea but maybe I do not see the whole point.
 
 void foo(int[] array) {
 	array.length += 1000; // may copy the array
 	array[0] = 1;
 }
 
 auto a = new int[1];
 foo(a);
 assert(a[0] == 1); // fails if a needs to copied inside foo
 
...
 I find this behavior rather strange. Arrays are neither passed by value
 (copying the whole array) nor by reference. I see reasons for doing it
 like this, e.g. doing array = array[1..$] inside should not affect the
 outside.
 But I wonder whether these semantics are well enough documented?
 I think I should use ref int[] in the example above, shouldn't I?
 
 Jens
But they are past by reference. You can modify the data all you want, but cannot reassign the reference itself. Resizing the array may cause a reassignment of that reference. It is not different from the following code except resizing does not guarantee a reference change. import std.stdio; void assignValue(A a) { a = new A(); a.value = 6; } class A { int value; } void main() { A a = new A(); assignValue(a); writeln(a.value); }
Nov 08 2010
next sibling parent reply spir <denis.spir gmail.com> writes:
On Mon, 08 Nov 2010 15:32:56 -0500
Jesse Phillips <jessekphillips+D gmail.com> wrote:

 But they are past by reference. You can modify the data all you want, but=
cannot reassign the reference itself. No, they are _not_ passed by reference. Stop saying that, this is precisely= what causes confusion. This is reference semantics: class C { int* ints; } void main () { auto c =3D new C(); auto d =3D c; auto ints =3D [1,2,3]; c.ints =3D &(ints[0]); assert(c.ints =3D=3D d.ints); assert(*(c.ints) =3D=3D *(d.ints)); } Whatever change one performs on object fields never breaks the relation wit= h other vars pointing to the same object; because the object itself is refe= renced. D arrays do not work that way: the kind of struct is *copied*, not referenc= ed; arrays themselves are *values*. So that changes to the content that req= uires reallocation breaks the relation. (Additional confusion is brought by the fact that, if a is an array, after = "b=3Da" (b is a) yields true, for any reason, but this is also wrong. I gue= ss 'is' is overloaded for arrays to compare the adresses of the contents in= stead of the ones of the array-structs, but this is misguided.) Denis -- -- -- -- -- -- -- vit esse estrany =E2=98=A3 spir.wikidot.com
Nov 08 2010
parent reply Jesse Phillips <jessekphillips+D gmail.com> writes:
spir Wrote:

 No, they are _not_ passed by reference. Stop saying that, this is precisely
what causes confusion. This is reference semantics:
void main() { auto a = new int[1]; auto b = a; a[0] = 5; assert(a == b); assert(a[0] == b[0]); assert(a is b); writefln("%x == %x", &a[0], &b[0]); moreExample(a, cast(uint) &a[0]); } void moreExample(int[] a, uint aAddr) { assert(cast(uint)&a[0] == aAddr); writefln("%x == %x", &a[0], aAddr); }
 D arrays do not work that way: the kind of struct is *copied*, not referenced;
arrays themselves are *values*. So that changes to the content that requires
reallocation breaks the relation.
The struct you are referring to _is_ the reference. There is no such thing as "copy by reference," everything is copied by value. However, some values are just a reference to another location.
 (Additional confusion is brought by the fact that, if a is an array, after
"b=a" (b is a) yields true, for any reason, but this is also wrong. I guess
'is' is overloaded for arrays to compare the adresses of the contents instead
of the ones of the array-structs, but this is misguided.)
The array-struct is the reference, so it is what gets compared. That means both the internal pointer and length must be the same. Just because the reference is more than an address does not make it any less a reference. void main() { auto a = new int[2]; a[0] = 5; a[1] = 10; auto b = a.dup; auto c = a[0..1]; assert(a == b); assert(a[0] == b[0]); assert(a !is b); assert(a !is c); } The distinction is that an Array can have its reference changed by resizing (which is not an option for other reference types).
Nov 08 2010
next sibling parent reply Daniel Gibson <metalcaedes gmail.com> writes:
On Tue, Nov 9, 2010 at 1:24 AM, Jesse Phillips
<jessekphillips+D gmail.com> wrote:
 The array-struct is the reference, so it is what gets compared. That means
both the internal pointer and length must be the same. Just because the
reference is more than an address does not make it any less a reference.
Unlike in C, a D array is more than a reference. An array is the data (or a reference to its data) + metadata (its length) - the metadata belongs to the array (and not to the reference). This means that, when you say "the array is passed by reference" one expects that also the arrays length is passed by reference - because the length belongs to the array.
 The distinction is that an Array can have its reference changed by resizing
(which is not an option for other reference types).
If that is not an option for any other reference type, why should it be an option for arrays? That is just inconsistent and doesn't make any sense. No, Arrays should not be considered reference types when passed to a function. As someone else said before: Logically you don't pass the array but a slice that contains the whole array. Cheers, - Daniel
Nov 08 2010
parent reply Jesse Phillips <jessekphillips+D gmail.com> writes:
Daniel Gibson Wrote:

 On Tue, Nov 9, 2010 at 1:24 AM, Jesse Phillips
 <jessekphillips+D gmail.com> wrote:
 The array-struct is the reference, so it is what gets compared. That means
both the internal pointer and length must be the same. Just because the
reference is more than an address does not make it any less a reference.
Unlike in C, a D array is more than a reference. An array is the data (or a reference to its data) + metadata (its length) - the metadata belongs to the array (and not to the reference).
If you are using the definition of reference which would include C pointers (i.e. not C++ references), then there is nothing in the definition of reference that precludes it from having meta data. "In distributed computing, the reference may contain more than an address or identifier; it may also include an embedded specification of the network protocols used to locate and access the referenced object, the way information is encoded or serialized." http://en.wikipedia.org/wiki/Reference_(computer_science)
 This means that, when you say "the array is passed by reference" one
 expects that also the arrays length is passed by reference - because
 the length belongs to the array.
This is a good point. Though if the length did stay with the array, it is still reasonable that a reallocation of the array would cause the reference to no longer point to the same array.
 The distinction is that an Array can have its reference changed by resizing
(which is not an option for other reference types).
If that is not an option for any other reference type, why should it be an option for arrays? That is just inconsistent and doesn't make any sense.
Well, if you can come up with a good definition for what "increasing the size of a class" would do, then maybe it should be added. It really doesn't matter. Arrays are their own type, the have their own semantics. It does help to think of them in terms of slices (which is what TDPL refers to them as), yet that does not remove the fact that they are in dead a reference type. Many times familiar terms are used so that a behavior is quickly understood. For example it is common to say that arrays in C is just a pointer into a memory location. But in reality that is not true. http://www.lysator.liu.se/c/c-faq/c-2.html
 
 No, Arrays should not be considered reference types when passed to a function.
 As someone else said before: Logically you don't pass the array but a
 slice that contains the whole array.
 
 Cheers,
 - Daniel
Nov 08 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 08 Nov 2010 21:29:42 -0500, Jesse Phillips  
<jessekphillips+D gmail.com> wrote:

 Well, if you can come up with a good definition for what "increasing the  
 size of a class" would do, then maybe it should be added.

 It really doesn't matter. Arrays are their own type, the have their own  
 semantics. It does help to think of them in terms of slices (which is  
 what TDPL refers to them as), yet that does not remove the fact that  
 they are in dead a reference type.

 Many times familiar terms are used so that a behavior is quickly  
 understood. For example it is common to say that arrays in C is just a  
 pointer into a memory location. But in reality that is not true.
It depends on your definition of reference type. I agree with Daniel. If you want to get into academic definitions, yes, 'technically' an array is a reference, but it's confusing to someone who's not interested in exploring the theoretical parts of computer science. I think of an array as a hybrid between a reference and a value type. The data is passed by reference, the length is passed by value. This mean changing the length only affects the local copy, but changing the data affects all arrays that point to that data. I think it also helps to think of arrays as slices (that's what they are anyways). There is no more distinction between slices or arrays, they are one and the same. An array does not own its data, and that is a major point of confusion. An array always just references data, it never owns it. -Steve
Nov 09 2010
next sibling parent bearophile <bearophileHUGS lycos.com> writes:
Steven Schveighoffer:

 I think of an array as a hybrid between a reference and a value type.  The  
 data is passed by reference, the length is passed by value.  This mean  
 changing the length only affects the local copy, but changing the data  
 affects all arrays that point to that data.
Let's play some more :-) The data is passed by pointer value, the length is passed by value: void foo1(ref int[2] arr) { int[2] a = [10, 20]; arr = a; } void foo2(int[] arr) { int[2] a = [10, 20]; arr = a; } void main() { int[2] arr; arr = [1, 2]; foo1(arr); assert(arr == [10, 20]); arr = [1, 2]; foo2(arr); assert(arr == [1, 2]); } Bye, bearophile
Nov 09 2010
prev sibling next sibling parent Jesse Phillips <jessekphillips+D gmail.com> writes:
Steven Schveighoffer Wrote:

 It depends on your definition of reference type.  I agree with Daniel.  If  
 you want to get into academic definitions, yes, 'technically' an array is  
 a reference, but it's confusing to someone who's not interested in  
 exploring the theoretical parts of computer science.
I think viewing a reference as a pointer is wrong. But I did already agree that since it is claimed to be passed by reference it is odd that the length is not part of it.
 I think of an array as a hybrid between a reference and a value type.  The  
 data is passed by reference, the length is passed by value.  This mean  
 changing the length only affects the local copy, but changing the data  
 affects all arrays that point to that data.
Yet this still isn't complete because if you resize the array the data may not point to that same data anymore. I do not see this as an issue, but if you understand this, the fact that the length is passed by value isn't important, because you won't be relying on whether it points to the same data or not. D arrays are done differently then other types/languages. I don't have an issue with changing the wording to clarify, but if we are going to do that suggestions should be made. But saying it isn't a reference does nothing.
 There is no more distinction between slices or arrays, they are  
 one and the same. 
Yeah, I'm glad about this. Arrays are much nice for this.
Nov 09 2010
prev sibling next sibling parent spir <denis.spir gmail.com> writes:
On Tue, 09 Nov 2010 07:42:13 -0500
"Steven Schveighoffer" <schveiguy yahoo.com> wrote:

 I think of an array as a hybrid between a reference and a value type.  Th=
e =20
 data is passed by reference, the length is passed by value.  This mean =20
 changing the length only affects the local copy, but changing the data =20
 affects all arrays that point to that data.
The pointer is passed by value as well. Thinking at an array as a fat point= er, a (pointer,length) tuple, well, the whole tuple is a plain value. Denis -- -- -- -- -- -- -- vit esse estrany =E2=98=A3 spir.wikidot.com
Nov 09 2010
prev sibling parent reply Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 09/11/2010 12:42, Steven Schveighoffer wrote:
 On Mon, 08 Nov 2010 21:29:42 -0500, Jesse Phillips
 <jessekphillips+D gmail.com> wrote:

 Well, if you can come up with a good definition for what "increasing
 the size of a class" would do, then maybe it should be added.

 It really doesn't matter. Arrays are their own type, the have their
 own semantics. It does help to think of them in terms of slices (which
 is what TDPL refers to them as), yet that does not remove the fact
 that they are in dead a reference type.

 Many times familiar terms are used so that a behavior is quickly
 understood. For example it is common to say that arrays in C is just a
 pointer into a memory location. But in reality that is not true.
It depends on your definition of reference type. I agree with Daniel. If you want to get into academic definitions, yes, 'technically' an array is a reference, but it's confusing to someone who's not interested in exploring the theoretical parts of computer science.
What do you mean "depends on your definition of reference type" ? I think that what a reference type is, is generally understood fairly well by the majority of developers, even if understood implicitly. I think our confusion here has more to do to what we consider an "array", rather than what we consider to "reference type to be. See below.
 I think of an array as a hybrid between a reference and a value type.
 The data is passed by reference, the length is passed by value. This
 mean changing the length only affects the local copy, but changing the
 data affects all arrays that point to that data.
Well, saying _dynamic arrays_ are a hybrid, like you mentioned, is perhaps the best way to describe them, with less misunderstanding. However, if I had to say whether dynamic arrays are value types or reference types, I would agree with Jesse and call them reference types, and I would not feel this is inaccurate. Let's try a test everyone, look at this code: void test() { int[] a = [1, 2, 3]; int[] b = a; int[] c = a; } and tell us, what you would you reply if asked "how many arrays are created during test's scope"? I would say "1", and not feel it is inaccurate. However, if asked how "many dynamic arrays are created during test's scope?", I would likely say "3". This is because what I consider an array to *be*, is its (contiguous & homogeneous) elements. With that definition, then D's static arrays are value types, because when you assign a static array value to a static array variable, the underlying _array_ (ie, the contents), get copied. Conversely, D's dynamic arrays are reference types, because when you assign a dynamic array value to a dynamic array variable, the underlying "array" (ie, the contents) are not copied, instead you get two references to exactly the same "array" (ie, same data). This viewpoint gets a bit murkier because an "array" can be contained inside another "array", but that doesn't fundamentally change the it (the viewpoint that is). But ultimately neither view is right or wrong, it just depends on how we define our terms, how we conceptualize things. However, it's probably best to stick to what the spec/TDPL says about it, if it says anything specific. (I don't think it does though :/ ) Or to avoid ambiguous designations. ... Oh man, I need to clear my head. -- Bruno Medeiros - Software Engineer
Nov 26 2010
next sibling parent spir <denis.spir gmail.com> writes:
On Fri, 26 Nov 2010 19:50:27 +0000
Bruno Medeiros <brunodomedeiros+spam com.gmail> wrote:

 On 09/11/2010 12:42, Steven Schveighoffer wrote:
 On Mon, 08 Nov 2010 21:29:42 -0500, Jesse Phillips
 <jessekphillips+D gmail.com> wrote:

 Well, if you can come up with a good definition for what "increasing
 the size of a class" would do, then maybe it should be added.

 It really doesn't matter. Arrays are their own type, the have their
 own semantics. It does help to think of them in terms of slices (which
 is what TDPL refers to them as), yet that does not remove the fact
 that they are in dead a reference type.

 Many times familiar terms are used so that a behavior is quickly
 understood. For example it is common to say that arrays in C is just a
 pointer into a memory location. But in reality that is not true.
It depends on your definition of reference type. I agree with Daniel. If you want to get into academic definitions, yes, 'technically' an array is a reference, but it's confusing to someone who's not interested in exploring the theoretical parts of computer science.
=20 What do you mean "depends on your definition of reference type" ? I=20 think that what a reference type is, is generally understood fairly well=
=20
 by the majority of developers, even if understood implicitly. I think=20
 our confusion here has more to do to what we consider an "array", rather=
=20
 than what we consider to "reference type to be. See below.
=20
=20
 I think of an array as a hybrid between a reference and a value type.
 The data is passed by reference, the length is passed by value. This
 mean changing the length only affects the local copy, but changing the
 data affects all arrays that point to that data.
=20 Well, saying _dynamic arrays_ are a hybrid, like you mentioned, is=20 perhaps the best way to describe them, with less misunderstanding. =20 However, if I had to say whether dynamic arrays are value types or=20 reference types, I would agree with Jesse and call them reference types,=
=20
 and I would not feel this is inaccurate. Let's try a test everyone, look=
=20
 at this code:
=20
 void test() {
    int[] a =3D [1, 2, 3];
    int[] b =3D a;
    int[] c =3D a;
 }
=20
 and tell us, what you would you reply if asked "how many arrays are=20
 created during test's scope"? I would say "1", and not feel it is=20
 inaccurate. However, if asked how "many dynamic arrays are created=20
 during test's scope?", I would likely say "3".
=20
 This is because what I consider an array to *be*, is its (contiguous &=20
 homogeneous) elements. With that definition, then D's static arrays are=20
 value types, because when you assign a static array value to a static=20
 array variable, the underlying _array_ (ie, the contents), get copied.
 Conversely, D's dynamic arrays are reference types, because when you=20
 assign a dynamic array value to a dynamic array variable, the underlying=
=20
 "array" (ie, the contents) are not copied, instead you get two=20
 references to exactly the same "array" (ie, same data). This viewpoint=20
 gets a bit murkier because an "array" can be contained inside another=20
 "array", but that doesn't fundamentally change the it (the viewpoint=20
 that is).
I think your explanation helps & clarify why there are misunderstandings (I= mean between people in the D community). Someone used to playing with _sta= tic_ arrays in a low-level language is used to identify an array with a mem= ory area. But at a slightly higher level, an array is seen by other people = as an element in a program like an int, a struct, a set or whatever; this a= pplies to any kind of array, not only static. The thin interface implemented by D [the (pointer,length) tuple] creates a = minimal abstraction that causes these 2 points of view to diverge in practi= ce. The structure, the array element is not referenced, but the memory area= is, well, "pointed".
 But ultimately neither view is right or wrong, it just depends on how we=
=20
 define our terms, how we conceptualize things. However, it's probably=20
 best to stick to what the spec/TDPL says about it, if it says anything=20
 specific. (I don't think it does though :/ ) Or to avoid ambiguous=20
 designations.
In many languages, there are data structures very similar to D's dynamic ar= rays, but where the element is referenced as well. So that when newcomers r= ead that D arrays are referenced, they can only think "yes, of course", and= fall into the trap. Note that the "pointing" here has no semantic value, unlike what is usually= meant by the notion of reference type; it is instead plain internal mechan= ics, a necessity to implement a value of variable length. Either copying the area on assignment (real value type), or on the contrary= referencing the element (real ref type), would both result in array types = in which these internal mechanics are opaque to the user. Note that I don't= advocate for this; I have finally understood the efficiency of D's choice.= But it's hard to get.
 ... Oh man, I need to clear my head.
... Denis -- -- -- -- -- -- -- vit esse estrany =E2=98=A3 spir.wikidot.com
Nov 26 2010
prev sibling next sibling parent Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 26/11/2010 19:50, Bruno Medeiros wrote:
 Well, saying _dynamic arrays_ are a hybrid, like you mentioned, is
 perhaps the best way to describe them, with less misunderstanding.

 However, if I had to say whether dynamic arrays are value types or
 reference types, I would agree with Jesse and call them reference types,
 and I would not feel this is inaccurate.
ARRGHH, actually this is not entirely true, I forgot about something: null vs. empty arrays. Indeed on assignment (and parameter passing) dynamic arrays work just like reference types. The reallocation operations also don't go against that either. However, in D's dynamic arrays, null is the same as an empty array! Which in practice means there is no proper null "value" for dynamic arrays, like there is for other reference types, and I think that is kinda of an essential characteristic for a reference type. So actually I would not call D's dynamic arrays "reference types" and feel this is really accurate. -- Bruno Medeiros - Software Engineer
Nov 26 2010
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 26 Nov 2010 14:50:27 -0500, Bruno Medeiros  
<brunodomedeiros+spam com.gmail> wrote:

 On 09/11/2010 12:42, Steven Schveighoffer wrote:
 On Mon, 08 Nov 2010 21:29:42 -0500, Jesse Phillips
 <jessekphillips+D gmail.com> wrote:

 Well, if you can come up with a good definition for what "increasing
 the size of a class" would do, then maybe it should be added.

 It really doesn't matter. Arrays are their own type, the have their
 own semantics. It does help to think of them in terms of slices (which
 is what TDPL refers to them as), yet that does not remove the fact
 that they are in dead a reference type.

 Many times familiar terms are used so that a behavior is quickly
 understood. For example it is common to say that arrays in C is just a
 pointer into a memory location. But in reality that is not true.
It depends on your definition of reference type. I agree with Daniel. If you want to get into academic definitions, yes, 'technically' an array is a reference, but it's confusing to someone who's not interested in exploring the theoretical parts of computer science.
What do you mean "depends on your definition of reference type" ? I think that what a reference type is, is generally understood fairly well by the majority of developers, even if understood implicitly. I think our confusion here has more to do to what we consider an "array", rather than what we consider to "reference type to be. See below.
A class is a reference type. Every operation on a class instance operates on all aliases to that class, only assignment to another class instance of the same type decouples it. Consider these two statements: "a class is a reference type" class C { int length; } foo(C c) { c.length = 5; // because a class is a reference type, this affects the original, right? } "an array is a reference type" foo(int[] arr) { arr.length = 5; // because an array is a reference type, this affects the original, right? No?!!! } This is the major confusion that I think people see. I would say people assume a "reference type" means that the reference's members are all shared between all aliases. For an array, it is one level removed, and that confuses the hell out of people. But the power gained by doing D's way is worth way waaaay more than confusing some noobs. Note that many people are used to object-based languages such as C# and Java where arrays actually *are* class instances, and therefore full reference types. -Steve
Nov 29 2010
next sibling parent Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 29/11/2010 14:13, Steven Schveighoffer wrote:
 On Fri, 26 Nov 2010 14:50:27 -0500, Bruno Medeiros
 <brunodomedeiros+spam com.gmail> wrote:

 On 09/11/2010 12:42, Steven Schveighoffer wrote:
 On Mon, 08 Nov 2010 21:29:42 -0500, Jesse Phillips
 <jessekphillips+D gmail.com> wrote:

 Well, if you can come up with a good definition for what "increasing
 the size of a class" would do, then maybe it should be added.

 It really doesn't matter. Arrays are their own type, the have their
 own semantics. It does help to think of them in terms of slices (which
 is what TDPL refers to them as), yet that does not remove the fact
 that they are in dead a reference type.

 Many times familiar terms are used so that a behavior is quickly
 understood. For example it is common to say that arrays in C is just a
 pointer into a memory location. But in reality that is not true.
It depends on your definition of reference type. I agree with Daniel. If you want to get into academic definitions, yes, 'technically' an array is a reference, but it's confusing to someone who's not interested in exploring the theoretical parts of computer science.
What do you mean "depends on your definition of reference type" ? I think that what a reference type is, is generally understood fairly well by the majority of developers, even if understood implicitly. I think our confusion here has more to do to what we consider an "array", rather than what we consider to "reference type to be. See below.
A class is a reference type. Every operation on a class instance operates on all aliases to that class, only assignment to another class instance of the same type decouples it. Consider these two statements: "a class is a reference type" class C { int length; } foo(C c) { c.length = 5; // because a class is a reference type, this affects the original, right? } "an array is a reference type" foo(int[] arr) { arr.length = 5; // because an array is a reference type, this affects the original, right? No?!!! } This is the major confusion that I think people see. I would say people assume a "reference type" means that the reference's members are all shared between all aliases. For an array, it is one level removed, and that confuses the hell out of people. But the power gained by doing D's way is worth way waaaay more than confusing some noobs.
Hum, I see your point, yeah, I guess I do agree somewhat. I mean, it goes back to the issue of what one thinks the "array" is. There is no misunderstanding in the code above if one considers the array to be just it's elements... However, as you point out, it would not be uncommon that a newbie would consider the "members" of the array to be part of it (and thus shared across all aliases). It's not even unreasonable to think that, in fact. But then with that mental model it would be quite inaccurate to say "an array is a reference type", yeah. (even disregarding the issues with empty vs null arrays, a different beast altogether) -- Bruno Medeiros - Software Engineer
Nov 29 2010
prev sibling parent spir <denis.spir gmail.com> writes:
On Mon, 29 Nov 2010 09:13:17 -0500
"Steven Schveighoffer" <schveiguy yahoo.com> wrote:

 A class is a reference type.  Every operation on a class instance operate=
s =20
 on all aliases to that class, only assignment to another class instance o=
f =20
 the same type decouples it.
=20
 Consider these two statements:
=20
 "a class is a reference type"
=20
 class C
 {
     int length;
 }
=20
 foo(C c)
 {
    c.length =3D 5; // because a class is a reference type, this affects t=
he =20
 original, right?
 }
=20
 "an array is a reference type"
=20
 foo(int[] arr)
 {
     arr.length =3D 5; // because an array is a reference type, this affec=
ts =20
 the original, right?  No?!!!
 }
=20
 This is the major confusion that I think people see.  I would say people =
=20
 assume a "reference type" means that the reference's members are all =20
 shared between all aliases.  For an array, it is one level removed, and =
=20
 that confuses the hell out of people.  But the power gained by doing D's =
=20
 way is worth way waaaay more than confusing some noobs.
=20
 Note that many people are used to object-based languages such as C# and =
=20
 Java where arrays actually *are* class instances, and therefore full =20
 reference types.
Very good explanation of the mental issue for newcomers. Note that this doe= s not apply only to people coming from languages similar to Java or C#, but= also from most, if not all, dynamic languages (including one that are not = "officially" OO, like Lua). Denis -- -- -- -- -- -- -- vit esse estrany =E2=98=A3 spir.wikidot.com
Nov 30 2010
prev sibling parent spir <denis.spir gmail.com> writes:
On Tue, 9 Nov 2010 02:14:17 +0100
Daniel Gibson <metalcaedes gmail.com> wrote:

 On Tue, Nov 9, 2010 at 1:24 AM, Jesse Phillips
 <jessekphillips+D gmail.com> wrote:
 The array-struct is the reference, so it is what gets compared. That me=
ans both the internal pointer and length must be the same. Just because the= reference is more than an address does not make it any less a reference.

=20
 Unlike in C, a D array is more than a reference.
 An array is the data (or a reference to its data) + metadata (its
 length) - the metadata belongs to the array (and not to the
 reference).
 This means that, when you say "the array is passed by reference" one
 expects that also the arrays length is passed by reference - because
 the length belongs to the array.
Exactly. When a D object (class instance) holds value or reference fields, changing = any of them affects other variables denoting the same object, right? This i= s not true for D dyn arrays. D object implement reference semantics, while = D arrays implement value semantics. Or rather, they do it superficially (sh= allow copy). To have true value semantics, one would need a kind of this(th= is) copy constructor that also copies the target memory area adressed by th= e array's internal pointer.
 The distinction is that an Array can have its reference changed by resi=
zing (which is not an option for other reference types).

=20
 If that is not an option for any other reference type, why should it
 be an option for arrays? That is just inconsistent and doesn't make
 any sense.
=20
 No, Arrays should not be considered reference types when passed to a func=
tion.
 As someone else said before: Logically you don't pass the array but a
 slice that contains the whole array.
Exactly, again. The comparison with slices makes sense. If I don't mess up = everything after all those discussions, "b =3D a;" has the same semantics a= s "b =3D a[0..$];". A new array struct is built with equal pointer & length= as the original one. Meaning both arrays _initially_ address the same memo= ry area. If D arrays were referenced instead, then no copy would happen at all, inst= ead a reference to the same struct would be created; so that later changes = would be shared -- however they internally happen, including reallocation. = Like for D objects. For people used to manually implement variable-size data structures (eg in = plain C), saying that a D dyn array is a kind of (pointer,length) tuple (a = fat pointer) defining an internal memory area (static array) is also useful= . They can imagine the internal mechanics and from there infer actual behav= iour. But saying that D dyn arrays are referenced can only bring confusion. And s= tating a parallel with D objects even more: they do _not_ behave the same w= ay. Denis -- -- -- -- -- -- -- vit esse estrany =E2=98=A3 spir.wikidot.com
Nov 09 2010
prev sibling next sibling parent reply Jens Mueller <jens.k.mueller gmx.de> writes:
Jesse Phillips wrote:
 Jens Mueller Wrote:
 
 Hi,
 
 I do not understand what's going on behind the scene with this code. Or
 better said I have some idea but maybe I do not see the whole point.
 
 void foo(int[] array) {
 	array.length += 1000; // may copy the array
 	array[0] = 1;
 }
 
 auto a = new int[1];
 foo(a);
 assert(a[0] == 1); // fails if a needs to copied inside foo
 
...
 I find this behavior rather strange. Arrays are neither passed by value
 (copying the whole array) nor by reference. I see reasons for doing it
 like this, e.g. doing array = array[1..$] inside should not affect the
 outside.
 But I wonder whether these semantics are well enough documented?
 I think I should use ref int[] in the example above, shouldn't I?
 
 Jens
But they are past by reference. You can modify the data all you want, but cannot reassign the reference itself. Resizing the array may cause a reassignment of that reference. It is not different from the following code except resizing does not guarantee a reference change. import std.stdio; void assignValue(A a) { a = new A(); a.value = 6; } class A { int value; } void main() { A a = new A(); assignValue(a); writeln(a.value); }
I like that explanation. Jonathan is saying the same, I think. I'll guess my misunderstanding is mainly caused by figuring out that a reassign is happening and that a reassign to a reference changes the reference. In C++ you cannot change a reference (I hope I'm right here.). When using a std::vector one does not need to think about this. What's the general use of a = new A() in the above code? Where is it useful? Jens
Nov 08 2010
next sibling parent reply Jesse Phillips <jessekphillips+D gmail.com> writes:
Jens Mueller Wrote:

 I like that explanation. Jonathan is saying the same, I think.
Yes, same thing.
 In C++ you cannot change a reference (I hope I'm right
 here.). When using a std::vector one does not need to think about this.
Don't know too much about C++ references, but as mentioned somewhere you can use the Array container which won't have this issue
 What's the general use of a = new A() in the above code? Where is it
 useful?
 
 Jens
I don't really have any good use-case examples. Maybe an initialization function? Developed your own number object (big int) and were thinking in terms of it being a refrence you thought a = a + BigInt(7); would result in a being resigned in the calling function. Or maybe just a function that swaps two class references: void swap(T)(T a, T b) { // Correct void swap(T)(ref T a, ref T b) { auto tmp = a; a = b; b = tmp; } Actually that turned out to be a pretty good one.
Nov 08 2010
next sibling parent Jens Mueller <jens.k.mueller gmx.de> writes:
 What's the general use of a = new A() in the above code? Where is it
 useful?
 
 Jens
I don't really have any good use-case examples. Maybe an initialization function? Developed your own number object (big int) and were thinking in terms of it being a refrence you thought a = a + BigInt(7); would result in a being resigned in the calling function. Or maybe just a function that swaps two class references: void swap(T)(T a, T b) { // Correct void swap(T)(ref T a, ref T b) { auto tmp = a; a = b; b = tmp; } Actually that turned out to be a pretty good one.
I do see why a = new A() is useful. But it makes only sense if I passed a/b as ref a/b. Basically I wonder why I do not get a warning when changing the reference in that situation. So my question is more why am I allowed to change the reference even though I didn't pass it as ref a. I was looking for a use of that. Assuming there is good use then there is no reason to forbid it. But if there is no good use I'd like to be warned when compiling the above swap. Because it's an error. For dynamic arrays slicing is a good example to allow it. But why should it be allowed for objects? Jens
Nov 09 2010
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday 09 November 2010 02:43:57 Jens Mueller wrote:
 What's the general use of a = new A() in the above code? Where is it
 useful?
 
 Jens
I don't really have any good use-case examples. Maybe an initialization function? Developed your own number object (big int) and were thinking in terms of it being a refrence you thought a = a + BigInt(7); would result in a being resigned in the calling function. Or maybe just a function that swaps two class references: void swap(T)(T a, T b) { // Correct void swap(T)(ref T a, ref T b) { auto tmp = a; a = b; b = tmp; } Actually that turned out to be a pretty good one.
I do see why a = new A() is useful. But it makes only sense if I passed a/b as ref a/b. Basically I wonder why I do not get a warning when changing the reference in that situation. So my question is more why am I allowed to change the reference even though I didn't pass it as ref a. I was looking for a use of that. Assuming there is good use then there is no reason to forbid it. But if there is no good use I'd like to be warned when compiling the above swap. Because it's an error. For dynamic arrays slicing is a good example to allow it. But why should it be allowed for objects?
Why wouldn't you be able to change it? You can change any parameter as long as it's not const or immutable (or in, which implies const). The fact that it's a referenc is irrelevant. I can assign whatever I want to references and pointers in a function whether they were passed in or not. Sure, if you want to alter the original pointer or reference, that's not going to work unless it was passed as ref, but that's the same as any other parameter. Why would you expect altering a reference to alter the original? Sure, altering what it _refers to_ should alter what the original refers to because they refer to the same thing, but altering the reference itself shouldn't alter the original because they're two different references. There is no reason why parameters should have to stay the same as what they were passed in as - regardless of whether they're value types or reference types. If you want that behavior, use const, in, or immutable. - Jonathan M Davis
Nov 09 2010
prev sibling parent reply Jens Mueller <jens.k.mueller gmx.de> writes:
 I don't really have any good use-case examples. Maybe an initialization
 function? Developed your own number object (big int) and were thinking
 in terms of it being a refrence you thought
 
 a = a + BigInt(7);
 
 would result in a being resigned in the calling function. Or maybe just a
 function that swaps two class references:
 
 void swap(T)(T a, T b) { // Correct void swap(T)(ref T a, ref T b) {
 
     auto tmp = a; a = b; b = tmp;
 
 }
 
 Actually that turned out to be a pretty good one.
I do see why a = new A() is useful. But it makes only sense if I passed a/b as ref a/b. Basically I wonder why I do not get a warning when changing the reference in that situation. So my question is more why am I allowed to change the reference even though I didn't pass it as ref a. I was looking for a use of that. Assuming there is good use then there is no reason to forbid it. But if there is no good use I'd like to be warned when compiling the above swap. Because it's an error. For dynamic arrays slicing is a good example to allow it. But why should it be allowed for objects?
Why wouldn't you be able to change it? You can change any parameter as long as it's not const or immutable (or in, which implies const). The fact that it's a referenc is irrelevant. I can assign whatever I want to references and pointers in a function whether they were passed in or not. Sure, if you want to alter the original pointer or reference, that's not going to work unless it was passed as ref, but that's the same as any other parameter. Why would you expect altering a reference to alter the original? Sure, altering what it _refers to_ should alter what the original refers to because they refer to the same thing, but altering the reference itself shouldn't alter the original because they're two different references. There is no reason why parameters should have to stay the same as what they were passed in as - regardless of whether they're value types or reference types. If you want that behavior, use const, in, or immutable.
I see your point. You argue that the behavior is consistent. My point is that this consistency can lead to bugs. I may forget the ref. But I'll keep in mind to never forget the ref if it is needed. Jens
Nov 09 2010
parent reply Jesse Phillips <jessekphillips+D gmail.com> writes:
Jens Mueller Wrote:

 I see your point. You argue that the behavior is consistent. My point is
 that this consistency can lead to bugs. I may forget the ref. But I'll
 keep in mind to never forget the ref if it is needed.
 
 Jens
Well, in the case of classes, I don't think it would be very common. For arrays it can be nice since you can assign back a slice of the array that you want to work with. void main(string args) { args = args[1..$]; } Otherwise I suggest you start labeling all parameters with in. Then you are prevented from modifying the reference, and can decide if it should be a ref parameter. Who knows, maybe you'll find a reason to leave it off.
Nov 09 2010
parent Jens Mueller <jens.k.mueller gmx.de> writes:
 I see your point. You argue that the behavior is consistent. My point is
 that this consistency can lead to bugs. I may forget the ref. But I'll
 keep in mind to never forget the ref if it is needed.
 
 Jens
Well, in the case of classes, I don't think it would be very common. For arrays it can be nice since you can assign back a slice of the array that you want to work with. void main(string args) { args = args[1..$]; } Otherwise I suggest you start labeling all parameters with in. Then you are prevented from modifying the reference, and can decide if it should be a ref parameter. Who knows, maybe you'll find a reason to leave it off.
With dynamic arrays I totally agree. Slicing is very useful. I just want a safe rule to work with it. Maybe that's not needed anymore because by now I spend enough time on this that probably I'll never forget. With in I also disallow changing the object itself, right? I want to only forbid the changing of the reference of an argument inside the function. With in/const I disallow every change. I may want to change data of an array. Jens PS The more we talk about it, the more I come to the conclusion that this is just something you need to know. I can live with that.
Nov 10 2010
prev sibling parent reply so <so so.do> writes:
 I like that explanation. Jonathan is saying the same, I think. I'll
 guess my misunderstanding is mainly caused by figuring out that a
 reassign is happening and that a reassign to a reference changes the
 reference. In C++ you cannot change a reference (I hope I'm right
 here.). When using a std::vector one does not need to think about this.
 What's the general use of a = new A() in the above code? Where is it
 useful?

 Jens
Yes in C++, you can't redirect a reference after initialization. And also can't have a std::vector of references which is mainly by this reason. -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Nov 08 2010
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, November 08, 2010 16:11:46 so wrote:
 I like that explanation. Jonathan is saying the same, I think. I'll
 guess my misunderstanding is mainly caused by figuring out that a
 reassign is happening and that a reassign to a reference changes the
 reference. In C++ you cannot change a reference (I hope I'm right
 here.). When using a std::vector one does not need to think about this.
 What's the general use of a = new A() in the above code? Where is it
 useful?
 
 Jens
Yes in C++, you can't redirect a reference after initialization. And also can't have a std::vector of references which is mainly by this reason.
D references are more like Java references. They're really pointers, but you don't have to dereference them, so you can reassign them to something else. You can't do that in C++, because references are treated more like name aliases of variables than pointers. So, if you think in terms of C++ pointers vs D references, then C++ and D function the same in this respect. - Jonathan M Davis
Nov 08 2010
parent reply Rainer Deyke <rainerd eldwood.com> writes:
On 11/8/2010 17:43, Jonathan M Davis wrote:
 D references are more like Java references.
That's true for class references. D also supports pass-by-reference through the 'ref' keyword, which works like C++ references. -- Rainer Deyke - rainerd eldwood.com
Nov 12 2010
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday 12 November 2010 17:55:31 Rainer Deyke wrote:
 On 11/8/2010 17:43, Jonathan M Davis wrote:
 D references are more like Java references.
That's true for class references. D also supports pass-by-reference through the 'ref' keyword, which works like C++ references.
True. But generally when talking about references, class references are what is being referred to. Personally, I would say that in the case of a ref parameter, the argument is being passed _by_ reference but not that it _is_ a reference. I suppose that the whole issue could get pretty confusing though if we're not clear on what we're referring to. - Jonathan M Davis
Nov 12 2010
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, November 08, 2010 13:54:28 spir wrote:
 On Mon, 08 Nov 2010 15:32:56 -0500
 
 Jesse Phillips <jessekphillips+D gmail.com> wrote:
 But they are past by reference. You can modify the data all you want, but
 cannot reassign the reference itself.
No, they are _not_ passed by reference. Stop saying that, this is precisely what causes confusion. This is reference semantics:
As Jesse says, they _are_ passed by reference. The struct itself _is_ the reference. What makes arrays odd is that anything that resizes it returns a _new_ reference or alters the current one. So, using ~= or length can result in a reference with a new ptr value and a new length value, and the reference then refers to a different block of memory. In the case of an object reference, you could implement ~= to return a new reference in exactly the same manner and get behavior similar to an array, but no one does that normally. _Everything_ which is not passed as ref or out is passed by value in D. And technically, ref and out probably wrap the value being passed in a pointer and that pointer is passed by value. Really, this is easier to understand when dealing with pointers than references, since references hide the dereferencing process. Take this program for instance: import std.stdio; void func1(int* b) { *b = 12; } void func2(int* c) { c = new int; *c = 7; } void func3(int** d) { *d = new int; **d = 3; } void main() { int* i = new int; int* j = i; *i = 5; writefln("%s %s", *i, *j); func1(i); writefln("%s %s", *i, *j); func2(i); writefln("%s %s", *i, *j); func3(&i); writefln("%s %s", *i, *j); } It prints out 5 5 12 12 12 12 3 12 i and j both point to the same memory, so when that memory is altered, the value that you get when you dereference them is altered. When func1() is called, the value of i is passed to func1() - that is the address that i points to - so b points to the same memory that i and j do, so altering the value in that memory, alters the memory for all three pointers. When calling func2(), c is a copy of i and holds the same address. However, setting c to a new address means that it no longer points to the same memory and any changes made to the memory that it points to does not alter the memory that i and j point to. So, they still print out the same value when dereferenced. func3() takes the address of i. That means that d the holds the address of i and can alter i itself rather than just the memory that it points to. c points to the memory that holds i itself. So, when assigning the dereferenced c to a new address, i itself receives a new address. Then when altering the memory that's pointed to by the address that c holds, you alter the memory that i points to and change that value. However, while j held the same address as i originally and altering the memory at that address therefore altered what both i and j pointed to, once i was given a new address by c, they i and j held different addresses and altering the memory that one pointed to did not alter the one that the other pointed to. I expect that you've gone over this sort of thing before, but you need to realize that references are _exactly_ the same. The only difference is the lack of dereferencing step when accessing what it points to. So, whether you're passing object reference or an array reference, you're passing that reference by value, and any changes to the copy won't change the original. Changes to what the reference refers to _will_ change what the original reference points to as well since they point to the same thing, but changes to the copied reference won't change the original. So, if make the copied reference point to something else, then it won't change what the original reference pointed to anymore. Having an array that you use ~= on is like doing objRef = new Foo(); on a reference that was passed into the function. It now points to something else, so it's not going to affect the original. What gets somewhat weird about it is that ~= doesn't necessarily return a new reference, and if it doesn't then the length is different, so it still hasn't affected the original reference (if it was going to, it would have resulted in a reallocation). The value of the copied reference has changed. I grant you that if you're expecting ~= to return the same reference every time and thereby somehow alter the original reference, then you're going to be surprised. But it's completely consistent with how pointers and references work. You can alter the memory that they point to, but since the pointer or reference itself was passed by value, altering it will not alter the original. - Jonathan M Davis
Nov 08 2010
prev sibling parent reply spir <denis.spir gmail.com> writes:
On Mon, 8 Nov 2010 17:08:32 -0800
Jonathan M Davis <jmdavisProg gmx.com> wrote:

 As Jesse says, they _are_ passed by reference. The struct itself _is_ the=
=20
 reference.=20
(Well, that is a sensible redefinition of "reference"; but it is simply _no= t_ what the word means in any other context.) It is true that the inner, hidden, memory area (static array) containing th= e elements is indeed referenced, actually "pointed", from the dynamic array= struct: struct ValueArray(Element) { Element* elements; uint length; } (Well, actually, this may not be a struct, but it's easier to imagine it so= .) But: the dyn array itself, meaning the struct, is not referenced: "a2 =3D a= 1" copies it, as well as parameter passing. And the fact that the internal = memory is referenced is an implementation detail that should *not* affect s= emantics. The inner pointer is there because we need some kind of indirecti= on to implement variable-size thingies, and the means for this is pointers. This is precisely where & why people get bitten: implementation leaks out i= nto semantics. Actually, one could conceptually replace the (pointer,length) pair by a sin= gle field of type MemoryArea -- which would be a plain value. Then, there w= ould be no more (visible) pointer in the dyn array, right? (Actually, it wo= uld just be hidden deeper inside the MemoryArea field... but that is again = implementation detail!) We should not mess up pointers used for implementation mechanics, like in t= he case of dyn arrays, or more generally variable size data structure, with= pointers used as true references carrying semantics, like in the case of t= he difference between struct and class. And precisely, replacing array struct by a class, or explicitely referencin= g the struct, would make a *reference* dyn array type. See below an example= of a primitive sort of such an array type (you can only put new elements i= n it ;-), implemented as class. After "a2 =3D a1", every change to one of the vars affects the other var; w= hether the change requires reallocation is irrelevant; this detail belongs = to implementation, not to semantics. Now, replace class with struct and you have a type for *value* dyn arrays. = Which works exactly like D ones. The assertion will fail; and output should be interesting ;-) Hope it's clear, because I cannot do better. I do not mean that D arrays are bad in any way. They work perfectly and are= very efficient. Enforcing a true interface between implementation and sema= ntics would certainly have a relevant cost in terms of space & time. But pl= ease, stop stating D arrays are referenced if you want newcomers to have a = chance & understand the actual behaviour, to use them without beeing consta= ntly bitten, and to stop & complain. Denis class RefArray(Element) { Element* elements; uint length; private uint capacity; this () { this.elements =3D cast(Element*) malloc(Element.sizeof); this.capacity =3D 1; this.length =3D 0; } void reAlloc() { writeln("realloc"); this.capacity *=3D 2; size_t memSize =3D this.capacity * Element.sizeof; realloc(this.elements, memSize); } void put(Element element) { if (this.length >=3D this.capacity) this.reAlloc(); this.elements[this.length] =3D element; ++ this.length; } void opBinary(string op) (Element element) if (op =3D=3D "+") { this.put(element); } } void main () { auto a1 =3D new RefArray!int(); auto a2 =3D a1; foreach (int i ; 1..8) {a2.put(i);} assert(a1.length =3D=3D a2.length); foreach (int i ; 0 .. a2.length) writef("%s=3D%s ", a1.elements[i], a2.elements[i]); writeln(); a1 + 8; a1 + 9; foreach (int i ; 0 .. a2.length) writef("%s=3D%s ", a1.elements[i], a2.elements[i]); writeln(); } -- -- -- -- -- -- -- vit esse estrany =E2=98=A3 spir.wikidot.com
Nov 09 2010
parent =?UTF-8?B?UGVsbGUgTcOlbnNzb24=?= <pelle.mansson gmail.com> writes:
On 11/09/2010 09:36 AM, spir wrote:
 On Mon, 8 Nov 2010 17:08:32 -0800
 Jonathan M Davis<jmdavisProg gmx.com>  wrote:

 As Jesse says, they _are_ passed by reference. The struct itself _is_ the
 reference.
(Well, that is a sensible redefinition of "reference"; but it is simply _not_ what the word means in any other context.) It is true that the inner, hidden, memory area (static array) containing the elements is indeed referenced, actually "pointed", from the dynamic array struct: struct ValueArray(Element) { Element* elements; uint length; } (Well, actually, this may not be a struct, but it's easier to imagine it so.) But: the dyn array itself, meaning the struct, is not referenced: "a2 = a1" copies it, as well as parameter passing. And the fact that the internal memory is referenced is an implementation detail that should *not* affect semantics. The inner pointer is there because we need some kind of indirection to implement variable-size thingies, and the means for this is pointers. This is precisely where& why people get bitten: implementation leaks out into semantics. Actually, one could conceptually replace the (pointer,length) pair by a single field of type MemoryArea -- which would be a plain value. Then, there would be no more (visible) pointer in the dyn array, right? (Actually, it would just be hidden deeper inside the MemoryArea field... but that is again implementation detail!) We should not mess up pointers used for implementation mechanics, like in the case of dyn arrays, or more generally variable size data structure, with pointers used as true references carrying semantics, like in the case of the difference between struct and class. And precisely, replacing array struct by a class, or explicitely referencing the struct, would make a *reference* dyn array type. See below an example of a primitive sort of such an array type (you can only put new elements in it ;-), implemented as class. After "a2 = a1", every change to one of the vars affects the other var; whether the change requires reallocation is irrelevant; this detail belongs to implementation, not to semantics. Now, replace class with struct and you have a type for *value* dyn arrays. Which works exactly like D ones. The assertion will fail; and output should be interesting ;-) Hope it's clear, because I cannot do better. I do not mean that D arrays are bad in any way. They work perfectly and are very efficient. Enforcing a true interface between implementation and semantics would certainly have a relevant cost in terms of space& time. But please, stop stating D arrays are referenced if you want newcomers to have a chance& understand the actual behaviour, to use them without beeing constantly bitten, and to stop& complain. Denis class RefArray(Element) { Element* elements; uint length; private uint capacity; this () { this.elements = cast(Element*) malloc(Element.sizeof); this.capacity = 1; this.length = 0; } void reAlloc() { writeln("realloc"); this.capacity *= 2; size_t memSize = this.capacity * Element.sizeof; realloc(this.elements, memSize); } void put(Element element) { if (this.length>= this.capacity) this.reAlloc(); this.elements[this.length] = element; ++ this.length; } void opBinary(string op) (Element element) if (op == "+") {
...wait! Did you just overload binary operator + to mean append?
Nov 09 2010
prev sibling parent reply so <so so.do> writes:
D arrays very powerful but you first need to understand what is going on.  
You should check the book.
An inconsistency is the copy of static arrays at assignment, but necessary  
one.
One thing i don't like about D arrays is an undefined case in dynamic  
array reallocation.

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/
Nov 08 2010
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 11/8/10 4:50 PM, so wrote:
 D arrays very powerful but you first need to understand what is going
 on. You should check the book.
Or a mildly outdated but accurate preview of the relevant chapter: http://erdani.com/d/thermopylae.pdf Andrei
Nov 08 2010
prev sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, November 08, 2010 16:50:46 so wrote:
 D arrays very powerful but you first need to understand what is going on.
 You should check the book.
 An inconsistency is the copy of static arrays at assignment, but necessary
 one.
 One thing i don't like about D arrays is an undefined case in dynamic
 array reallocation.
It's perfectly defined, just not knowable at compile time. You can even check the array's capacity if you want to try and figure out when it's going to happen. And there's not really any reasonable alternative. What would have happen instead? Make an array reallocate _every_ time that it's resized? That would be highly inefficient and could really degrade performance. Appending becomes O(n) instead of amortized O(1). If you're not altering the actual elements of the array, then the current implementation is great. If you _are_ altering them, then simply dup the array to guarantee that it's been reallocated. - Jonathan M Davis
Nov 08 2010
next sibling parent so <so so.do> writes:
I didn't mean that one, check page 112 on  
http://erdani.com/d/thermopylae.pdf

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/
Nov 08 2010
prev sibling next sibling parent so <so so.do> writes:
Oh yeh you are right, i said reallocation. Should have said assignment.

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/
Nov 08 2010
prev sibling parent reply Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 09/11/2010 01:43, Jonathan M Davis wrote:
 On Monday, November 08, 2010 16:50:46 so wrote:
 D arrays very powerful but you first need to understand what is going on.
 You should check the book.
 An inconsistency is the copy of static arrays at assignment, but necessary
 one.
 One thing i don't like about D arrays is an undefined case in dynamic
 array reallocation.
It's perfectly defined, just not knowable at compile time. You can even check the array's capacity if you want to try and figure out when it's going to happen. And there's not really any reasonable alternative. What would have happen instead? Make an array reallocate _every_ time that it's resized? That would be highly inefficient and could really degrade performance. Appending becomes O(n) instead of amortized O(1). If you're not altering the actual elements of the array, then the current implementation is great. If you _are_ altering them, then simply dup the array to guarantee that it's been reallocated. - Jonathan M Davis
Making the array reallocate _every_ time that it's resized (to a greater length) is actually not that unreasonable. Would it be highly inneficient? Only if you write bad code. TDPL agrees with you, I quote: " One easy way out would be to always reallocate a upon appending to it [...] Although that behavior is easiest to implement, it has serious efficiency problems. For example, oftentimes arrays are iteratively grown in a loop: int[] a; foreach (i; 0 .. 100) { a ~= i; } " Hum, "oftentimes"? I wonder if such code is really that common (and what languages are we talking about here?) But more importantly, there is a simple solution: don't write such code, don't use arrays like if they are lists, preallocate instead and then fill the array. So with this alternative behavior, you can still write efficient code, and nearly as easily. The only advantage of the current behavior is that it is more noob friendly, which is an advantage of debatable value. -- Bruno Medeiros - Software Engineer
Nov 26 2010
next sibling parent reply =?ISO-8859-9?Q?Pelle_M=E5nsson?= <pelle.mansson gmail.com> writes:
On 11/26/2010 07:22 PM, Bruno Medeiros wrote:
 But more importantly, there is a simple solution: don't write such code,
 don't use arrays like if they are lists, preallocate instead and then
 fill the array. So with this alternative behavior, you can still write
 efficient code, and nearly as easily.
What about when you don't know the length before, or working with immutable elements?
 The only advantage of the current behavior is that it is more noob
 friendly, which is an advantage of debatable value.
I believe you will find to have that exactly backwards.
Nov 26 2010
parent Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 26/11/2010 19:36, Pelle Mnsson wrote:
 On 11/26/2010 07:22 PM, Bruno Medeiros wrote:
 But more importantly, there is a simple solution: don't write such code,
 don't use arrays like if they are lists, preallocate instead and then
 fill the array. So with this alternative behavior, you can still write
 efficient code, and nearly as easily.
What about when you don't know the length before, or working with immutable elements?
I must recognize I made a huge blunder with that, my reasoning was indeed very wrong. :( don't know the length: Do the exponentional growth yourself. Double the array length when it gets full, and keep track of real length in a separate variable. At the end, downsize the array to real length. - This version is significantly more complex than the code with the current behavior, significantly enough to counter my argument (the "nearly as easily" part). - It's actually also slightly less efficient, because whenever you grow the array, half of the elements have to be default-initialized, which is not the case when you just grow the capacity (using current resize behavior). I think one might resolve this by doing some cast to void[]'s and back, but that would make this code version even more complex. immutable: preallocate another array _whose elements are typed as tail-immutable_, fill it, at the end cast it to the array typed with immutable elements. - Ouch, this one is even worse, especially depending on what the type of the immutable elements are, because of the need to determine the tail-immutable version of that type. If the elements are Objects, it's not even possible to do that, so you would need to type the temporary array as void*[]. And that would make the code even more complex if you'd want to access and use the elements while the array is being constructed. (alternatively you could cast away all the immutable from the element type, but it's less safe, on a more complex loop you would risk modifying them by mistake) :/ -- Bruno Medeiros - Software Engineer
Nov 26 2010
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 11/26/10 12:22 PM, Bruno Medeiros wrote:
 On 09/11/2010 01:43, Jonathan M Davis wrote:
 On Monday, November 08, 2010 16:50:46 so wrote:
 D arrays very powerful but you first need to understand what is going
 on.
 You should check the book.
 An inconsistency is the copy of static arrays at assignment, but
 necessary
 one.
 One thing i don't like about D arrays is an undefined case in dynamic
 array reallocation.
It's perfectly defined, just not knowable at compile time. You can even check the array's capacity if you want to try and figure out when it's going to happen. And there's not really any reasonable alternative. What would have happen instead? Make an array reallocate _every_ time that it's resized? That would be highly inefficient and could really degrade performance. Appending becomes O(n) instead of amortized O(1). If you're not altering the actual elements of the array, then the current implementation is great. If you _are_ altering them, then simply dup the array to guarantee that it's been reallocated. - Jonathan M Davis
Making the array reallocate _every_ time that it's resized (to a greater length) is actually not that unreasonable. Would it be highly inneficient? Only if you write bad code. TDPL agrees with you, I quote: " One easy way out would be to always reallocate a upon appending to it [...] Although that behavior is easiest to implement, it has serious efficiency problems. For example, oftentimes arrays are iteratively grown in a loop: int[] a; foreach (i; 0 .. 100) { a ~= i; } " Hum, "oftentimes"? I wonder if such code is really that common (and what languages are we talking about here?)
It would be difficult to challenge the assumption that appends in a loop are common.
 But more importantly, there is a simple solution: don't write such code,
 don't use arrays like if they are lists, preallocate instead and then
 fill the array. So with this alternative behavior, you can still write
 efficient code, and nearly as easily.
I disagree. Often you don't know the length to preallocate (e.g. input is from a file etc). The fact that there's a convenient append operator only makes things more in favor of supporting such idioms. The technique (exponential capacity growth) is well known.
 The only advantage of the current behavior is that it is more noob
 friendly, which is an advantage of debatable value.
I don't think the current behavior favors noobs. Andrei
Nov 26 2010
next sibling parent reply Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 26/11/2010 19:16, Andrei Alexandrescu wrote:
 On 11/26/10 12:22 PM, Bruno Medeiros wrote:

 But more importantly, there is a simple solution: don't write such code,
 don't use arrays like if they are lists, preallocate instead and then
 fill the array. So with this alternative behavior, you can still write
 efficient code, and nearly as easily.
I disagree. Often you don't know the length to preallocate (e.g. input is from a file etc). The fact that there's a convenient append operator only makes things more in favor of supporting such idioms. The technique (exponential capacity growth) is well known.
 The only advantage of the current behavior is that it is more noob
 friendly, which is an advantage of debatable value.
I don't think the current behavior favors noobs. Andrei
You could still do exponential capacity growth by manipulating the length property, but yeah, that would create a host of complexity and other issues (see my reply to Pelle). Yeah, my reasoning was really broken. :'( (I need some R&R, lol) -- Bruno Medeiros - Software Engineer
Nov 26 2010
parent reply spir <denis.spir gmail.com> writes:
On Fri, 26 Nov 2010 21:59:37 +0000
Bruno Medeiros <brunodomedeiros+spam com.gmail> wrote:

 You could still do exponential capacity growth by manipulating the=20
 length property, but yeah, that would create a host of complexity and=20
 other issues (see my reply to Pelle). Yeah, my reasoning was really=20
 broken. :'(
=20 What is the reason why D does not internally manages exp growth with a (pri= vate) capacity field? It's very common, and I thought the reason was precis= ely efficiency as it does not require reallocating so often, esp in cases s= uch as feeding an array in a loop like in Andrei's example.) Denis -- -- -- -- -- -- -- vit esse estrany =E2=98=A3 spir.wikidot.com
Nov 26 2010
parent Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 26/11/2010 22:12, spir wrote:
 On Fri, 26 Nov 2010 21:59:37 +0000
 Bruno Medeiros<brunodomedeiros+spam com.gmail>  wrote:

 You could still do exponential capacity growth by manipulating the
 length property, but yeah, that would create a host of complexity and
 other issues (see my reply to Pelle). Yeah, my reasoning was really
 broken. :'(
What is the reason why D does not internally manages exp growth with a (private) capacity field? It's very common, and I thought the reason was precisely efficiency as it does not require reallocating so often, esp in cases such as feeding an array in a loop like in Andrei's example.) Denis -- -- -- -- -- -- -- vit esse estrany ☣ spir.wikidot.com
But D does exactly that, there is a capacity field (internal to the GC), and array growth is managed automatically, in an exponential way. -- Bruno Medeiros - Software Engineer
Nov 26 2010
prev sibling parent Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 26/11/2010 19:16, Andrei Alexandrescu wrote:
 "
 One easy way out would be to always reallocate a upon appending to it
 [...]
 Although that behavior is easiest to implement, it
 has serious efficiency problems. For example, oftentimes arrays are
 iteratively grown in a loop:

 int[] a;
 foreach (i; 0 .. 100) {
 a ~= i;
 }

 "

 Hum, "oftentimes"? I wonder if such code is really that common (and what
 languages are we talking about here?)
It would be difficult to challenge the assumption that appends in a loop are common.
Well, there was actually no assumption yet, I wanted first of all to know what languages you had in mind, because I'm wasn't sure I understood you correctly. C and C++ don't even have (dynamic) arrays. Java, Javascript (and C# as well, I think) have them but there is no append operation, arrays cannot be resized. So trivially that idiom is not common in these languages. :) I don't know about Python, Ruby, Erlang, Haskell, Perl, PHP. Or perhaps you were just being very liberal in your meaning of "arrays", and were also thinking of constructs like C++'s std::vector ? -- Bruno Medeiros - Software Engineer
Nov 26 2010
prev sibling next sibling parent spir <denis.spir gmail.com> writes:
On Fri, 26 Nov 2010 18:22:46 +0000
Bruno Medeiros <brunodomedeiros+spam com.gmail> wrote:

 Making the array reallocate _every_ time that it's resized (to a greater=
=20
 length) is actually not that unreasonable. Would it be highly=20
 inneficient? Only if you write bad code. TDPL agrees with you, I quote:
=20
 "
 One easy way out would be to always reallocate a upon appending to it
 [...]
 Although that behavior is easiest to implement, it
 has serious efficiency problems. For example, oftentimes arrays are=20
 iteratively grown in a loop:
=20
    int[] a;
    foreach (i; 0 .. 100) {
      a ~=3D i;
    }
=20
 "
=20
 Hum, "oftentimes"? I wonder if such code is really that common (and what=
=20
 languages are we talking about here?)
=20
 But more importantly, there is a simple solution: don't write such code,=
=20
 don't use arrays like if they are lists, preallocate instead and then=20
 fill the array. So with this alternative behavior, you can still write=20
 efficient code, and nearly as easily.
=20
 The only advantage of the current behavior is that it is more noob=20
 friendly, which is an advantage of debatable value.
Well, except that "noobs" usually don't care about performance. (Anybody would else preallocate, I guess, if only because it is just a few = more key strokes; but the corresponding idiom is not that obvious: T[] xxx =3D new T[yyy.length]; ) Denis -- -- -- -- -- -- -- vit esse estrany =E2=98=A3 spir.wikidot.com
Nov 26 2010
prev sibling parent reply Kagamin <spam here.lot> writes:
Bruno Medeiros Wrote:

 Making the array reallocate _every_ time that it's resized (to a greater 
 length) is actually not that unreasonable. Would it be highly 
 inneficient? Only if you write bad code. TDPL agrees with you, I quote:
Challenge: make D slower than C#.
Nov 26 2010
parent Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 26/11/2010 21:30, Kagamin wrote:
 Bruno Medeiros Wrote:

 Making the array reallocate _every_ time that it's resized (to a greater
 length) is actually not that unreasonable. Would it be highly
 inneficient? Only if you write bad code. TDPL agrees with you, I quote:
Challenge: make D slower than C#.
Huh? -- Bruno Medeiros - Software Engineer
Nov 26 2010