www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Thinking about the difference between fixed and 'dynamic' arrays.

reply DLearner <bmqazwsx123 gmail.com> writes:
To me, it appears that there are really two (_entirely separate_) 
concepts:

A. Supporting the useful concept of variable length (but 
otherwise entirely conventional) arrays;
B. Supporting a language feature that acts as a window to an 
array, through which that array can be manipulated.

And currently these two concepts are combined.

Suggestion: it would be clearer if the two concepts were 
separated:
1. Convert 'int[] VarArr;' so it produces a straightforward 
_value-type_ variable array, called 'VarArr';
2. Implement a new concept 'int slice Window;' to produce an 
object of type 'int slice', called 'Window'.
    'Window' is a 'slice' into an int array, not an array itself 
or even a variable.

Opinions?
Nov 29 2022
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
Okay you have misunderstand a lot here.

We have two types of arrays:

- Static, fixed sized stored on stack.
- Dynamic, variable sized, stored on the heap.

However dynamic arrays are not actually a distinct type in the type 
system, its a language extension to use runtime hooks using the GC.

What dynamic arrays are in the language is just slices.

A slice is a length + pointer pair. This is where almost all of the 
syntax for dynamic arrays come from.

```d
int[] slice;
```

That is a slice.

```d
slice ~= 32;
```

Now it is a dynamic array as it was allocated via the GC.

```d
int[4] staticArray;
slice = staticArray[];
```

The slice is now able to modify the staticArray!
Nov 29 2022
parent reply DLearner <bmqazwsx123 gmail.com> writes:
On Tuesday, 29 November 2022 at 19:06:20 UTC, rikki cattermole 
wrote:
[...]

Please see the following example:
```
void main() {

    import std.stdio;

    int[] VarArr1, VarArr2;

    VarArr1.length = 6;
    VarArr1[5] = 10;
    VarArr1[4] = 9;
    VarArr1[3] = 8;
    VarArr1[2] = 7;
    VarArr1[1] = 6;
    VarArr1[0] = 5;

    VarArr2 = VarArr1;
    writeln("VarArr1 = ", VarArr1);
    writeln("VarArr2 = ", VarArr2);

    VarArr1[3] = 40;
    writeln("VarArr1 = ", VarArr1);
    writeln("VarArr2 = ", VarArr2);

    return;
}
```

And it's result:
```
VarArr1 = [5, 6, 7, 8, 9, 10]
VarArr2 = [5, 6, 7, 8, 9, 10]
VarArr1 = [5, 6, 7, 40, 9, 10]
VarArr2 = [5, 6, 7, 40, 9, 10]
```
Many languages have fixed-length arrays, D's such construct works 
as someone approaching the language would expect.
Many languages also have variable length arrays, I suggest D's 
'dynamic array' _does not_ operate as expected.
I'm not suggesting that the result contradicts D's definition of 
'dynamic array', nor it's implementation, just that 'dynamic 
array' is not a reasonable description for a construct that 
behaves like VarArr2[3] becoming 40.
Nov 29 2022
next sibling parent matheus <matheus gmail.com> writes:
On Tuesday, 29 November 2022 at 23:25:46 UTC, DLearner wrote:
 On Tuesday, 29 November 2022 at 19:06:20 UTC, rikki cattermole 
 wrote:
 [...]

 Please see the following example:
 ...
I think this was discussed before a few weeks ago here (But I don't remember the thread), and this is a design choice, for example this: VarArr2 = VarArr1; VarArr2 is just pointing to the same address of VarArr1 as you can see by: writeln(VarArr1.ptr); writeln(VarArr2.ptr); To do what you want, you need to use "dup": VarArr2 = VarArr1.dup; Now it will work as you expect. I think this is confusing but in the end it's a design choice, instead of copy just point, and if you need to copy, you need to it explicitly. Matheus.
Nov 29 2022
prev sibling next sibling parent Siarhei Siamashka <siarhei.siamashka gmail.com> writes:
On Tuesday, 29 November 2022 at 23:25:46 UTC, DLearner wrote:
 Many languages also have variable length arrays, I suggest D's 
 'dynamic array' _does not_ operate as expected.
 I'm not suggesting that the result contradicts D's definition 
 of 'dynamic array', nor it's implementation, just that 'dynamic 
 array' is not a reasonable description for a construct that 
 behaves like VarArr2[3] becoming 40.
Which programming languages set your expectations this way? Many programming languages have the concept of "deep" vs. "shallow" copy. D is a part of a big crowd: D: ```D import std; void main() { auto a = [1, 2, 3, 4, 5]; auto b = a; auto c = a.dup; a[1] = 99; writeln(a); // [1, 99, 3, 4, 5] writeln(b); // [1, 99, 3, 4, 5] writeln(c); // [1, 2, 3, 4, 5] } ``` Python: ```Python a = [1, 2, 3, 4, 5] b = a c = a.copy() a[1] = 99 ``` Ruby/Crystal: ```Ruby a = [1, 2, 3, 4, 5] b = a c = a.dup a[1] = 99 ``` Kotlin: ```Kotlin fun main() { var a = intArrayOf(1, 2, 3, 4, 5) var b = a var c = a.copyOf() a[1] = 99 println(a.contentToString()) // [1, 99, 3, 4, 5] println(b.contentToString()) // [1, 99, 3, 4, 5] println(c.contentToString()) // [1, 2, 3, 4, 5] } ``` I could list even more languages.
Nov 29 2022
prev sibling parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 11/29/22 15:25, DLearner wrote:

 'dynamic array' is
 not a reasonable description for a construct that behaves like
 VarArr2[3] becoming 40.
I agree with you: It has always bothered me to call the following a dynamic array: int[] arr; 'arr' is not a dynamic array but the slice interface. Dynamic arrays are owned by the D runtime and are always nameless. Ali
Nov 30 2022
prev sibling next sibling parent reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Tuesday, 29 November 2022 at 18:59:46 UTC, DLearner wrote:
 Suggestion: it would be clearer if the two concepts were 
 separated:
 1. Convert 'int[] VarArr;' so it produces a straightforward 
 _value-type_ variable array, called 'VarArr';
 2. Implement a new concept 'int slice Window;' to produce an 
 object of type 'int slice', called 'Window'.
    'Window' is a 'slice' into an int array, not an array itself 
 or even a variable.

 Opinions?
Yes, that's what Rust does. It has first-class variable-size value types, D doesn't have such a feature. It is too late to change it in D, nor is it often useful in practice.
Nov 29 2022
next sibling parent Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Wednesday, 30 November 2022 at 00:40:57 UTC, Vladimir 
Panteleev wrote:
 On Tuesday, 29 November 2022 at 18:59:46 UTC, DLearner wrote:
 Suggestion: it would be clearer if the two concepts were 
 separated:
 1. Convert 'int[] VarArr;' so it produces a straightforward 
 _value-type_ variable array, called 'VarArr';
 2. Implement a new concept 'int slice Window;' to produce an 
 object of type 'int slice', called 'Window'.
    'Window' is a 'slice' into an int array, not an array 
 itself or even a variable.

 Opinions?
Yes, that's what Rust does. It has first-class variable-size value types, D doesn't have such a feature.
Cool page explaining this in The Rustonomicon: https://doc.rust-lang.org/nomicon/exotic-sizes.html
Nov 29 2022
prev sibling parent reply Siarhei Siamashka <siarhei.siamashka gmail.com> writes:
On Wednesday, 30 November 2022 at 00:40:57 UTC, Vladimir 
Panteleev wrote:
 On Tuesday, 29 November 2022 at 18:59:46 UTC, DLearner wrote:
 Suggestion: it would be clearer if the two concepts were 
 separated:
 1. Convert 'int[] VarArr;' so it produces a straightforward 
 _value-type_ variable array, called 'VarArr';
 2. Implement a new concept 'int slice Window;' to produce an 
 object of type 'int slice', called 'Window'.
    'Window' is a 'slice' into an int array, not an array 
 itself or even a variable.

 Opinions?
Yes, that's what Rust does. It has first-class variable-size value types, D doesn't have such a feature.
I'm not really familiar with Rust, but it also seems to have the concept of either making a full copy or creating a slice with a view into the existing array. Just the default assignment via `"let c = a;"` creates a copy. While creating a slice needs a more elaborate explicit syntax. Rust also appears to be picky about the order of operations: ```Rust fn main() { let mut a = [1, 2, 3, 4, 5]; let c = a; let b = &mut a; b[1] = 99; println!("{:?}", b); // [1, 99, 3, 4, 5] println!("{:?}", a); // [1, 99, 3, 4, 5] println!("{:?}", c); // [1, 2, 3, 4, 5] } ```
 It is too late to change it in D, nor is it often useful in 
 practice.
If this is really desired, then the D compiler can probably introduce a more verbose syntax for shallow array copies and start spitting out warnings about simple assignments like `"auto b = a;"`. A few years later the old syntax can be dropped. ```D import std; void main() { auto a = [1, 2, 3, 4, 5]; auto b = a.slice; // Not supported right now, but maybe is more readable? auto c = a.dup; a[1] = 99; writeln(a); // [1, 99, 3, 4, 5] writeln(b); // [1, 99, 3, 4, 5] writeln(c); // [1, 2, 3, 4, 5] } ``` But way too many languages behave in the same way as D right now. I personally don't see any problem.
Nov 29 2022
parent Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Wednesday, 30 November 2022 at 01:53:10 UTC, Siarhei Siamashka 
wrote:


 Rust also appears to be picky about the order of operations:

 ```Rust
 fn main() {
     let mut a = [1, 2, 3, 4, 5];
     let c = a;
     let b = &mut a;

     b[1] = 99;

     println!("{:?}", b); // [1, 99, 3, 4, 5]
     println!("{:?}", a); // [1, 99, 3, 4, 5]
     println!("{:?}", c); // [1, 2, 3, 4, 5]
 }
 ```
This seems unsurprising to me, `b` is a slice. The same in D: ```d import std.array, std.stdio; void main() { version (dynamic) { auto a = [1, 2, 3, 4, 5]; auto c = a.dup; auto b = a; } else { auto a = [1, 2, 3, 4, 5].staticArray; auto c = a; auto b = a[]; } b[1] = 99; writeln(b); // [1, 99, 3, 4, 5] writeln(a); // [1, 99, 3, 4, 5] writeln(c); // [1, 2, 3, 4, 5] } ``` I agree the syntax is inconsistent.
 It is too late to change it in D, nor is it often useful in 
 practice.
If this is really desired, then the D compiler can probably introduce a more verbose syntax for shallow array copies and start spitting out warnings about simple assignments like `"auto b = a;"`. A few years later the old syntax can be dropped.
I only meant that variable-sized value types are not often useful in practice.
 But way too many languages behave in the same way as D right 
 now. I personally don't see any problem.
Slices are such a fundamental feature of D that it is unrealistic to think about changing syntax there. It would effectively be a new language, because almost no programs from before the change would compile after the change.
Nov 29 2022
prev sibling next sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Tuesday, 29 November 2022 at 18:59:46 UTC, DLearner wrote:
 To me, it appears that there are really two (_entirely 
 separate_) concepts:

 A. Supporting the useful concept of variable length (but 
 otherwise entirely conventional) arrays;
 B. Supporting a language feature that acts as a window to an 
 array, through which that array can be manipulated.

 And currently these two concepts are combined.
Yes, this is correct.
 Suggestion: it would be clearer if the two concepts were 
 separated:
 1. Convert 'int[] VarArr;' so it produces a straightforward 
 _value-type_ variable array, called 'VarArr';
 2. Implement a new concept 'int slice Window;' to produce an 
 object of type 'int slice', called 'Window'.
    'Window' is a 'slice' into an int array, not an array itself 
 or even a variable.

 Opinions?
IMO you have it the wrong way around. The built in `T[]` type should remain the way it is and be used if you want a slice (i.e., a "window"). If you want a dynamic array with value semantics, you should use a library-defined container type (e.g., `struct DynamicArray`). Also, to avoid confusion, we should probably go through the language spec and documentation and change it to say "slice" instead of "dynamic array" whenever it refers to a `T[]`.
Nov 29 2022
parent reply DLearner <bmqazwsx123 gmail.com> writes:
On Wednesday, 30 November 2022 at 02:29:03 UTC, Paul Backus wrote:

[...]
 If you want a dynamic array with value semantics, you should 
 use a library-defined container type (e.g., `struct 
 DynamicArray`).
I agree should not change existing meaning of ``` int[] A; ``` But why not allow a construct for value-type variable arrays like: ``` int[*] B; ``` Best regards
Dec 03 2022
parent Paul Backus <snarwin gmail.com> writes:
On Saturday, 3 December 2022 at 22:46:31 UTC, DLearner wrote:
 I agree should not change existing meaning of
 ```
 int[] A;
 ```

 But why not allow a construct for value-type variable arrays 
 like:
 ```
 int[*] B;
 ```
There's no reason to add more complexity to the language for this when the same result can already be achieved using existing language features.
Dec 03 2022
prev sibling parent reply Basile B. <b2.temp gmx.com> writes:
On Tuesday, 29 November 2022 at 18:59:46 UTC, DLearner wrote:
 To me, it appears that there are really two (_entirely 
 separate_) concepts:

 A. Supporting the useful concept of variable length (but 
 otherwise entirely conventional) arrays;
 B. Supporting a language feature that acts as a window to an 
 array, through which that array can be manipulated.

 And currently these two concepts are combined.

 Suggestion: it would be clearer if the two concepts were 
 separated:
 1. Convert 'int[] VarArr;' so it produces a straightforward 
 _value-type_ variable array, called 'VarArr';
 2. Implement a new concept 'int slice Window;' to produce an 
 object of type 'int slice', called 'Window'.
    'Window' is a 'slice' into an int array, not an array itself 
 or even a variable.

 Opinions?
I have implemented that in [styx](https://gitlab.com/styx-lang/styx). 1. You have the type for dynamic arrays, called TypeRcArray, syntax is `Type[+]` 2. You have the type for slices (what you describe as a window), syntax is `Type[]` but it is mostly obtained using expressions, e.g `mySlice = myRcArray[lo .. hi]` or `myStaticArray[lo .. hi]` or `myPointer[lo .. hi]`. This sounded like a good idea but it [has appeared very quickly](https://styx-lang.gitlab.io/styx/type.html#noteonlifetime) that slices are not so useful, especially when management is based on reference counting because then slices requires a form of management too. Here is why: Main caracteristics of a slice are - they cannot modify the identity of their sources. The identity of the source is what makes the integrity of a dynamic array, what makes their references countable. So it is the content pointer and the length. In consequence you cannot change the length of the source, you can only reduce the view. You can change the elements in the view. - the length and the pointer are cached as a value on the stack while for a dynamic array this is stored before that data, on the heap. Problems start happening when you escape a slice ```d struct S { var s32[] memberSlice; } function f(var S s): auto { var s32[+] a = (new s32[+])(2); // problem 1 : `s` lifetime > `a` lifetime s = (a[]).tupleof; // note: tuples are used in pace of struct literals // problem 2 return a[1 .. $]; // a is automatically decref'd on return // so the caller pulls a dead heap block. } ``` Essentially slices are only useful to be consumed locally, typically ```d while mySlice.length do { slice = slice[1..$]; } ``` And that's it. So at first glance slices are some cool, simplified, functionally stripped down arrays but they introduce new problems, at least when management of dynamic arrays is based on reference counting. Those new problems can only be solved using lifetime analysis (so a compile-time check... still better than runtime ref counting however).
Nov 29 2022
next sibling parent Basile.B <b2.temp gmx.com> writes:
On Wednesday, 30 November 2022 at 03:04:47 UTC, Basile B. wrote:
 Essentially slices are only useful to be consumed locally, 
 typically

 ```d
 while mySlice.length do
 {
    slice = slice[1..$];
 }
 ```
sorry I cant force push, it was obviously meant to be written as ```d while mySlice.length do { mySlice = mySlice[1..$]; } ```
Nov 30 2022
prev sibling parent reply Salih Dincer <salihdb hotmail.com> writes:
On Wednesday, 30 November 2022 at 03:04:47 UTC, Basile B. wrote:
 I have implemented that in 
 [styx](https://gitlab.com/styx-lang/styx).

 1. You have the type for dynamic arrays, called TypeRcArray, 
 syntax is  `Type[+]`
 2. You have the type for slices (what you describe as a 
 window), syntax is `Type[]`
 but it is mostly obtained using expressions, e.g `mySlice = 
 myRcArray[lo .. hi]` or
 `myStaticArray[lo .. hi]` or `myPointer[lo .. hi]`.

 This sounded like a good idea but it [has appeared very 
 quickly](https://styx-lang.gitlab.io/styx/type.html#noteonlifetime) that
slices are not so useful...
Recently DIP1044 was published about enum and although we can use `with()` instead we waste time unnecessarily... I've never tried it, but the feature shown below is not available in D! Why not such good things? ```d enum Foo { a0 = 1, a1 } enum Bar: Foo { a2 // 3 } ``` Thanks... SDB 79
Dec 04 2022
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 4 December 2022 at 09:54:53 UTC, Salih Dincer wrote:
 Recently DIP1044 was published about enum and although we can 
 use `with()` instead we waste time unnecessarily...
With cannot be used in calls. Hence when calling a function you need to either spell out the enum type or wrap the call in a with or multiple withs.
 I've never tried it, but the feature shown below is not 
 available in D! Why not such good things?

 ```d
 enum Foo
 {
      a0 = 1,
      a1
 }

 enum Bar: Foo
 {
     a2 // 3
 }
 ```

 Thanks...

 SDB 79
This is easy to add and costs almost no complexity
Dec 05 2022