www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - The Phobos Put

reply Salih Dincer <salihdb hotmail.com> writes:
Why does my `put` work but the Phobos `put` doesn't work with a 
slice?


 onlineapp.d(11): Error: none of the overloads of template 
 `std.range.primitives.put` are callable using argument types 
 `!()(int[], int[])`


/dlang/dmd/linux/bin64/../../src/phobos/std/range/primitives.d(386):       
Candidate is: `put(R, E)(ref R r, E e)`

```d
void main()
{
     import std.range : phobos_put = put;
   
     enum testLen = 4;
     auto a = new int[testLen];
     auto b = new int[testLen];

     auto slice = a[1..$-1];
     slice.phobos_put([2, 3]);
     //a[1..$-1].phobos_put([2, 3]);
     b[1..$-1].put([2, 3]);

     import std.conv : text;
     assert(a == b, text(a));
}

void put(R)(R[] range, R[] source)
{
   assert(source.length <= range.length);
   foreach(element; source)
   {
     range[0] = element;  // range.front()
     range = range[1..$]; // range popFront()
   }
}
```

SDB 79
Mar 29 2023
parent reply Dennis <dkorpel gmail.com> writes:
On Wednesday, 29 March 2023 at 11:10:42 UTC, Salih Dincer wrote:

 Why does my `put` work but the Phobos `put` doesn't work with a 
 slice?


Your `put` doesn't take `range` by `ref`, so it allows you to 
pass an rvalue. Consequently, it doesn't advance the range from 
the callers perspective.
Mar 29 2023
parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 3/29/23 04:48, Dennis wrote:

 On Wednesday, 29 March 2023 at 11:10:42 UTC, Salih Dincer wrote:

 Why does my `put` work but the Phobos `put` doesn't work with a slice?


 Your `put` doesn't take `range` by `ref`, so it allows you to pass an
 rvalue. Consequently, it doesn't advance the range from the callers
 perspective.


And that 'ref' is necessary because not every OutputRange can be sliced 
for further calls to put(). Salih does not have that problem because he 
is working with slices, which are (usually) trivially slicable for the 
next portion to be passed to put().

On the other hand, Phobos's put() works with any OutputRange so it has 
to take a 'ref' to advance to know where it is left off. This behavior 
makes its use with slices weird but sometimes such is life. :)

Ali
Mar 29 2023
parent reply Salih Dincer <salihdb hotmail.com> writes:
On Wednesday, 29 March 2023 at 15:01:27 UTC, Ali Çehreli wrote:

 On 3/29/23 04:48, Dennis wrote:
 On the other hand, Phobos's put() works with any OutputRange so 
 it has to take a 'ref' to advance to know where it is left off. 
 This behavior makes its use with slices weird but sometimes 
 such is life. :)


Would not adding a prototype without a ref cause ambiguity? In 
this way, it could also be used directly with slices. For example:

```d
auto put(R)(R[] range, R[] source)
   => putImpl(range, source);

auto put(R)(ref R[] range, R[] source)
   => putImpl(range, source);

void putImpl(R)(ref R[] range, R[] source)
{
   assert(source.length <= range.length);
   foreach(element; source)
   {
     range[0] = element;  // range.front()
     range = range[1..$]; // range.popFront()
   }
}

void main() {
   enum data = [1, 0, 0, 4];
   auto arr = data;
   auto slice = arr[1..$-1];

   slice.put([2]);
   assert(arr == [1, 2, 0, 4]);

   slice.put([3]);
   assert(arr == [1, 2, 3, 4]);

   arr[1..$-1].put([0, 0]);
   assert(arr == data);
}
```

SDB 79
Mar 29 2023
parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 3/29/23 09:27, Salih Dincer wrote:


 In this way,
 it could also be used directly with slices. For example:



 auto put(R)(R[] range, R[] source)
    => putImpl(range, source);


That's for rvalues.


 auto put(R)(ref R[] range, R[] source)
    => putImpl(range, source);


That's for lvalues.

If you are proposing keeping the current ref-taking Phobos put() as 
well, then the following call would be ambiguous:

   slice.put([2]);  // Compilation ERROR

That can be remedied by using template constraints for 'slices' vs. 
other output ranges. But that would be against the idea of "D slices are 
most capable ranges". What I mean is, I should be able to write any 
range algorithm and pass a slice to it and it should work. If we go with 
your proposal, then we would have to check for that case for some 
algorithms like put().

Further, I think the user of put() in templates would have to check 
whether they are dealing with a slice or not:

void foo(R)(R range) {
   range.put(/* ... */);

   // If we go with your proposal, whether 'range' changed depends
   // on whether R is a slice or not. Do we have to check with
   // 'static if' in such range algorithms?
}

Note that I am not defending the behavior of put(). I am just trying to 
explain why it is so.

Ali
Mar 29 2023
parent reply ag0aep6g <anonymous example.com> writes:
On Wednesday, 29 March 2023 at 16:44:31 UTC, Ali Çehreli wrote:

 On 3/29/23 09:27, Salih Dincer wrote:


 In this way,
 it could also be used directly with slices. For example:



 auto put(R)(R[] range, R[] source)
    => putImpl(range, source);


 That's for rvalues.


 auto put(R)(ref R[] range, R[] source)
    => putImpl(range, source);


 That's for lvalues.

 If you are proposing keeping the current ref-taking Phobos 
 put() as well, then the following call would be ambiguous:

   slice.put([2]);  // Compilation ERROR


As far as I understand, you're saying that we cannot overload on 
`ref`. But we can. Salih's code demonstrates just that.

void f(ref int x) {}
void f(int x) {}
void main() { int x; f(x); f(42); } /* no errors */
Mar 29 2023
parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 3/29/23 12:21, ag0aep6g wrote:


 As far as I understand, you're saying that we cannot overload on `ref`.
 But we can. Salih's code demonstrates just that.

 void f(ref int x) {}
 void f(int x) {}
 void main() { int x; f(x); f(42); } /* no errors */


I thought Salih was proposing two more overloads to the existing put(). 
When I copy the existing put(), which takes 'ref R', not R[], then the 
code does not compile:

auto put(R)(R[] range, R[] source)
   => putImpl(range, source);

auto put(R)(ref R[] range, R[] source)
   => putImpl(range, source);

void putImpl(R)(ref R[] range, R[] source)
{
   assert(source.length <= range.length);
   foreach(element; source)
   {
     range[0] = element;  // range.front()
     range = range[1..$]; // range.popFront()
   }
}

void put(R, E)(ref R r, E e)
{
   // This is from Phobos <-------
}

void main() {
   enum data = [1, 0, 0, 4];
   auto arr = data;
   auto slice = arr[1..$-1];

   slice.put([2]);              // <-- ERROR
   assert(arr == [1, 2, 0, 4]);

   slice.put([3]);
   assert(arr == [1, 2, 3, 4]);

   arr[1..$-1].put([0, 0]);
   assert(arr == data);
}

Ali
Mar 29 2023
next sibling parent reply ag0aep6g <anonymous example.com> writes:
On Wednesday, 29 March 2023 at 19:49:47 UTC, Ali Çehreli wrote:

 I thought Salih was proposing two more overloads to the 
 existing put(). When I copy the existing put(), which takes 
 'ref R', not R[], then the code does not compile:

 auto put(R)(R[] range, R[] source)
   => putImpl(range, source);

 auto put(R)(ref R[] range, R[] source)
   => putImpl(range, source);


[...]

 void put(R, E)(ref R r, E e)
 {
   // This is from Phobos <-------
 }


I understand Salih's ref `put` to be same as yours: a stand-in 
for the one that is already in Phobos. So he's proposing to add 
the non-ref one.

But regardless of Salih's exact intent, the broader point is: a 
non-ref overload could be added to Phobos. And that would enable 
`a[1..$-1].phobos_put([2, 3])`. Which is what he asked about 
originally.
Mar 29 2023
next sibling parent Salih Dincer <salihdb hotmail.com> writes:
On Wednesday, 29 March 2023 at 20:29:24 UTC, ag0aep6g wrote:

 But regardless of Salih's exact intent, the broader point is: a 
 non-ref overload could be added to Phobos. And that would 
 enable `a[1..$-1].phobos_put([2, 3])`. Which is what he asked 
 about originally.


Yes, that was it, but even more. Both functions serve the same 
thing. Why don't we combine them?

SDB 79
Mar 29 2023
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 3/29/23 4:29 PM, ag0aep6g wrote:


 But regardless of Salih's exact intent, the broader point is: a non-ref 
 overload could be added to Phobos. And that would enable 
 `a[1..$-1].phobos_put([2, 3])`. Which is what he asked about originally.


I think the idea of requiring ref output ranges is that you can then let 
the range keep track of its output state.

An input range with lvalue elements is therefore an output range, but 
only if it's accepted via ref, since it has to be iterated as it goes. 
If you iterate it only internally, then it's either in an undetermined 
state when you exit `put`, or it is a forward range that was copied 
without using `save`.

It's not the greatest situation. I feel like we probably shouldn't have 
made lvalue input ranges be output ranges automatically.

-Steve
Mar 29 2023
next sibling parent reply Salih Dincer <salihdb hotmail.com> writes:
On Wednesday, 29 March 2023 at 20:50:04 UTC, Steven Schveighoffer 
wrote:

 On 3/29/23 4:29 PM, ag0aep6g wrote:


 But regardless of Salih's exact intent, the broader point is: 
 a non-ref overload could be added to Phobos. And that would 
 enable `a[1..$-1].phobos_put([2, 3])`. Which is what he asked 
 about originally.


 I think the idea of requiring ref output ranges is that you can 
 then let the range keep track of its output state.


So why not copying the range to a static array?  The error during 
compilation of the code is as follows:


 onlineapp.d(6): Error: none of the overloads of template 
 `std.algorithm.mutation.copy` are callable using argument types 
 `!()(int[], int[8])`


/dlang/dmd/linux/bin64/../../src/phobos/std/algorithm/mutation.d(367):       
Candidate is: `copy(SourceRange, TargetRange)(SourceRange source, TargetRange
target)`
   with `SourceRange = int[],
        TargetRange = int[8]`
   must satisfy the following constraint:
`       isOutputRange!(TargetRange, ElementType!SourceRange)`

```d
import std.algorithm.mutation : copy;
void main()
{
   int[8] buf;
   auto dig = [1, 2, 3, 4];
   auto rem = dig.copy(buf);
   assert(rem.length == 4);
}

```

Looks like 'copy' has the same overload issue.

SDB 79
Mar 29 2023
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 3/29/23 11:01 PM, Salih Dincer wrote:


 ```d
 import std.algorithm.mutation : copy;
 void main()
 {
    int[8] buf;
    auto dig = [1, 2, 3, 4];
    auto rem = dig.copy(buf);
    assert(rem.length == 4);
 }
 
 ```
 
 Looks like 'copy' has the same overload issue.


A static array is not a range of any kind. What would `popFront` do on a 
static array, since the length is part of the type?

But you can do `dig.copy(buf[])` since a dynamic array is.

-Steve
Mar 30 2023
parent Salih Dincer <salihdb hotmail.com> writes:
On Thursday, 30 March 2023 at 13:27:33 UTC, Steven Schveighoffer 
wrote:

 But you can do `dig.copy(buf[])` since a dynamic array is.



Apparently, we will not be able to get rid of the necessity of 
using slices. 😀 Neither with "copy" nor with "put"...

```d
import std.algorithm.mutation : copy;
import std.range.primitives : put;

void main()
{
   int[8] buf;// = new int[8];
   auto dig = [1, 2, 3, 4];

   //dig.copy(buf[2..$-2]);/*
   auto slice = buf[2..$-2];
   dig.copy(slice);
   
   assert(buf == [0, 0, 1, 2, 3, 4, 0, 0]);
   buf = 0;
   slice.put(dig);//*/
   assert(buf == [0, 0, 1, 2, 3, 4, 0, 0]);
}
```
SDB 79
Mar 30 2023
prev sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Wednesday, 29 March 2023 at 20:50:04 UTC, Steven Schveighoffer 
wrote:

 On 3/29/23 4:29 PM, ag0aep6g wrote:


 But regardless of Salih's exact intent, the broader point is: 
 a non-ref overload could be added to Phobos. And that would 
 enable `a[1..$-1].phobos_put([2, 3])`. Which is what he asked 
 about originally.


 I think the idea of requiring ref output ranges is that you can 
 then let the range keep track of its output state.

 An input range with lvalue elements is therefore an output 
 range, but only if it's accepted via ref, since it has to be 
 iterated as it goes. If you iterate it only internally, then 
 it's either in an undetermined state when you exit `put`, or it 
 is a forward range that was copied without using `save`.

 It's not the greatest situation. I feel like we probably 
 shouldn't have made lvalue input ranges be output ranges 
 automatically.


It should be fine to have both a `ref` and non-`ref` overload for 
`put`, though, right? If the non-`ref` overload is only called 
with rvalues, then it's fine to leave them in an undetermined 
state, because nothing can access them afterward anyway.
Mar 30 2023
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 3/30/23 11:44 AM, Paul Backus wrote:


 It should be fine to have both a `ref` and non-`ref` overload for `put`, 
 though, right? If the non-`ref` overload is only called with rvalues, 
 then it's fine to leave them in an undetermined state, because nothing 
 can access them afterward anyway.


There's a certain attempt in phobos in some places to try and ensure 
code that is going to confuse will not compile. I think this is one of 
those attempts.

Consider that if you pass a slice into `put`, then it returns nothing. 
There is no indication of what actually was written. It's essentially an 
inconclusive call, because the "result" is the output range itself. How 
many elements were written? You can't tell.

I'd argue that the way input ranges are used as output ranges today is 
extremely confusing. It makes sort of a logical sense, but the fact that 
you need to store your "original" range, and then do some length math to 
figure out what was written makes such code very awkward all around. The 
output is decipherable, but not obvious.

I stand by my assertion that probably lvalue input ranges should never 
have been treated as output ranges implicitly. They should have had to 
go through some sort of wrapper.

-Steve
Mar 30 2023
parent Paul Backus <snarwin gmail.com> writes:
On Friday, 31 March 2023 at 02:23:29 UTC, Steven Schveighoffer 
wrote:

 There's a certain attempt in phobos in some places to try and 
 ensure code that is going to confuse will not compile. I think 
 this is one of those attempts.

 Consider that if you pass a slice into `put`, then it returns 
 nothing. There is no indication of what actually was written. 
 It's essentially an inconclusive call, because the "result" is 
 the output range itself. How many elements were written? You 
 can't tell.


This is a general issue with the `put` interface—it has no 
standardized way to report failure or partial success to the 
caller. And the problem applies to both lvalue and rvalue output 
ranges. For example:

```d
void main()
{
     import std.stdio, std.range;

     auto sink = stdout.lockingTextWriter;
     put(sink, "hello");
}
```

How many characters were written? Nobody knows! Maybe you can 
find out with `File.tell`, if the file is seekable, but there's 
no guarantee it is. And if you're working in a generic context, 
where you don't know the specific type of output range you're 
dealing with? Forget it.


 I'd argue that the way input ranges are used as output ranges 
 today is extremely confusing. It makes sort of a logical sense, 
 but the fact that you need to store your "original" range, and 
 then do some length math to figure out what was written makes 
 such code very awkward all around. The output is decipherable, 
 but not obvious.

 I stand by my assertion that probably lvalue input ranges 
 should never have been treated as output ranges implicitly. 
 They should have had to go through some sort of wrapper.


IMO this is downstream of the issue with `put`. If `put` itself 
returned information about what was written, then there would be 
no need to try and recreate that information via side channels 
like this (or like `File.tell`).

I agree that there are ways we could change things to make the 
side channels easier to use, but ultimately that's treating the 
symptom, not the disease.
Mar 31 2023
prev sibling parent Salih Dincer <salihdb hotmail.com> writes:
On Wednesday, 29 March 2023 at 19:49:47 UTC, Ali Çehreli wrote:

 On 3/29/23 12:21, ag0aep6g wrote:


 As far as I understand, you're saying that we cannot overload


 on `ref`.

 But we can. Salih's code demonstrates just that.

 void f(ref int x) {}
 void f(int x) {}
 void main() { int x; f(x); f(42); } /* no errors */


 I thought Salih was proposing two more overloads to the 
 existing put(). When I copy the existing put(), which takes 
 'ref R', not R[], then the code does not compile:


Wait a minute, isn't `copy` actually a `put` as well? Forget 
about `ref` for a moment, please. Actually, logically, the 
parameters have been swapped between the two functions. Well, if 
we only had `copy` and we used `put` like `copy`, what is the 
need for `put`? Examples are below: :)

```d
//version = simple;/*
version = standart;//*/

version(simple) {
   auto put(R)(R[] range, R[] source)
     => copyImpl(source, range);

   auto copy(R)(R[] source, R[] range)
     => copyImpl(source, range);

   auto copyImpl(R)(R[] source, R[] range)
   {
     assert(source.length <= range.length);
     foreach(element; source)
     {
       range[0] = element;  // range.front()
       range = range[1..$]; // range.popFront()
     }
     return range;
   }

   void swap (ref int x, ref int y)
   {
     x = x ^ y;
     y = y ^ x;
     x = x ^ y;
   }
}

version(standart)
   import std.algorithm.mutation,
          std.range : put;

void main()
{
   // copy() example:
   enum len = 10;
   auto buf = new int[len]; // buffer
   auto dig = [7, 1, 2, 3]; // digits

   auto diff = len - dig.length;
   auto rem = copy(dig, buf);

   assert(buf[0..$ - diff] == [7, 1, 2, 3]);

   swap(buf[0], rem[0]);
   assert(rem == [7, 0, 0, 0, 0, 0]);

   // put() example 1:
   put(rem, [4, 5, 6, 7, 8, 9]);
   assert(buf == [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]);

   // put() example 2:
   enum data = [1, 0, 0, 4];
   auto arr = data;
   auto slice = arr[1..$-1];

   version(standart)
     put(slice, [2]);
   version(simple)
     slice = put(slice, [2]);
   assert(arr == [1, 2, 0, 4]);

   version(standart)
     put(slice, [3]);
   version(simple)
     slice = put(slice, [3]);
   assert(arr == [1, 2, 3, 4]);

   version(standart) {
     auto slc = arr[1..$-1];
     put(slc, [0, 0]);
   }
   version(simple)
     arr[1..$-1].put([0, 0]);
   assert(arr == data);
}
```
All you have to do is remove the // mark in the first line. The 
code compiles for me, what about you?

SDB 79
Mar 29 2023