www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Detecting ElementType of OutputRange

reply Vijay Nayar <madric gmail.com> writes:
I was working on a project where it dealt with output ranges, but 
these ranges would ultimately sink into a source that would be 
inefficient if every single `.put(T)` call was made one at a time.

Naturally, I could make a custom OutputRange for just this 
resource, but I also got the idea that I could make a generalized 
`BufferedOutputRange` that would save individual `.put(T)` calls 
into memory until a threshold is reached, and then make one bulk 
call to the output stream it wraps with a single `.put(T[])` call.

While working on the template for this buffering OutputRange, I 
originally used `ElementType` on the output range I was given, 
hoping to detect what type of `.put(T)` is permitted. However, I 
found out that `ElementType` only works for input ranges as you 
can see here: 
https://github.com/dlang/phobos/blob/6bf43144dbe956cfc16c00f0bff7a264fa62408e/std/range/primitives.d#L1265

Trying to find a workaround, I ultimately created this, and  my 
question is, is using such a template a good idea or a terrible 
idea? Is it safe to assume that ranges should have a put method 
that may take arrays that I can detect? Should I give up on the 
idea of detecting the OutputRange type, and instead require the 
programmer to explicitly declare the output type for fear of them 
using a range that doesn't take arrays?

Here is the source of what I was thinking of, let me know your 
thoughts:

``` d
import std.range;
import std.traits;

// A specialization of std.range.ElementType which also considers 
output ranges.
template ElementType(R)
if  (is(typeof(R.put) == function))  // Avoid conflicts with 
std.range.ElementType.
{
   // Static foreach generates code, it is not a true loop.
   static foreach (t; __traits(getOverloads, R, "put")) {
     pragma(msg, "Found put method, params=", 
Parameters!(t).length);
     // Because all code gets generated, we use a 'done' alias to
     // tell us when to stop.
     static if (!is(done)) {
       // Attempts to save Parameters!(t) into a variable fail, so 
it is repeated.
       static if (Parameters!(t).length == 1 && 
is(Parameters!(t)[0] T : T[])) {
         pragma(msg, "put for array found");
         // Setting the name of the template replaces calls
         // to ElementType!(...) with T.
         alias ElementType = T;
         alias done = bool;
       } else static if (Parameters!(t).length == 1 && 
is(Parameters!(t)[0] T)) {
         pragma(msg, "put for single found");
         alias ElementType = T;
         alias done = bool;
       }
     }
   }
   static if (!is(done)) {
     alias ElementType = void;
   }
}

unittest {
   // Works for simple 1-element puts for structs.
   struct Ham0(T) {
     void put(T d) {}
   }
   assert(is(ElementType!(Ham0!float) == float));

   // Works for classes too, which have array-based puts.
   class Ham1(T) {
     void put(T[] d) {}
   }
   assert(is(ElementType!(Ham1!float) == float));

   // Distracting functions are ignored, and if single & array
   // puts are supported, the element type is still correct.
   struct Ham2(T) {
     void put() {}
     void put(float f, T[] d) {}
     void put(T[] d) {}
     void put(T d) {}
   }
   assert(is(ElementType!(Ham2!int) == int));
}
```
Feb 26 2022
parent reply Stanislav Blinov <stanislav.blinov gmail.com> writes:
https://dlang.org/phobos/std_range_primitives.html#isOutputRange
Feb 26 2022
parent reply Vijay Nayar <madric gmail.com> writes:
On Saturday, 26 February 2022 at 11:44:35 UTC, Stanislav Blinov 
wrote:
 https://dlang.org/phobos/std_range_primitives.html#isOutputRange
This method requires the caller to explicitly declare the output range element type, which I was hoping to have to avoid, if it can be detected using reasonable assumptions.
Feb 26 2022
parent reply Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Saturday, 26 February 2022 at 12:26:21 UTC, Vijay Nayar wrote:
 On Saturday, 26 February 2022 at 11:44:35 UTC, Stanislav Blinov 
 wrote:
 https://dlang.org/phobos/std_range_primitives.html#isOutputRange
This method requires the caller to explicitly declare the output range element type, which I was hoping to have to avoid, if it can be detected using reasonable assumptions.
Considering that `put` is quite typically implemented as a template, I don't think that would be possible in general. The question is, do you really need that? Your `BufferedOutputRange` can test the underlying range using `isOutputRange` in its own implementation of `put`, where the type of element is known, i.e. to test whether it can bulk-write a slice (or a range of) elements or has to make per-element calls to `put`.
Feb 26 2022
parent Vijay Nayar <madric gmail.com> writes:
On Saturday, 26 February 2022 at 12:39:51 UTC, Stanislav Blinov 
wrote:
 Considering that `put` is quite typically implemented as a 
 template, I don't think that would be possible in general.
That is what I found as well, for example, the implementation of `put` from `Appender` and `RefAppender`, thus the algorithm fails to detect the `put` methods.
 The question is, do you really need that? Your 
 `BufferedOutputRange` can test the underlying range using 
 `isOutputRange` in its own implementation of `put`, where the 
 type of element is known, i.e. to test whether it can 
 bulk-write a slice (or a range of) elements or has to make 
 per-element calls to `put`.
This is exactly what I was doing, having the user pass in the element type themselves, and then the BufferedOutputRange check for both `isOutputRange!(ORangeT, ElemT)` and `is(typeof(ORangeT.init.put([ ElemT.init ])))`. In short, `isOutputRange` has many ways to be satisfied, (https://dlang.org/phobos/std_range_primitives.html#.put), but I want to restrict things further to only output streams that can do a bulk put. Thanks for you advice. While automatic detection would be a nice convenience, it's not a deal breaker and it's also not unreasonable to expect the caller to know the element type they are inserting either.
Feb 26 2022