www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Storing a lambda alongside type-erased data

reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
I am sure nothing is new here and I may have thought of this before but 
it was a revelation today. :)

I've been trying to come up with a way of storing arbitrary number of 
objects of arbitrary types, which means I would be using a ubyte array.

But then how do I use the data later without needing to remember its 
type perhaps with TypeInfo? I first considered registering data up front 
with something like

   SumType!(Tuple(int, string, double),
            Tuple(S, char))

but I couldn't make it work and it wasn't useful having to register 
valid sets of data like that.

I looked at how std.variant.VariantN prints the correct type and failed 
to understand the magic there. :(

Then I came up with storing a lambda that is created when the exact type 
is known. The following simple variant can carry arbitrary set of data 
because the data is provided as sequence template parameters (aka variadic).

Note that set() member function is  nogc because the data is placed in 
an existing ubyte array. (That was the main motivation for this design.) 
(I left notes about 3 bugs in there, which can all be taken care of.)

Then the stored lambda is used to print the data. (I am sure the lambda 
can do other things.) I chose 'function' in order to be  nogc. When not 
required, it could be a 'delegate' as well.

import std; // Sorry :(

struct V_(size_t size)
{
     // This is where the data will reside; we have no idea on
     // what actual types of data will be used.
     ubyte[size] mem;

     // This is the lambda that remembers how to use the data
     // (printing to an output sink in this case.)
     void function(void delegate(in char[]), const(ubyte)*) dataToStr;

     // We can set any data into our data buffer
     void set(Args...)(Args args)  nogc nothrow pure {
         // Thank you, Tuple! :)
         alias Data = Tuple!Args;

         // Place the tuple of arguments
         //
         // BUG 1: Must consider alignment of Data
         // BUG 2: Must check that size is sufficient
         // BUG 3: The destructor of old data should be run
         //        (optionally?)
         emplace(cast(Data*)(mem.ptr), Data(args));

         // This is the interesting bit: Storing the lambda that
         // knows how to print this type.
         dataToStr = (sink, ptr) {
             // Cast back to the actual type. We know the type here.
             auto d = cast(Data*)(ptr);
             static foreach (i; 0 .. args.length) {
                 if (i != 0) {
                     sink(", ");
                 }
                 sink((*d)[i].to!string);
             }
         };
     }

     void toString(scope void delegate(in char[]) sink) const {
         dataToStr(sink, mem.ptr);
     }
}

// A convenience function to avoid needing to specify the
// template parameter. (The syntax is noisy otherwise.)
auto V(size_t size = 1024)() {
     return V_!size();
}

void main() {
     // Start with an empty variant
     auto v = V();

     // Store some data in it
     v.set(42, "hello", 2.5);
     writeln(v);

     // Now set different types of data
     struct S {
         int i;
     }
     v.set(S(7), 'a');
     writeln(v);
}

Ali
Sep 07 2022
parent reply Paul Backus <snarwin gmail.com> writes:
On Thursday, 8 September 2022 at 03:18:08 UTC, Ali Çehreli wrote:
 I looked at how std.variant.VariantN prints the correct type 
 and failed to understand the magic there. :(

 Then I came up with storing a lambda that is created when the 
 exact type is known. The following simple variant can carry 
 arbitrary set of data because the data is provided as sequence 
 template parameters (aka variadic).
This is actually pretty much exactly what VariantN does, except instead of storing a pointer to a lambda, it stores a pointer to an instance of a template function. The member variable `fptr` [1] is the equivalent of your `dataToStr`. It stores a pointer to an instance of the `handler` template [2]. Whenever a new value is assigned to the VariantN, `fptr` is updated to point to the template instance corresponding to the new value's type [3]. [1] https://github.com/dlang/phobos/blob/v2.100.1/std/variant.d#L217-L218 [2] https://github.com/dlang/phobos/blob/v2.100.1/std/variant.d#L260-L645 [3] https://github.com/dlang/phobos/blob/v2.100.1/std/variant.d#L731
Sep 08 2022
next sibling parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 9/8/22 08:02, Paul Backus wrote:

 This is actually pretty much exactly what VariantN does
Great information, thanks! I am slowly getting up there. :) Ali
Sep 08 2022
prev sibling parent apz28 <home home.com> writes:
On Thursday, 8 September 2022 at 15:02:13 UTC, Paul Backus wrote:
 On Thursday, 8 September 2022 at 03:18:08 UTC, Ali Çehreli 
 wrote:
 I looked at how std.variant.VariantN prints the correct type 
 and failed to understand the magic there. :(

 Then I came up with storing a lambda that is created when the 
 exact type is known. The following simple variant can carry 
 arbitrary set of data because the data is provided as sequence 
 template parameters (aka variadic).
This is actually pretty much exactly what VariantN does, except instead of storing a pointer to a lambda, it stores a pointer to an instance of a template function. The member variable `fptr` [1] is the equivalent of your `dataToStr`. It stores a pointer to an instance of the `handler` template [2]. Whenever a new value is assigned to the VariantN, `fptr` is updated to point to the template instance corresponding to the new value's type [3]. [1] https://github.com/dlang/phobos/blob/v2.100.1/std/variant.d#L217-L218 [2] https://github.com/dlang/phobos/blob/v2.100.1/std/variant.d#L260-L645 [3] https://github.com/dlang/phobos/blob/v2.100.1/std/variant.d#L731
My implement is similar but a pointer to template struct with various functions. The advantage is that you can add various attributes to those functions https://github.com/apz28/dlang/blob/main/source/pham/utl/utl_variant.d#L904 https://github.com/apz28/dlang/blob/main/source/pham/utl/utl_variant.d#L1394
Sep 08 2022