www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - lodash like utility/algorithms library for D

reply aliak <something something.com> writes:
Hi,

I've been working for fun on a library [0] that is inspired by a 
library from the javascript world called lodash [1]. I basically 
liked the flexibility and thought I'd try and implement a few 
things as it was about the time I started learning D. It 
basically tried to do the same with algorithms as in lodash but 
adds the usage of Optional!T and Expect!(T, U) as ways to returns 
results.

The readme has some more info on how things were approached: 
https://github.com/aliak00/ddash

I've also been inspired by Scala, so stuff from there also got 
thrown in (e.g. Try). I still consider this library largely 
experimental, but it's turning out nice IMO.

Anyway, code:

import std.experimental.all;

void main() {
     import ddash:
         flatMap,
         try_,
         cond,
         compact,
         isFalsey,
         differenceBy;

     // This could throw.
     alias toInt = (string str) => str.to!int;

     // Call toInt using try_ and flatMap so that you only have
     // valid data points
     alias parseDataSet = (arr) => arr.flatMap!(try_!toInt);

     auto cosmicPoints = dataSets
         .map!parseDataSet
          // Get rid of data sets that are empty
         .compact!isFalsey
         // Convert them to CosmicDataPoint type
         .map!(arr => CosmicDataPoint(arr.sum))
         // Remove the known dead-end values by getting a 
difference set
         // based on the "value" member type
         // This can also be:
         // .difference!((a, b) => a.value == b.value)(...)
         // Or in the case of a D value type just:
         // .difference!(...)
         .differenceBy!"value"(knownDeadEndValues);

     assert(cosmicPoints.walkLength == 3);

     // Pattern match on CosmicDataPoint types and process 
accordingly
     alias process = cond!(
         CosmicDataPoint(29), processBlackMatter,
         CosmicDataPoint(30), processGravitationalWave,
         a => a.value > 30 && a.value < 100, processPromisingData,
         a => writeln("This data is useless: ", a),
     );

     cosmicPoints.each!process;
}

// Simulated data sets that result in CosmicDataPoint
immutable dataSets = [
     ["2", "3", "2"],            // Produces CosmicDataPoint(7)
     ["22", "~1", "7"],          // Produces CosmicDataPoint(29)
     ["!$", "88", "3"],          // Produces CosmicDataPoint(91)
     ["junk", "junk", "junk"],   // All junk data points
     ["99", "44"],               // Produces CosmicDataPoint(143)
];

// A cosmic data point
static struct CosmicDataPoint {
     int value;
}

// Simulate values that you know occur but are a dead end
auto knownDeadEndValues = [1, 7, 42].map!(a => 
CosmicDataPoint(a));

// Define some data processing functions
static void processBlackMatter(CosmicDataPoint) {
     writeln("processing black matter discovery");
}
static void processGravitationalWave(CosmicDataPoint) {
     writeln("processing gravitational wave");
}
static void processPromisingData(CosmicDataPoint) {
     writeln("processing data that's close");
}

Cheers,
- Ali

[0] https://ddash.dub.pm
[1] https://lodash.com/
Sep 28
next sibling parent aliak <something something.com> writes:
On Friday, 28 September 2018 at 14:02:48 UTC, aliak wrote:
 Hi,
 [...]
PS Docs: https://aliak00.github.io/ddash/ddash/algorithm.html
Sep 28
prev sibling next sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Friday, 28 September 2018 at 14:02:48 UTC, aliak wrote:
 Hi,

 [...]
Lots of good stuff here! I'm curious about your approach to `Expect`, since I've written a version of it myself. How useful have you found being able to use unexpected values of any type, as opposed to just exceptions?
Sep 28
parent reply aliak <something something.com> writes:
On Friday, 28 September 2018 at 17:33:04 UTC, Paul Backus wrote:
 On Friday, 28 September 2018 at 14:02:48 UTC, aliak wrote:
 Hi,

 [...]
Lots of good stuff here! I'm curious about your approach to `Expect`, since I've written a version of it myself. How useful have you found being able to use unexpected values of any type, as opposed to just exceptions?
Thanks! Still all rough around the edges. About Expect, I've found that being able to define a set of expected unexpected values is quite practical, and if they're all based on the class Exception then there're two problems. 1, it's a class so it comes with all those constraints. 2, there's no way to close the value domain over the unexpeceted (if that makes sense?). Also some functions (take the classic errono catastrophy in C) may want to return error codes as unexpected values that are ints, and at the same time have a valid value as an int as well. I.e. by allowing you to define the unexepcted you could for instance: enum JSONError { invalidKey, notString, notNumber } auto a = parse(jsonData); a.getAsString("key").match!( (string value) => // yay (JSONError error) => // small domain of what went wrong ); A bit contrived here, but it actually comes form production code (https://github.com/schibsted/account-sdk-ios/blob/master/Source/Core/JSON/JSONObject.swift). Different language of course, but in that repo there's a type called Result, which is basically Expect. But in swift you have something called protocols, which lets you contstrain template types (in a different way than isInputRange works). And there's a standard protocol called Error. So you can do: Result<T, JSONError> where JSONError is defined as: struct JSONError: Error {} // conforms to error protocol. There's actually a D DIP which could allow for similar behavior: https://github.com/rikkimax/DIPs/blob/master/DIPs/DIP1xxx-RC.md But for now since D does not have that. Another approach would be duck typying I guess. And make a isExceptionType trait in D that makes sure some functions are supported (i.e. msg, and that it can be constructed with __FILE__, __LINE__ ). I have not thought out completely how those semantics would work, just thinking out loud right now.
Sep 29
parent reply Paul Backus <snarwin gmail.com> writes:
On Saturday, 29 September 2018 at 12:40:14 UTC, aliak wrote:
 I.e. by allowing you to define the unexepcted you could for 
 instance:

 enum JSONError {
   invalidKey, notString, notNumber
 }

 auto a = parse(jsonData);

 a.getAsString("key").match!(
     (string value) => // yay
     (JSONError error) => // small domain of what went wrong
 );
I agree that this is useful, but why not just return a naked `SumType!(string, JSONError)` in that case? Is there some additional value added by the `Expect` wrapper that I'm not seeing?
Sep 29
parent reply aliak <something something.com> writes:
On Saturday, 29 September 2018 at 19:27:29 UTC, Paul Backus wrote:
 On Saturday, 29 September 2018 at 12:40:14 UTC, aliak wrote:
 I.e. by allowing you to define the unexepcted you could for 
 instance:

 enum JSONError {
   invalidKey, notString, notNumber
 }

 auto a = parse(jsonData);

 a.getAsString("key").match!(
     (string value) => // yay
     (JSONError error) => // small domain of what went wrong
 );
I agree that this is useful, but why not just return a naked `SumType!(string, JSONError)` in that case? Is there some additional value added by the `Expect` wrapper that I'm not seeing?
That's an option as well I guess. But then you'd need to rely on convention and you couldn't do SumType!(int, int) f(), and Expect also gives you some more purposeful APIs that makes the code's intent clear. Plus I'm considering range behavior as well. Could you also return a union voldermort type then instead of a SumType?
Sep 30
parent reply Paul Backus <snarwin gmail.com> writes:
On Sunday, 30 September 2018 at 22:17:05 UTC, aliak wrote:
 On Saturday, 29 September 2018 at 19:27:29 UTC, Paul Backus 
 wrote:
 I agree that this is useful, but why not just return a naked 
 `SumType!(string, JSONError)` in that case? Is there some 
 additional value added by the `Expect` wrapper that I'm not 
 seeing?
That's an option as well I guess. But then you'd need to rely on convention and you couldn't do SumType!(int, int) f(), and Expect also gives you some more purposeful APIs that makes the code's intent clear. Plus I'm considering range behavior as well.
Is being able to write `Expect!(int, int)` actually desirable, though? It seems to me like being forced to write something like `SumType!(int, ErrorCode)` to distinguish the two cases would be a good thing, even if ErrorCode itself is just a renamed int (e.g., `struct ErrorCode { int code; alias code this; }`). I guess you could argue that `return typeof(return).unexpected(...)` is better than `return typeof(return)(ErrorCode(...))`, which is what you'd get with SumType, but they look equally ugly to me. What's really needed to make that look nice is implicit constructors. Treating an Expect as a range basically turns it into an Optional, in the sense that it collapses any error information it contains down to the boolean of empty vs not-empty. In fact, probably the easiest way to add range behavior to Expect would be to add a method that returns an Optional containing the expected value, since Optional already has range behavior.
 Could you also return a union voldermort type then instead of a 
 SumType?
Raw unions in D are horrifically unsafe, so I wouldn't recommend it. If you want a voldemort SumType, you can get one like this: auto f() { struct Result { SumType(T, U) data; alias data this; } return Result(...); }
Sep 30
parent aliak <something something.com> writes:
On Monday, 1 October 2018 at 00:51:24 UTC, Paul Backus wrote:
 On Sunday, 30 September 2018 at 22:17:05 UTC, aliak wrote:
 On Saturday, 29 September 2018 at 19:27:29 UTC, Paul Backus 
 wrote:
 I agree that this is useful, but why not just return a naked 
 `SumType!(string, JSONError)` in that case? Is there some 
 additional value added by the `Expect` wrapper that I'm not 
 seeing?
That's an option as well I guess. But then you'd need to rely on convention and you couldn't do SumType!(int, int) f(), and Expect also gives you some more purposeful APIs that makes the code's intent clear. Plus I'm considering range behavior as well.
Is being able to write `Expect!(int, int)` actually desirable, though? It seems to me like being forced to write something like `SumType!(int, ErrorCode)` to distinguish the two cases would be a good thing, even if ErrorCode itself is just a renamed int (e.g., `struct ErrorCode { int code; alias code this; }`).
Hard to say, I would try to avoid it if possible, but why should it not be allowed if someone wants it? For now it feels like an opinionated restriction that I think is better left out of generic code when possible - at least for now. If it turns out otherwise I'll change it - this is all still quite experimental in me head. Using SumType to denote success and failure would be more of a convention though and would make for more "huh?" moments for readability/maintainability, IMO. I like typing intents. And being able to add an "alias this" in Expect for e.g. might be nice. Or if I want to add a "make match has exactly two handlers" so that you have to handle both cases would also be a plus.
 I guess you could argue that `return 
 typeof(return).unexpected(...)` is better than `return 
 typeof(return)(ErrorCode(...))`, which is what you'd get with 
 SumType, but they look equally ugly to me. What's really needed 
 to make that look nice is implicit constructors.
*nods*
 Treating an Expect as a range basically turns it into an 
 Optional, in the sense that it collapses any error information 
 it contains down to the boolean of empty vs not-empty. In fact, 
 probably the easiest way to add range behavior to Expect would 
 be to add a method that returns an Optional containing the 
 expected value, since Optional already has range behavior.
Good point. Agreed!
 Could you also return a union voldermort type then instead of 
 a SumType?
Raw unions in D are horrifically unsafe, so I wouldn't recommend it. If you want a voldemort SumType, you can get one like this: auto f() { struct Result { SumType(T, U) data; alias data this; } return Result(...); }
Oct 01
prev sibling parent reply Robby Marki <giusoin bobmail.info> writes:
On Friday, 28 September 2018 at 14:02:48 UTC, aliak wrote:
 Hi,

 I've been working for fun on a library [0] that is inspired by 
 a library from the javascript world called lodash [1]. I 
 basically liked the flexibility and thought I'd try and 
 implement a few things as it was about the time I started 
 learning D. It basically tried to do the same with algorithms 
 as in lodash but adds the usage of Optional!T and Expect!(T, U) 
 as ways to returns results.

 [...]
In this example https://aliak00.github.io/ddash/ddash/functional/try_.html where does the match function come from? I get this error when trying to compile: onlineapp.d(16): Error: no property match for type int /dlang/dmd/linux/bin64/../../src/phobos/std/algorithm/iteration.d(500): instantiated from here: MapResult!(__lambda1, int[]) onlineapp.d(15): instantiated from here: map!(int[])
Sep 28
parent reply aliak <something something.com> writes:
On Saturday, 29 September 2018 at 01:40:34 UTC, Robby Marki wrote:
 On Friday, 28 September 2018 at 14:02:48 UTC, aliak wrote:
 Hi,

 I've been working for fun on a library [0] that is inspired by 
 a library from the javascript world called lodash [1]. I 
 basically liked the flexibility and thought I'd try and 
 implement a few things as it was about the time I started 
 learning D. It basically tried to do the same with algorithms 
 as in lodash but adds the usage of Optional!T and Expect!(T, 
 U) as ways to returns results.

 [...]
In this example https://aliak00.github.io/ddash/ddash/functional/try_.html where does the match function come from? I get this error when trying to compile: onlineapp.d(16): Error: no property match for type int /dlang/dmd/linux/bin64/../../src/phobos/std/algorithm/iteration.d(500): instantiated from here: MapResult!(__lambda1, int[]) onlineapp.d(15): instantiated from here: map!(int[])
Haha ok I'm a bit stumped right now. That example is wrong, so you get the correct error, but it actually compiles on me machine right now. I changed Try.front to return the expected return value of a tryable function. And in that example that would clearly be an int. So there is no "match" on type int is what I should be seeing as well. But for some reason that I'm going to have to dig in to, it's compiling a running in the actual project code O_o.
Sep 29
parent aliak <something something.com> writes:
On Saturday, 29 September 2018 at 12:44:38 UTC, aliak wrote:
 On Saturday, 29 September 2018 at 01:40:34 UTC, Robby Marki 
 wrote:
 On Friday, 28 September 2018 at 14:02:48 UTC, aliak wrote:
 [...]
In this example https://aliak00.github.io/ddash/ddash/functional/try_.html where does the match function come from? I get this error when trying to compile: onlineapp.d(16): Error: no property match for type int /dlang/dmd/linux/bin64/../../src/phobos/std/algorithm/iteration.d(500): instantiated from here: MapResult!(__lambda1, int[]) onlineapp.d(15): instantiated from here: map!(int[])
Haha ok I'm a bit stumped right now. That example is wrong, so you get the correct error, but it actually compiles on me machine right now. I changed Try.front to return the expected return value of a tryable function. And in that example that would clearly be an int. So there is no "match" on type int is what I should be seeing as well. But for some reason that I'm going to have to dig in to, it's compiling a running in the actual project code O_o.
Sorry no, it is working as expected. I see that you have an error on onlineapp.d(16) ... are you using run.dlang.io? I just tried a new project and it compiles fine. Match is part of the Try type. Cheers, - Ali
Sep 29