digitalmars.D.learn - How to implement filterMap
- Christian =?UTF-8?B?S8O2c3RsaW4=?= (61/61) Dec 29 2023 Is there a way to implement filterMap (meaning do mapping of a
- Siarhei Siamashka (8/11) Dec 29 2023 It's probably not a good idea to do this in general. Expecting a
- Christian =?UTF-8?B?S8O2c3RsaW4=?= (9/20) Dec 30 2023 Thanks for the feedback.
- Siarhei Siamashka (42/61) Dec 31 2023 I still think that a much better design is to have a try/catch
- Siarhei Siamashka (48/55) Dec 31 2023 And for comparison, here's a Ruby/Crystal example too:
- Alexandru Ermicioi (4/12) Dec 31 2023 `CheckedInt` should be another solution for overflows, if you
- Siarhei Siamashka (10/12) Jan 27 CheckedInt is worse than nothing. It exists to give an illusion
- Alexandru Ermicioi (10/13) Dec 30 2023 Perhaps try map to null cases that exception is thrown then
- Christian =?UTF-8?B?S8O2c3RsaW4=?= (9/22) Dec 30 2023 asNullForException could be nicely implemented with lazy
Is there a way to implement filterMap (meaning do mapping of a
range, but if something happens during the map, leave this
element out of the resulting range).
I have two solutions (one is with evaluating the mapping function
several times), and one tries to store the result for the next
front call like this.
Both do not seem very clean for all possible types. Here the
version that tries to cache the mapping results.
```d
template filterMap(mapping...) if (mapping.length == 1)
{
auto filterMap(Range)(Range range)
{
import std.range : ElementType, empty;
import std.functional : unaryFun;
alias RangeElement = ElementType!Range;
alias mappingFunction = unaryFun!mapping;
typeof(mappingFunction(RangeElement.init)) result;
void findNext() {
while (!range.empty)
{
try
{
result = mappingFunction(range.front);
break;
}
catch (Exception e)
{
range.popFront;
}
}
}
findNext();
struct FilterMap
{
bool empty()
{
return range.empty;
}
auto ref front()
{
return result;
}
void popFront()
{
range.popFront;
findNext();
}
}
return FilterMap();
}
}
("filterMap") unittest {
import std.conv : to;
import std.array : array;
["1", "2", "abc", ""].filterMap!(s => s.to!int *
2).array.should == [2, 4];
}
```
Kind regards,
Christian
Dec 29 2023
On Friday, 29 December 2023 at 23:10:47 UTC, Christian Köstlin wrote:Is there a way to implement filterMap (meaning do mapping of a range, but if something happens during the map, leave this element out of the resulting range).It's probably not a good idea to do this in general. Expecting a lot of exceptions handling happening during normal program execution (if you want to filter out roughly half of the input array) will result in a major performance loss. Exceptions are best left to just do error handling on a very rarely used code path for troubleshooting purposes.
Dec 29 2023
On Saturday, 30 December 2023 at 01:22:31 UTC, Siarhei Siamashka wrote:On Friday, 29 December 2023 at 23:10:47 UTC, Christian Köstlin wrote:Thanks for the feedback. This might be true, but in my example I would parse the input always with conv.to, so I would need to handle the exception(s). The "original" https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.filter_map works with the Option(Some/None) ... Kind regards, ChristianIs there a way to implement filterMap (meaning do mapping of a range, but if something happens during the map, leave this element out of the resulting range).It's probably not a good idea to do this in general. Expecting a lot of exceptions handling happening during normal program execution (if you want to filter out roughly half of the input array) will result in a major performance loss. Exceptions are best left to just do error handling on a very rarely used code path for troubleshooting purposes.
Dec 30 2023
On Saturday, 30 December 2023 at 13:25:00 UTC, Christian Köstlin wrote:On Saturday, 30 December 2023 at 01:22:31 UTC, Siarhei Siamashka wrote:I still think that a much better design is to have a try/catch block much higher in the call stack and print a "malformed input" error message to the user. Or handle the error in some other way (for example, allow the user to try again with a different input data).On Friday, 29 December 2023 at 23:10:47 UTC, Christian Köstlin wrote:Thanks for the feedback. This might be true, but in my example I would parse the input always with conv.to, so I would need to handle the exception(s).Is there a way to implement filterMap (meaning do mapping of a range, but if something happens during the map, leave this element out of the resulting range).It's probably not a good idea to do this in general. Expecting a lot of exceptions handling happening during normal program execution (if you want to filter out roughly half of the input array) will result in a major performance loss. Exceptions are best left to just do error handling on a very rarely used code path for troubleshooting purposes.The "original" https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.filter_map works with the Option(Some/None) ...Here's an example with `Nullable`, inspired by the earlier comment from Alexandru Ermicioi: ```D import std; auto toNullableInt(string s) { try { return s.to!int.nullable; } catch (Exception e) {} Nullable!int ret; return ret; } void main() { // prints [2, 4] auto a = ["1", "2", "abc", ""]; a.map!toNullableInt.filter!"!a.isNull".map!"a.get * 2".writeln; // prints [2, 4, 198] auto b = ["1", "2", "abc", "", "99999999999999999999", "99"]; b.map!toNullableInt.filter!"!a.isNull".map!"a.get * 2".writeln; // prints [2, 4, -294967298, 198] auto c = ["1", "2", "abc", "", "1999999999", "99"]; c.map!toNullableInt.filter!"!a.isNull".map!"a.get * 2".writeln; } ``` Take a look at the `b` array. With this particular design, the "99999999999999999999" value is going to be silently filtered out. If you or your users happen to expect only non-digit string literals or empty strings to be filtered out, then there may be a very nasty and difficult to debug unexpected surprise awaiting down the road. Also take a look at the `c` array. The handling of arithmetic overflows is a safety problem of the D language design. Certain types of input may cause overflows, which result in producing bogus data as a result of running your program and are very difficult to troubleshoot. The use of the GDC's `-ftrapv` option surely helps in troubleshooting such cases, but some software or D libraries may intentionally rely on the D's arithmetic wraparound feature.
Dec 31 2023
Siarhei Siamashka <siarhei.siamashka gmail.com> On Sunday, 31 December 2023 at 09:47:27 UTC, Siarhei Siamashka wrote:On Saturday, 30 December 2023 at 13:25:00 UTC, Christian Köstlin wrote:And for comparison, here's a Ruby/Crystal example too: ```Ruby class String def to_i?; self.to_i rescue nil if self =~ /^\s*\-?[0-9]+\s*$/ end end a = ["1", "2", "abc", ""] pp a.map {|s| (v = s.to_i?) ? v * 2 : nil}.compact pp a.map {|s| s.to_i?}.compact.map {|v| v * 2} b = ["1", "2", "abc", "", "99999999999999999999", "99"] pp b.map {|s| (v = s.to_i?) ? v * 2 : nil}.compact pp b.map {|s| s.to_i?}.compact.map {|v| v * 2} c = ["1", "2", "abc", "", "1999999999", "99"] pp c.map {|s| (v = s.to_i?) ? v * 2 : nil}.compact pp c.map {|s| s.to_i?}.compact.map {|v| v * 2} ``` Running as a Ruby program: ``` [2, 4] [2, 4] [2, 4, 199999999999999999998, 198] [2, 4, 199999999999999999998, 198] [2, 4, 3999999998, 198] [2, 4, 3999999998, 198] ``` Running as a Crystal program: ``` [2, 4] [2, 4] [2, 4, 198] [2, 4, 198] Unhandled exception: Arithmetic overflow (OverflowError) ``` Ruby is an interpreter with an arbitrarily large BigInt integer type. Crystal is a native compiler with the default 32-bit integer type (but it also supports 64-bit integers, 128-bit integers and BigInts via optional type annotations). This particular Crystal code is badly written and stinks because it's silently filtering out "99999999999999999999" (similar to my earlier D example, which has exactly the same problem). But the arithmetic overflow during multiplication is at least caught at runtime. The code can be of course updated to allow wraparounds or filter out the arithmetic overflows from the resulting array too, but I think that this is a dubious approach in general. By the way, under the hood Crystal is effectively using a more powerful generalization of D's `Nullable`: https://crystal-lang.org/reference/1.10/syntax_and_semantics/union_types.htmlThe "original" https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.filter_map works with the Option(Some/None) ...Here's an example with `Nullable`, inspired by the earlier comment from Alexandru Ermicioi: [...]
Dec 31 2023
On Sunday, 31 December 2023 at 09:47:27 UTC, Siarhei Siamashka wrote:Also take a look at the `c` array. The handling of arithmetic overflows is a safety problem of the D language design. Certain types of input may cause overflows, which result in producing bogus data as a result of running your program and are very difficult to troubleshoot. The use of the GDC's `-ftrapv` option surely helps in troubleshooting such cases, but some software or D libraries may intentionally rely on the D's arithmetic wraparound feature.`CheckedInt` should be another solution for overflows, if you really need it to throw exceptions, on overflow errors.
Dec 31 2023
On Sunday, 31 December 2023 at 14:47:27 UTC, Alexandru Ermicioi wrote:`CheckedInt` should be another solution for overflows, if you really need it to throw exceptions, on overflow errors.CheckedInt is worse than nothing. It exists to give an illusion of having at least something to address this problem in D language, but it only acts as a distraction and drains valuable time. I don't see any practical scenario, where it would be useful. My current solution is to use a custom build of GDC with `-ftrapv` option for arithmetic overflows diagnostics. It's far from perfect, but at least it does the job in a non-invasive way without having any need to patch my code.
Jan 27
On Friday, 29 December 2023 at 23:10:47 UTC, Christian Köstlin wrote:Is there a way to implement filterMap (meaning do mapping of a range, but if something happens during the map, leave this element out of the resulting range).Perhaps try map to null cases that exception is thrown then filter out those, and then unwrap from nullable type: ```d myRange.map!(e => e.to!T.asNullForException).filter!(e => !e.isNull).map!(e => e.get).myOtherAlgorithm() ``` where asNullForException will return empty `Nullable` in case conversion fails.
Dec 30 2023
On Saturday, 30 December 2023 at 18:08:55 UTC, Alexandru Ermicioi wrote:On Friday, 29 December 2023 at 23:10:47 UTC, Christian Köstlin wrote:asNullForException could be nicely implemented with lazy parameters i guess. then its back to the map/filter/map chain that motivated filter_map in rust ... thanks for this idea... kind regards, ChristianIs there a way to implement filterMap (meaning do mapping of a range, but if something happens during the map, leave this element out of the resulting range).Perhaps try map to null cases that exception is thrown then filter out those, and then unwrap from nullable type: ```d myRange.map!(e => e.to!T.asNullForException).filter!(e => !e.isNull).map!(e => e.get).myOtherAlgorithm() ``` where asNullForException will return empty `Nullable` in case conversion fails.
Dec 30 2023