www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Use UFCS for reducing dependencies

reply Hipreme <msnmancini hotmail.com> writes:
Old habits die hard.

If you're coding in Java, imagine that you want to read a PNG 
from a file, you'll basically have a class like:
```d
class PNG
{
     public bool loadFromMemory(ubyte[] data){return decode(data);}
     public bool loadFromFile(string filePath){return 
readFromMemory(file.read(filePath));}
}
```

Usually it is very common adding API for loading files in your 
decoders, specially in high level wrappers, but this creates a 
problem: Decoding your file is totally unrelated to reading the 
file, yet, you did created the dependency between file and 
decoder.

After some refactoring in my project, trying to reduce 
dependencies, I came to a solution that we should not create 
dependencies like that.

Thanks to UFCS, we can extend the class without actually creating 
such dependency, even in high level wrappers, the way to extend 
your class without creating that kind of dependency is by 
basically creating a extension module:

```d
module png.extension;

//This module contain png extensions based on other libraries.

version(FileSystemPNG)
bool loadFromFile(PNG png, string filePath)
{
     return png.loadFromMemory(file.ready(filePath));
}

```

That way, one could easily call it like:
```d
import png.extension;

//One could even create a file which would import both 
png.decoder and png.extension

PNG png = new PNG();
png.loadFromFile("somewhere.png");
```

You could even save the path inside the PNG at the constructor 
and .loadFromFile would directly access its member.


I came here to show this technique because often people will try 
coding D without really thinking the D way to solve problems, as 
it happened to me. The code become a lot cleaner, one less 
function to worry in your class, one less dependency. I have been 
doing a real refactor on my code around that concept, that way, 
one could easily use any kind of file system reading without even 
needing to refactor the code. Even better, you could create your 
own extension without needing to modify the PNG code.


This is, together with the Range interfaces, one of really valid 
and useful usecase for UFCS. If you guys have any other 
techniques you use for reducing dependencies, I would be glad to 
know :)
Jul 16 2022
next sibling parent reply Bastiaan Veelo <Bastiaan Veelo.net> writes:
On Saturday, 16 July 2022 at 22:10:13 UTC, Hipreme wrote:

 Old habits die hard.


[...]


 After some refactoring in my project, trying to reduce 
 dependencies, I came to a solution that we should not create 
 dependencies like that.


Instead of using versions and sub-modules, using templates and 
conditional compilation is probably easier:

```d
struct PNG
{
     this(T)(T source)
     {
         static if (is(T == string))
         {
             pragma(msg, "conditional dependency on stdio");
             import std.stdio;

             // Load from file
         }
         else static if (is(T == ubyte[]))
         {
             // Decode from memory
         }
         else
             static assert(false, "Unsupported source type " ~ 
T.stringof);
     }
}

void main()
{
     // auto png = PNG("somewhere.png");
     auto png = PNG(new ubyte[10]);
}
```

Only the code that corresponds to the type that the constructor 
is called with gets compiled in, including any imports that are 
there. So the import of `std.stdio` only happens if a PNG is read 
from file.

Another tip: unless you need polymorphism or reference behaviour 
for your type, a `struct` is often preferred over a `class`. Old 
habits die hard :-)

-- Bastiaan.
Jul 17 2022
parent reply Paul Backus <snarwin gmail.com> writes:
On Sunday, 17 July 2022 at 12:08:38 UTC, Bastiaan Veelo wrote:

 On Saturday, 16 July 2022 at 22:10:13 UTC, Hipreme wrote:

 Old habits die hard.


 [...]


 After some refactoring in my project, trying to reduce 
 dependencies, I came to a solution that we should not create 
 dependencies like that.


 Instead of using versions and sub-modules, using templates and 
 conditional compilation is probably easier:

 ```d
 struct PNG
 {
     this(T)(T source)
     {
         static if (is(T == string))
         {
             pragma(msg, "conditional dependency on stdio");
             import std.stdio;

             // Load from file
         }
         else static if (is(T == ubyte[]))
         {
             // Decode from memory
         }
         else
             static assert(false, "Unsupported source type " ~ 
 T.stringof);
     }
 }
 ```


Another possibility:

```d
struct PNG
{
     static PNG load(Source)(Source source)
         if (isInputRange!Source && is(ElementType!Source == 
ubyte))
     {
         // etc.
     }
}
```

This way, the PNG module itself is completely agnostic about what 
data source it loads from, and has no explicit dependencies 
(except on `std.range`, I guess).
Jul 17 2022
parent reply Hipreme <msnmancini hotmail.com> writes:
On Sunday, 17 July 2022 at 16:15:07 UTC, Paul Backus wrote:

 Another possibility:

 ```d
 struct PNG
 {
     static PNG load(Source)(Source source)
         if (isInputRange!Source && is(ElementType!Source == 
 ubyte))
     {
         // etc.
     }
 }
 ```

 This way, the PNG module itself is completely agnostic about 
 what data source it loads from, and has no explicit 
 dependencies (except on `std.range`, I guess).



Could you extend it a bit further how would that work?
Jul 18 2022
parent reply Paul Backus <snarwin gmail.com> writes:
On Monday, 18 July 2022 at 10:40:23 UTC, Hipreme wrote:

 On Sunday, 17 July 2022 at 16:15:07 UTC, Paul Backus wrote:

 Another possibility:

 ```d
 struct PNG
 {
     static PNG load(Source)(Source source)
         if (isInputRange!Source && is(ElementType!Source == 
 ubyte))
     {
         // etc.
     }
 }
 ```

 This way, the PNG module itself is completely agnostic about 
 what data source it loads from, and has no explicit 
 dependencies (except on `std.range`, I guess).



 Could you extend it a bit further how would that work?


Well, I don't know the algorithm for PNG decoding, so I don't 
know whether it requires an input range, a forward range, or a 
random access range. But the basic idea is, you declare your 
`load` function as taking a generic range type, and then you can 
load from anything that implements the appropriate range 
interface.

Because the algorithm (`PNG.load`) and the data sources 
communicate only via the range API, neither one needs to have any 
special knowledge of, or explicit dependency on, the other.

This is the fundamental idea behind ranges. If you have N data 
sources that implement the range interface, and M algorithms that 
consume the range interface, then you do not need to implement 
all M×N combinations by hand--all of your data sources and all of 
your algorithms will Just Work™ with each other.
Jul 18 2022
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 19/07/2022 12:46 AM, Paul Backus wrote:

 Well, I don't know the algorithm for PNG decoding, so I don't know 
 whether it requires an input range, a forward range, or a random access 
 range.


For AV handling, ranges are not the right tool for the job, too inefficient.

You want to be working with arrays directly.
Jul 18 2022
parent reply Paul Backus <snarwin gmail.com> writes:
On Monday, 18 July 2022 at 12:55:39 UTC, rikki cattermole wrote:

 On 19/07/2022 12:46 AM, Paul Backus wrote:

 Well, I don't know the algorithm for PNG decoding, so I don't 
 know whether it requires an input range, a forward range, or a 
 random access range.


 For AV handling, ranges are not the right tool for the job, too 
 inefficient.

 You want to be working with arrays directly.


You can have a fast path for arrays and fall back to the slower, 
more generic version for other ranges.

But yes, the whole point of ranges is to decouple your algorithm 
from the source of the data it operates on. If you *want* your 
algorithm to be coupled to a particular source of data, then 
ranges will only get in your way. (Although it may be worth 
asking yourself: are you really sure that's what you want?)
Jul 18 2022
parent rikki cattermole <rikki cattermole.co.nz> writes:
On 19/07/2022 1:43 AM, Paul Backus wrote:

 You can have a fast path for arrays and fall back to the slower, more 
 generic version for other ranges.
 
 But yes, the whole point of ranges is to decouple your algorithm from 
 the source of the data it operates on. If you *want* your algorithm to 
 be coupled to a particular source of data, then ranges will only get in 
 your way. (Although it may be worth asking yourself: are you really sure 
 that's what you want?)


AV data gets very large, and has complex processing that can be easily 
vectorized.

Formats like PNG, support compression via say zlib. So you're going to 
need to send a block of memory over to it.

Point is, using ranges for this is just shooting yourself in the foot if 
you want any type of performance that isn't bad.
Jul 18 2022
prev sibling parent Adam D Ruppe <destructionator gmail.com> writes:
On Monday, 18 July 2022 at 12:46:20 UTC, Paul Backus wrote:

 But the basic idea is, you declare your `load` function as 
 taking a generic range type, and then you can load from 
 anything that implements the appropriate range interface.


https://github.com/adamdruppe/arsd/blob/master/png.d#L1145

This was actually one of the first times I tried to write a range 
consumer, so.... not very good code. (Hit the git blame button 
and find that whole block was committed in June 2013... before I 
wrote the range chapter of my book lol)

But still, it shows the rough concept in a real implementation.
Jul 18 2022
prev sibling parent reply Dave P. <dave287091 gmail.com> writes:
On Saturday, 16 July 2022 at 22:10:13 UTC, Hipreme wrote:

 [...]


I like this. What's annoying is that it doesn't work smoothly 
with structs. Methods automatically deference pointers, but UFCS 
functions don't.

```D
struct Foo {
     int x;
     void mutate(){
         x++;
     }
}

void mutate2(ref Foo foo){
     foo.x++;
}

void main(){
     Foo foo;
     Foo* pfoo = &foo;

     // Both ref and pointer work for methods
     foo.mutate();
     pfoo.mutate();

     // Ref works for ufcs
     foo.mutate2();
     // But pointer doesn't work for UFCS

     // Can't do this:
     // pfoo.mutate2();

     // Must do this:
     (*pfoo).mutate2();

     assert(foo.x == 4);
}

```
Jul 17 2022
parent Bastiaan Veelo <Bastiaan Veelo.net> writes:
On Sunday, 17 July 2022 at 16:25:52 UTC, Dave P. wrote:

 On Saturday, 16 July 2022 at 22:10:13 UTC, Hipreme wrote:

 [...]


 I like this. What's annoying is that it doesn't work smoothly 
 with structs. Methods automatically deference pointers, but 
 UFCS functions don't.

 ```D
 struct Foo {
     int x;
     void mutate(){
         x++;
     }
 }

 void mutate2(ref Foo foo){
     foo.x++;
 }

 void main(){
     Foo foo;
     Foo* pfoo = &foo;

     // Both ref and pointer work for methods
     foo.mutate();
     pfoo.mutate();

     // Ref works for ufcs
     foo.mutate2();
     // But pointer doesn't work for UFCS

     // Can't do this:
     // pfoo.mutate2();

     // Must do this:
     (*pfoo).mutate2();

     assert(foo.x == 4);
 }

 ```


It works if you add an overload:
```
void mutate2(Foo *foo)
{
     mutate2(*foo);
}
```
Jul 17 2022