www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - Article: Writing Julia style multiple dispatch code in D

reply data pulverizer <data.pulverizer gmail.com> writes:
Hi all,

I have written an article about writing Julia style multiple 
dispatch code in D 
(https://github.com/dataPulverizer/dispatch-it-like-julia). I am 
hoping that it is appropriate for the D blog.

Reviews please.

Many Thanks!
Aug 24
next sibling parent reply David Gileadi <gileadisNOSPM gmail.com> writes:
On 8/24/17 9:10 AM, data pulverizer wrote:
 Hi all,
 
 I have written an article about writing Julia style multiple dispatch 
 code in D (https://github.com/dataPulverizer/dispatch-it-like-julia). I 
 am hoping that it is appropriate for the D blog.
 
 Reviews please.
 
 Many Thanks!
Very interesting! I have a couple suggestions: for newbies like me, it would be nice to include a short explanation of multiple dispatch, and maybe a link to a longer description. Also it wouldn't hurt to include a short excerpt of how this code would look in Julia, along with the D examples. Finally, I wonder if Discrete and Continuous couldn't be user-defined attributes.
Aug 24
parent data pulverizer <data.pulverizer gmail.com> writes:
On Thursday, 24 August 2017 at 16:41:54 UTC, David Gileadi wrote:
 Very interesting!
Thank you
 I have a couple suggestions: for newbies like me, it would be 
 nice to include a short explanation of multiple dispatch, and 
 maybe a link to a longer description.
Wikipedia's description of multiple dispatch is pretty good, I'll include it and add a short description.
 ... Also it wouldn't hurt to include a short excerpt of how 
 this code would look in Julia, along with the D examples.
I included a link to Julia package as an example but I could add a short snippet as illustration.
 Finally, I wonder if Discrete and Continuous couldn't be 
 user-defined attributes.
They could but the code closely models Julia's typing system, but UDAs could add good usability to this style of programming as well.
Aug 24
prev sibling next sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 08/24/2017 09:10 AM, data pulverizer wrote:
 Hi all,

 I have written an article about writing Julia style multiple dispatch
 code in D (https://github.com/dataPulverizer/dispatch-it-like-julia). I
 am hoping that it is appropriate for the D blog.

 Reviews please.

 Many Thanks!
This works only with compile-time dispatch, right? Does Julia support dynamic multiple dispatch? In any case, Jean-Louis Leroy did some magic recently to support multiple dynamic dispatch in D. :) http://forum.dlang.org/post/cigbfrgipbokyetskypk forum.dlang.org Ali
Aug 24
parent reply data pulverizer <data.pulverizer gmail.com> writes:
On Thursday, 24 August 2017 at 17:01:38 UTC, Ali Çehreli wrote:
 This works only with compile-time dispatch, right?
Yes
 ... Does Julia support dynamic multiple dispatch?
Okay Julia is my second favourite language next to D and one of it's cool features is that even though it is a dynamic programming language, methods are compiled for specific call signatures on first use of the function. So officially Julia does dynamic dispatch, but it pre-compiles function signatures.
 In any case, Jean-Louis Leroy did some magic recently to 
 support multiple dynamic dispatch in D. :)

 http://forum.dlang.org/post/cigbfrgipbokyetskypk forum.dlang.org
I haven't seen this. I'll have to get back to you when I have read it. Thanks
Aug 24
parent jmh530 <john.michael.hall gmail.com> writes:
On Thursday, 24 August 2017 at 17:20:20 UTC, data pulverizer 
wrote:
 In any case, Jean-Louis Leroy did some magic recently to 
 support multiple dynamic dispatch in D. :)

 http://forum.dlang.org/post/cigbfrgipbokyetskypk forum.dlang.org
I haven't seen this. I'll have to get back to you when I have read it.
I wouldn't expect it to be that useful for univariate distributions as you wouldn't have much reason to have a different implmentation at run-time. However, it might be useful with multivariate distributions for the same reason that you might want to specialize matrix math by the size of the matrix.
Aug 24
prev sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Thursday, 24 August 2017 at 16:10:32 UTC, data pulverizer 
wrote:
 Hi all,

 I have written an article about writing Julia style multiple 
 dispatch code in D 
 (https://github.com/dataPulverizer/dispatch-it-like-julia). I 
 am hoping that it is appropriate for the D blog.

 Reviews please.

 Many Thanks!
I think at one point I had actually suggested that dstats or something be re-written in a Julia-like way (before I realized how much work that would be!). It looks very pretty. Nevertheless, you might be re-inventing the wheel a bit if you want to build a whole library in this style. My recommendation would be to write a front-end for the dstats.distrib and dstats.random submodules in this style. That way you won't need to re-write all the functions, you can just call ones from dstats that have already been tested. More generally, I prefer the structs because they don't rely on the garbage collector, but the class/interface version is prettier. Atila's concepts library has implements, which you might find helpful. I have gently nudged him to work on something that also can tell if a type is a subtype of another type more generally (like a struct being a subtype of another struct). I think this would really be a good use case for that functionality.
Aug 24
parent reply data pulverizer <data.pulverizer gmail.com> writes:
On Thursday, 24 August 2017 at 18:16:21 UTC, jmh530 wrote:
 I think at one point I had actually suggested that dstats or 
 something be re-written in a Julia-like way (before I realized 
 how much work that would be!). It looks very pretty.
Thanks. I think most of that is down to D's nice syntax and how it easily and clearly emulates Julia. I think that there is certainly more to say on this especially around strategies of how represent concrete types. David Gileadi's point about UDAs could add an interesting spin on things, and Ali's point on dynamic dispatching.
 Nevertheless, you might be re-inventing the wheel a bit if you 
 want to build a whole library in this style.
True. I have found a couple of projects that fresh targets that I am working on and writing code in this style.
 My recommendation would be to write a front-end for the 
 dstats.distrib and dstats.random submodules in this style. That 
 way you won't need to re-write all the functions, you can just 
 call ones from dstats that have already been tested.
True code reuse is important, especially when you want to get something working quickly.
 More generally, I prefer the structs because they don't rely on 
 the garbage collector, but the class/interface version is 
 prettier.
Aha! I was wandering why I see people avoid classes even when using them is clearly the best way to represent their objects. For some reason it never occurred to me that they where just trying to avoid the GC. I just thought they didn't want to use reference objects.
 Atila's concepts library has implements, which you might find 
 helpful. I have gently nudged him to work on something that 
 also can tell if a type is a subtype of another type more 
 generally (like a struct being a subtype of another struct). I 
 think this would really be a good use case for that 
 functionality.
I was thinking about this article for some time, it was Atila's article (https://atilanevesoncode.wordpress.com/2017/08/23/on-the-novelty-factor-of-compil -time-duck-typing/) that was a trigger for me writing it.
Aug 24
parent reply jmh530 <john.michael.hall gmail.com> writes:
On Thursday, 24 August 2017 at 20:11:32 UTC, data pulverizer 
wrote:
 Thanks. I think most of that is down to D's nice syntax and how 
 it easily and clearly emulates Julia. I think that there is 
 certainly more to say on this especially around strategies of 
 how represent concrete types. David Gileadi's point about UDAs 
 could add an interesting spin on things, and Ali's point on 
 dynamic dispatching.
On UDAs, at least in the current implementation, I think that the actual issue you are trying to address is to force the type in the distribution to be convertible to double in the continuous case and convertible to long in the discrete case. All things considered, that can be implemented with template constraints, as in class Gamma(T): if(isFloatingPoint!T) { immutable(T) shape; immutable(T) scale; this(T shape, T scale) { this.shape = shape; this.scale = scale; } } though you could probably take a more abstract approach. (I'm also not 100% on having immutable member variables). Also, density's signature could then avoid the template constraint. auto density(D: Gamma!T, U : T, T)(D d, U x) Even better, if you're calling the dstats functions, you could re-write density as something like auto pdf(D: Dist!T, U : T, Dist, T)(U x, D d) { mixin("return x." ~ lookupdstatdensity!Dist ~ "(" ~ stringmembersofd ~ ")"; } and create a lookupdstatdensity function that returns a string of the relevant dstats function at compile-time (and a function returning a string of the members of d) (I also would re-name density to pdf and switch the order of x and d). This would probably be the most DRY approach. On Ali's point on dynamic dispatching, Julia is a scripting language with a JIT compiler. So if you call a function with some types known at compile time and the overload exists, it will compile the correct version of the function for the relevant types. It will then cache that so that if you need it again you don't pay any additional cost. So it's similar to what you're doing on that respect. However, there is a runtime dispatch component that would take something like openmethods to implement, I think.
Aug 24
parent reply data pulverizer <data.pulverizer gmail.com> writes:
On Thursday, 24 August 2017 at 21:13:10 UTC, jmh530 wrote:
 On UDAs, at least in the current implementation, I think that 
 the actual issue you are trying to address is to force the type 
 in the distribution to be convertible to double in the 
 continuous case and convertible to long in the discrete case. 
 All things considered, that can be implemented with template 
 constraints, as in

 class Gamma(T):
     if(isFloatingPoint!T)
 {
     immutable(T) shape;
     immutable(T) scale;
     this(T shape, T scale)
     {
         this.shape = shape;
         this.scale = scale;
     }
 }
Okay, I admit that I try to avoid using template constraints whenever I can - i.e. unless the code breaks! It's a habit I am trying to break. In reality I will have to have them everywhere and the code will end up looking much less pretty, in reality, I know I'll probably need one or two catchall functions that look something like this ``` double density(D: UnivariateDistribution!Discrete, U = getVariateType!D, T = GetDistributionParameterType!D)(D d, U x) if(!is(D == Poisson!T)) { assert(false, "density function unimplemented for this distribution: " ~ D.stringof); } double density(D: UnivariateDistribution!Continuous, U = getVariateType!D, T = GetDistributionParameterType!D)(D d, U x) if(!is(D == Gamma!T) && !is(D == Gaussian!T) && !is(D == Uniform!T) && !is(D == Exponential!T)) { assert(false, "density function unimplemented for this distribution: " ~ D.stringof); } ``` She's not so pretty anymore captain! This is why some time ago I suggested introducing the Union keyword that Julia has https://forum.dlang.org/post/lkcmqlpsdcopfebwgikj forum.dlang.org
 though you could probably take a more abstract approach. (I'm 
 also not 100% on having immutable member variables).
I am 100% sure that I want either the instantiated distribution object to be immutable, or the parameters to be immutable once instantiated. Its a safety feature that I don't see a need for mutable distribution objects. Once the parameters change, its not the same distribution. Ideally I want to be able to say immutable class(T...){...} that this class can only create immutable objects without having to write immutable everywhere and or a UDA, but making every member immutable accomplishes the same thing.
 Also, density's signature could then avoid the template 
 constraint.

 auto density(D: Gamma!T, U : T, T)(D d, U x)
Sorry U is not T, T is the type of the parameters, U is the type of the variate.
 Even better, if you're calling the dstats functions, you could 
 re-write density as something like

 auto pdf(D: Dist!T, U : T, Dist, T)(U x, D d) {
     mixin("return x." ~ lookupdstatdensity!Dist ~ "(" ~ 
 stringmembersofd ~ ")";
 }

 and create a lookupdstatdensity function that returns a string 
 of the relevant dstats function at compile-time (and a function 
 returning a string of the members of d) (I also would re-name 
 density to pdf and switch the order of x and d). This would 
 probably be the most DRY approach.
Sounds like a reasonable approach, though I haven't looked at the dstats package in great detail.
 Julia is a scripting language with a JIT compiler. So if you 
 call a function with some types known at compile time and the 
 overload exists, it will compile the correct version of the 
 function for the relevant types.
Yes, I guess you could say that Julia is an interactive compiler, where you can create new compiled types and methods in the same session.
 So it's similar to what you're doing on that respect. However, 
 there is a runtime dispatch component that would take something 
 like openmethods to implement, I think.
I find OOP-polymorphic types ultimately unsatisfying, but I don't know of anyway to write, compile and load a D script with new types and methods on the fly into the same session.
Aug 24
next sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Thursday, 24 August 2017 at 23:50:21 UTC, data pulverizer 
wrote:
 ```
 double density(D: UnivariateDistribution!Discrete, U = 
 getVariateType!D, T = GetDistributionParameterType!D)(D d, U x)
 if(!is(D == Poisson!T))
 {
     assert(false, "density function unimplemented for this 
 distribution: " ~ D.stringof);
 }

 double density(D: UnivariateDistribution!Continuous, U = 
 getVariateType!D, T = GetDistributionParameterType!D)(D d, U x)
 if(!is(D == Gamma!T) && !is(D == Gaussian!T) && !is(D == 
 Uniform!T) && !is(D == Exponential!T))
 {
     assert(false, "density function unimplemented for this 
 distribution: " ~ D.stringof);
 }
What you seem concerned about here is how to produce a meaningful error message for distribution that you do not have implementations for. A slightly more elegant solution would be to pack the structs into an AliasSeq and then use something like !allSatisfies to test them all. I'm sure there's a more elegant solution, but that's the first thing I thought of.
 immutable class(T...){...}

 that this class can only create immutable objects without 
 having to write immutable everywhere and or a UDA, but making 
 every member immutable accomplishes the same thing.
What you're looking for is an immutable constructor: class C { this() immutable; }
Aug 24
parent reply data pulverizer <data.pulverizer gmail.com> writes:
On Friday, 25 August 2017 at 00:35:24 UTC, jmh530 wrote:
 What you seem concerned about here is how to produce a 
 meaningful error message for distribution that you do not have 
 implementations for. A slightly more elegant solution would be 
 to pack the structs into an AliasSeq and then use something 
 like !allSatisfies to test them all. I'm sure there's a more 
 elegant solution, but that's the first thing I thought of.
Andrei suggested allSatisfies that as an alternative approach to a Union keyword similar to Julia, at the time I was still stuck on how cool having a Union keyword like Julia's in D would be.
 immutable class(T...){...}
What you're looking for is an immutable constructor: class C { this() immutable; }
Aha, thanks!
Aug 24
parent reply jmh530 <john.michael.hall gmail.com> writes:
On Friday, 25 August 2017 at 01:04:31 UTC, data pulverizer wrote:
 [snip]
With respect to your point about immutability, you might be interested in the parameterize function in dstats.distrib. I hadn't noticed that was there, but I think it accomplishes, to a limited extent, the behavior of what you want. It returns a delegate with the values of the distribution fixed in there. Along the same lines, I think below is how I would set it up, rather than the mixin approach I discussed above. While it does not currently work with the parametrize funciton currently, I believe that with some simple adjustments it could. import std.stdio : writeln; struct Normal { import dstats : normalCDF, normalCDFR, normalPDF, invNormalCDF; alias cdf = normalCDF; alias cdfr = normalCDFR; alias pdf = normalPDF; alias density = pdf; alias icdf = invNormalCDF; } struct LogNormal { import dstats : logNormalCDF, logNormalCDFR, logNormalPDF; alias cdf = logNormalCDF; alias cdfr = logNormalCDFR; alias pdf = logNormalPDF; alias density = pdf; //no icdf for LogNormal in dstats } private void hasMemberCheck(alias T, string x)() { import std.traits : hasMember; static assert(hasMember!(T, x), T.stringof ~ " must have " ~ x ~" member to call " ~ x ~ " function"); } auto cdf(alias T, U...)(U u) { hasMemberCheck!(T, "cdf"); return T.cdf(u); } auto pdf(alias T, U...)(U u) { hasMemberCheck!(T, "pdf"); return T.pdf(u); } auto pmf(alias T, U...)(U u) { hasMemberCheck!(T, "pmf"); return T.pmf(u); } auto cdfr(alias T, U...)(U u) { hasMemberCheck!(T, "cdfr"); return T.cdfr(u); } auto density(alias T, U...)(U u) { hasMemberCheck!(T, "density"); return T.density(u); } auto icdf(alias T, U...)(U u) { hasMemberCheck!(T, "icdf"); return T.icdf(u); } void main() { writeln(Normal.cdf(0.5, 0.0, 1.0)); writeln(cdf!Normal(0.5, 0.0, 1.0)); writeln(icdf!Normal(cdf!Normal(0.5, 0.0, 1.0), 0.0, 1.0)); writeln(LogNormal.cdf(0.5, 0.0, 1.0)); writeln(cdf!LogNormal(0.5, 0.0, 1.0)); //writeln(icdf!LogNormal(cdf!LogNormal(0.5, 0.0, 1.0), 0.0, 1.0)); //error }
Aug 25
parent reply data pulverizer <data.pulverizer gmail.com> writes:
On Friday, 25 August 2017 at 14:30:03 UTC, jmh530 wrote:
 On Friday, 25 August 2017 at 01:04:31 UTC, data pulverizer 
 wrote:
 [snip]
With respect to your point about immutability, you might be interested in the parameterize function in dstats.distrib. I hadn't noticed that was there, but I think it accomplishes, to a limited extent, the behavior of what you want. It returns a delegate with the values of the distribution fixed in there. Along the same lines, I think below is how I would set it up, rather than the mixin approach I discussed above. While it does not currently work with the parametrize funciton currently, I believe that with some simple adjustments it could. import std.stdio : writeln; struct Normal { ...
Your wrapping strategy looks sensible though I would probably generate them all using string mixins.
Aug 25
next sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Friday, 25 August 2017 at 16:01:27 UTC, data pulverizer wrote:
 Your wrapping strategy looks sensible though I would probably 
 generate them all using string mixins.
That might require less maintenance going forward.
Aug 25
prev sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Friday, 25 August 2017 at 16:01:27 UTC, data pulverizer wrote:
 Your wrapping strategy looks sensible though I would probably 
 generate them all using string mixins.
See below. I haven't implemented the random variables yet, but otherwise it seems to be working well. There is some trickiness with deprecated stuff that I had to hard code, but other than that it's pretty generic. Also, I think it is ignoring my check to only include public/export stuff. Not sure why that is. module distribAlt; private template isMemberOf(alias T, string x) { import std.traits : hasMember; enum bool isMemberOf = hasMember!(T, x); } private void hasMemberCheck(alias T, string x)() { static assert(isMemberOf!(T, x), T.stringof ~ " must have " ~ x ~" member to call " ~ x ~ " function"); } private string genStructInternals(string funcName, string structName)() { import dstats.distrib; import std.array : appender; import std.algorithm.searching : endsWith; enum spaces = " "; auto aliasBuf = appender!string(); auto importBuf = appender!string(); enum string invName = "inv" ~ structName; enum bool anyPDForPMF = false; importBuf.put(spaces); importBuf.put("import dstats.distrib : "); foreach(member; __traits(allMembers, dstats.distrib)) { static if (__traits(getProtection, member) == "public" || __traits(getProtection, member) == "export") { import std.algorithm.searching : startsWith, findSplitAfter; import std.string : toLower; static if (startsWith(member, funcName)) { enum string memberAfter = findSplitAfter(member, funcName)[1]; enum string lowerMemberAfter = toLower(memberAfter); importBuf.put(member ~ ", "); aliasBuf.put(spaces); aliasBuf.put("alias " ~ lowerMemberAfter ~ " = " ~ member ~ ";"); aliasBuf.put("\n"); static if ((lowerMemberAfter == "pdf") || (lowerMemberAfter == "pmf")) { aliasBuf.put(spaces); aliasBuf.put("alias density = " ~ lowerMemberAfter ~ ";"); aliasBuf.put("\n"); } } else static if (startsWith(member, invName)) { enum string memberAfter = findSplitAfter(member, invName)[1]; importBuf.put(member ~ ", "); aliasBuf.put(spaces); aliasBuf.put("alias i" ~ toLower(memberAfter) ~ " = " ~ member ~ ";"); aliasBuf.put("\n"); } } } if (endsWith(importBuf.data, ", ")) { string importOut = importBuf.data[0 .. ($ - (", ".length))] ~";\n"; if (endsWith(aliasBuf.data, "\n")) return importOut ~ aliasBuf.data[0 .. ($ - ("\n").length)]; else assert(0, "No relevant functions in dstats.distrib"); } else { assert(0, "No relevant functions in dstats.distrib"); } } private string toLowerFirst(string name)() { import std.string : toLower; import std.conv : to; string firstLetter = name[0].toLower.to!string; return firstLetter ~ name[1 .. $]; } private string toUpperFirst(string name)() { import std.string : toUpper; import std.conv : to; string firstLetter = name[0].toUpper.to!string; return firstLetter ~ name[1 .. $]; } private template GenDistStruct(string name) { const char[] GenDistStruct = "///"~ "\n" ~ "struct " ~ toUpperFirst!(name) ~ "\n" ~ "{\n" ~ genStructInternals!(name, toUpperFirst!(name)) ~ "\n" ~ "}"; } string GenDistStructs() { import dstats.distrib; import std.array : appender; import std.algorithm.searching : startsWith, endsWith, canFind, findSplitBefore, findSplitAfter; string[__traits(allMembers, dstats.distrib).length] createdStructs; size_t i; auto structsBuf = appender!string(); foreach(member; __traits(allMembers, dstats.distrib)) { static if (__traits(getProtection, member) == "public" || __traits(getProtection, member) == "export") { static if ((member.endsWith("PDF") || member.endsWith("PMF") || member.endsWith("CDF") || member.endsWith("CDFR"))) { static if (member.endsWith("PDF")) enum string memberBefore = findSplitBefore(member, "PDF")[0]; else static if (member.endsWith("PMF")) enum string memberBefore = findSplitBefore(member, "PMF")[0]; else static if (member.endsWith("CDF")) enum string memberBefore = findSplitBefore(member, "CDF")[0]; else static if (member.endsWith("CDFR")) enum string memberBefore = findSplitBefore(member, "CDFR")[0]; static if (member.startsWith("inv")) enum string newMember = toLowerFirst!(findSplitAfter(memberBefore, "inv")[1]); else enum string newMember = memberBefore; static if (member != "chiSqrCDF" && member != "chiSqrCDFR" && member != "invChiSqrCDFR" && member != "invChiSqCDFR") //Deprecated: Easiest way I found to fix it { if (i == 0 || !(createdStructs[0 .. i].canFind(newMember))) { structsBuf.put(GenDistStruct!newMember); structsBuf.put("\n"); createdStructs[i] = newMember; i++; } } } } } return structsBuf.data; } mixin(GenDistStructs()); private template GenDistFunc(string name) { const char[] GenDistFunc = "auto " ~ name ~ "(alias T, U...)(U u)\n" ~ "{\n" ~ ` hasMemberCheck!(T, "` ~ name ~ `");` ~ "\n" ~ " return T." ~ name ~ "(u);\n" ~ "}"; } mixin(GenDistFunc!("pdf")); mixin(GenDistFunc!("pmf")); mixin(GenDistFunc!("cdf")); mixin(GenDistFunc!("cdfr")); mixin(GenDistFunc!("icdf")); mixin(GenDistFunc!("density")); void main() { import std.stdio : writeln; writeln(Normal.cdf(0.5, 0.0, 1.0)); writeln(cdf!Normal(0.5, 0.0, 1.0)); writeln(icdf!Normal(cdf!Normal(0.5, 0.0, 1.0), 0.0, 1.0)); writeln(LogNormal.cdf(0.5, 0.0, 1.0)); writeln(cdf!LogNormal(0.5, 0.0, 1.0)); }
Aug 25
parent reply data pulverizer <data.pulverizer gmail.com> writes:
On Friday, 25 August 2017 at 20:54:05 UTC, jmh530 wrote:
 See below. I haven't implemented the random variables yet, but 
 otherwise it seems to be working well. There is some trickiness 
 with deprecated stuff that I had to hard code, but other than 
 that it's pretty generic. Also, I think it is ignoring my check 
 to only include public/export stuff. Not sure why that is.

 module distribAlt;
 ...
Wow, I didn't realise that you'd go off and work on it. You need to start a package for it! I'll bookmark this one as a little reminder of mixin techniques.
Aug 25
parent jmh530 <john.michael.hall gmail.com> writes:
On Saturday, 26 August 2017 at 02:14:59 UTC, data pulverizer 
wrote:
 On Friday, 25 August 2017 at 20:54:05 UTC, jmh530 wrote:
 See below. I haven't implemented the random variables yet, but 
 otherwise it seems to be working well. There is some 
 trickiness with deprecated stuff that I had to hard code, but 
 other than that it's pretty generic. Also, I think it is 
 ignoring my check to only include public/export stuff. Not 
 sure why that is.

 module distribAlt;
 ...
Wow, I didn't realise that you'd go off and work on it. You need to start a package for it! I'll bookmark this one as a little reminder of mixin techniques.
Something I had wanted for a long time and once your article got my juices flowing. I had a hard time stopping once you got me started! I'm going to try to add support for the random number generators and then create a PR for dstats.
Aug 26
prev sibling next sibling parent reply Jean-Louis Leroy <jl leroy.nyc> writes:
On Thursday, 24 August 2017 at 23:50:21 UTC, data pulverizer 
wrote:
 I find OOP-polymorphic types ultimately unsatisfying, but I 
 don't know of anyway to write, compile and load a D script with 
 new types and methods on the fly into the same session.
That is why binding membership and polymorphism together is a historical wrong turn. CLOS had it right but the world followed the Simula/Smalltalk path because of a nice metaphor (objects sending messages to each other). My openmethods library allows you to add methods "from outside" and also supports dynamic loading: you can add new methods to existing classes and new classes to hierarchies that have methods. See the blog post that just came up.
Aug 28
parent reply data pulverizer <data.pulverizer gmail.com> writes:
On Monday, 28 August 2017 at 13:19:19 UTC, Jean-Louis Leroy wrote:
 On Thursday, 24 August 2017 at 23:50:21 UTC, data pulverizer 
 wrote:
 I find OOP-polymorphic types ultimately unsatisfying, but I 
 don't know of anyway to write, compile and load a D script 
 with new types and methods on the fly into the same session.
That is why binding membership and polymorphism together is a historical wrong turn. CLOS had it right but the world followed the Simula/Smalltalk path because of a nice metaphor (objects sending messages to each other). My openmethods library allows you to add methods "from outside" and also supports dynamic loading: you can add new methods to existing classes and new classes to hierarchies that have methods. See the blog post that just came up.
Thanks, just read it. Looks like a useful package. As you said it is about being able to dispatch with polymorphism ... where implicit promotions occur or explicit polymorphism is required. One thing that confused me was examples like this ... method Matrix _plus(DiagonalMatrix a, DiagonalMatrix b) { // just add the elements on diagonals // return a DiagonalMatrix } Which is marked as returning a DiagonalMatrix rather than a Matrix by polymorphism however the function is marked Matrix return type. You mentioned Julia in your article, however for clarity I would point out that Julia doesn't have OOP-type polymorphism. There is no notion of being able to do something like: Animal snoopy = new Dog(); It's dispatch method is more like compile-time over as discussed here https://github.com/dataPulverizer/dispatch-it-like-julia - but you can recompile 'online' which I guess is what they mean by "dynamic dispatch" - but there's no polymorphism. Type hierarchies are basically for dispatching methods with varying amounts of specificity which can be accomplished using templates and only explicitly defined type conversions from one concrete type to another are allowed.
Aug 30
next sibling parent reply data pulverizer <data.pulverizer gmail.com> writes:
On Wednesday, 30 August 2017 at 16:45:19 UTC, data pulverizer 
wrote:
 You mentioned Julia in your article, however for clarity I 
 would point out that Julia doesn't have OOP-type polymorphism. 
 There is no notion of being able to do something like:

 Animal snoopy = new Dog();
p.s. my bad, I was wrong about that! Turns out you can do something like this in Julia (apologies for the Julia code in a D forum): abstract type Animal end struct Dog <: Animal end struct Cat <: Animal end x = Array{Animal}(3) x[1] = Cat(); x[2] = Dog(); x[3] = Cat(); x # returns 3-element Array{Animal,1}: Cat() Dog() Cat() Which is polymorphism
Aug 30
parent reply data pulverizer <data.pulverizer gmail.com> writes:
On Wednesday, 30 August 2017 at 17:14:37 UTC, data pulverizer 
wrote:
 On Wednesday, 30 August 2017 at 16:45:19 UTC, data pulverizer 
 wrote:
 You mentioned Julia in your article, however for clarity I 
 would point out that Julia doesn't have OOP-type polymorphism. 
 There is no notion of being able to do something like:

 Animal snoopy = new Dog();
p.s. my bad, I was wrong about that! Turns out you can do something like this in Julia (apologies for the Julia code in a D forum): abstract type Animal end struct Dog <: Animal end struct Cat <: Animal end x = Array{Animal}(3) x[1] = Cat(); x[2] = Dog(); x[3] = Cat(); x # returns 3-element Array{Animal,1}: Cat() Dog() Cat()
p.p.s typeof(x[1]) # returns Cat so it isn't really polymorphism - the object is never converted to the "parent" type! Lol ... sorry for the confusion!
 Which is polymorphism
Aug 30
next sibling parent reply Jean-Louis Leroy <jl leroy.nyc> writes:
On Wednesday, 30 August 2017 at 17:16:59 UTC, data pulverizer 
wrote:
 p.p.s

 typeof(x[1]) # returns Cat

 so it isn't really polymorphism - the object is never converted 
 to the "parent" type! Lol ... sorry for the confusion!
 Which is polymorphism
Haha what I know of Julia is what wikipedia says. Confusing indeed...
Aug 30
parent reply data pulverizer <data.pulverizer gmail.com> writes:
On Wednesday, 30 August 2017 at 17:29:42 UTC, Jean-Louis Leroy 
wrote:
 On Wednesday, 30 August 2017 at 17:16:59 UTC, data pulverizer 
 wrote:
 p.p.s

 typeof(x[1]) # returns Cat

 so it isn't really polymorphism - the object is never 
 converted to the "parent" type! Lol ... sorry for the 
 confusion!
Haha what I know of Julia is what wikipedia says. Confusing indeed...
To be fair they say it is parametric polymorphism - dispatching basically template style, rather than subtyping polymorphism (OOP type), (more wikipedia https://en.wikipedia.org/wiki/Polymorphism_(computer_science)). The reason I have never really been comfortable with sub-typing is that the polymorphic types are a black-box, my preference is certainly for parametric type polymorphism. The main disadvantage with parametric polymorphism in compiled languages is that array containers only operate under subtyping polymorphism. In the above Julia example the array container is essentially acting exactly like a compile-time dispatch (overloaded) function - the array is dispatching on a specific set of types defined by the abstract parent type. That kind of construct would be very desirable to me in D. The closest such thing you can have to that in D are tuples.
Aug 30
parent reply data pulverizer <data.pulverizer gmail.com> writes:
On Wednesday, 30 August 2017 at 17:57:49 UTC, data pulverizer 
wrote:
 The reason I have never really been comfortable with sub-typing 
 is that the polymorphic types are a black-box, my preference is 
 certainly for parametric type polymorphism. The main 
 disadvantage with parametric polymorphism in compiled languages 
 is that array containers only operate under subtyping 
 polymorphism. In the above Julia example the array container is 
 essentially acting exactly like a compile-time dispatch 
 (overloaded) function - the array is dispatching on a specific 
 set of types defined by the abstract parent type. That kind of 
 construct would be very desirable to me in D. The closest such 
 thing you can have to that in D are tuples.
I suspect the reason you can't have parametric typed array containers in statically typed compiled languages is that underneath, they are doubly/linked lists, and there is no way of resolving the types at the end of the array, and allowing appending at the same time ... someone correct me if I am wrong.
Aug 30
parent data pulverizer <data.pulverizer gmail.com> writes:
On Wednesday, 30 August 2017 at 18:48:58 UTC, data pulverizer 
wrote:
 I suspect the reason you can't have parametric typed array 
 containers in statically typed compiled languages is that 
 underneath, they are doubly/linked lists, and there is no way 
 of resolving the types at the end of the array, and allowing 
 appending at the same time ... someone correct me if I am wrong.
The only other way would be to create a wrapper interface and classes for each type you want to include in the array, which takes you back to sub-type (OOP) polymorphism.
Aug 30
prev sibling parent reply Jean-Louis Leroy <jl leroy.nyc> writes:
On Wednesday, 30 August 2017 at 17:16:59 UTC, data pulverizer 
wrote:
 On Wednesday, 30 August 2017 at 17:14:37 UTC, data pulverizer 
 wrote:
 On Wednesday, 30 August 2017 at 16:45:19 UTC, data pulverizer 
 wrote:
 You mentioned Julia in your article, however for clarity I 
 would point out that Julia doesn't have OOP-type 
 polymorphism. There is no notion of being able to do 
 something like:

 Animal snoopy = new Dog();
p.s. my bad, I was wrong about that! Turns out you can do something like this in Julia (apologies for the Julia code in a D forum): abstract type Animal end struct Dog <: Animal end struct Cat <: Animal end x = Array{Animal}(3) x[1] = Cat(); x[2] = Dog(); x[3] = Cat(); x # returns 3-element Array{Animal,1}: Cat() Dog() Cat()
p.p.s typeof(x[1]) # returns Cat so it isn't really polymorphism - the object is never converted to the "parent" type! Lol ... sorry for the confusion!
 Which is polymorphism
After mulling over this example, I don't see how this proves that Julia does *not* support run time polymorphism. On the contrary. If you translate this to D you get the same result by the way: import std.stdio; class Animal {} class Cat : Animal {} void main() { Animal[] array; array ~= new Cat(); writeln(typeid(array[0])); // typeid.Cat }
Aug 30
parent reply data pulverizer <data.pulverizer gmail.com> writes:
On Wednesday, 30 August 2017 at 20:40:38 UTC, Jean-Louis Leroy 
wrote:
 After mulling over this example, I don't see how this proves 
 that Julia does *not* support run time polymorphism. On the 
 contrary. If you translate this to D you get the same result by 
 the way:

 import std.stdio;

 class Animal {}
 class Cat : Animal {}

 void main()
 {
   Animal[] array;
   array ~= new Cat();
   writeln(typeid(array[0])); // typeid.Cat
 }
writeln(typeof(array[0]).stringof); // this is an Animal The return type of any item in the array would be of type Animal. You would have to do a type cast to really get type Cat. In Julia there is no real notion that any item in Array{Animal, 1} is an Animal. Their types are never masked by Animal, which only really serves as a way of dispatching types to the array, unlike in D where the actual type is Animal. However if your multi-methods is dispatching methods using typeid (which I am guessing is the case) this distinction no longer matters. It may be better to say that typeof() in Julia is a run-time type and there is no notion of typeof() as in D. In the light of this I think your package just became more interesting to me.
Aug 30
next sibling parent data pulverizer <data.pulverizer gmail.com> writes:
On Wednesday, 30 August 2017 at 21:30:29 UTC, data pulverizer 
wrote:
 On Wednesday, 30 August 2017 at 20:40:38 UTC, Jean-Louis Leroy 
 wrote:
 After mulling over this example, I don't see how this proves 
 that Julia does *not* support run time polymorphism. On the 
 contrary.
In that case you are right! Julia is doing run-time polymorphism and dynamic multiple dispatch ... because there is no sense in which compile time types exist. Functions are pre-compiled but types are always run time. Phew, I think got there in the end
Aug 30
prev sibling parent reply Jean-Louis Leroy <jl leroy.nyc> writes:
On Wednesday, 30 August 2017 at 21:30:29 UTC, data pulverizer 
wrote:
 In the light of this I think your package just became more 
 interesting to me.
I think that your work and mine are complementary :-)
Aug 30
next sibling parent data pulverizer <data.pulverizer gmail.com> writes:
On Wednesday, 30 August 2017 at 22:10:38 UTC, Jean-Louis Leroy 
wrote:
 On Wednesday, 30 August 2017 at 21:30:29 UTC, data pulverizer 
 wrote:
 In the light of this I think your package just became more 
 interesting to me.
I think that your work and mine are complementary :-)
Yes, one of the problems I have been trying to solve is the best way of writing a table structure similar to a data frame in R. Till now polymorphism never really appealed to me but with your package writing methods for such structures become much nicer, a simple prototype could be this: ``` import std.stdio: writeln; class GenericVector{} class Vector(T): GenericVector{ T[] data; this(T[] data) { this.data = data; } } void main() { GenericVector[] myTable = [new Vector!double([1., 2., 3.]), new Vector!string(["a", "b", "c"])]; writeln(typeid(myTable[0])); writeln(typeid(myTable[1])); } ``` Then your openmethods package can dispatch on these types of objects. Very cool!
Aug 30
prev sibling parent reply data pulverizer <data.pulverizer gmail.com> writes:
On Wednesday, 30 August 2017 at 22:10:38 UTC, Jean-Louis Leroy 
wrote:
 On Wednesday, 30 August 2017 at 21:30:29 UTC, data pulverizer 
 wrote:
 In the light of this I think your package just became more 
 interesting to me.
I think that your work and mine are complementary :-)
Here is one strange difference between inheriting from an interface and a class: ``` interface Animal{} class Dog: Animal{} class Cat: Animal{} void main() { Animal[] x; x ~= new Cat(); x ~= new Dog(); x ~= new Cat(); writeln(typeid(x[0])); // Gives Animal } ``` But if Animal is set to a class the typeid gives Cat, why does this happen? Does this mean that inheriting from an interface is not really polymorphism?
Aug 30
next sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Wednesday, 30 August 2017 at 22:30:12 UTC, data pulverizer 
wrote:
 On Wednesday, 30 August 2017 at 22:10:38 UTC, Jean-Louis Leroy 
 wrote:
 On Wednesday, 30 August 2017 at 21:30:29 UTC, data pulverizer 
 wrote:
 In the light of this I think your package just became more 
 interesting to me.
I think that your work and mine are complementary :-)
Here is one strange difference between inheriting from an interface and a class: ``` interface Animal{} class Dog: Animal{} class Cat: Animal{} void main() { Animal[] x; x ~= new Cat(); x ~= new Dog(); x ~= new Cat(); writeln(typeid(x[0])); // Gives Animal } ``` But if Animal is set to a class the typeid gives Cat, why does this happen? Does this mean that inheriting from an interface is not really polymorphism?
Is there a reason you're not using writeln(typeid(typeof(x[0]))); I pretty much always write it that way.
Aug 30
parent reply data pulverizer <data.pulverizer gmail.com> writes:
On Wednesday, 30 August 2017 at 22:49:54 UTC, jmh530 wrote:
 On Wednesday, 30 August 2017 at 22:30:12 UTC, data pulverizer 
 wrote:
 On Wednesday, 30 August 2017 at 22:10:38 UTC, Jean-Louis Leroy 
 wrote:
 On Wednesday, 30 August 2017 at 21:30:29 UTC, data pulverizer 
 wrote:
 In the light of this I think your package just became more 
 interesting to me.
I think that your work and mine are complementary :-)
Here is one strange difference between inheriting from an interface and a class: ``` interface Animal{} class Dog: Animal{} class Cat: Animal{} void main() { Animal[] x; x ~= new Cat(); x ~= new Dog(); x ~= new Cat(); writeln(typeid(x[0])); // Gives Animal } ``` But if Animal is set to a class the typeid gives Cat, why does this happen? Does this mean that inheriting from an interface is not really polymorphism?
Is there a reason you're not using writeln(typeid(typeof(x[0]))); I pretty much always write it that way.
typeid() will give you the run-time type while typeof() gives the declared (compile time) type, typeid(typeof()) will not give you the run-time type - which in our case is what we want if we are using sub-typing polymorphism.
Aug 30
parent jmh530 <john.michael.hall gmail.com> writes:
On Wednesday, 30 August 2017 at 23:45:13 UTC, data pulverizer 
wrote:
 typeid() will give you the run-time type while typeof() gives 
 the declared (compile time) type, typeid(typeof()) will not 
 give you the run-time type - which in our case is what we want 
 if we are using sub-typing polymorphism.
Ah, news to me. Also, I have pushed my branch of the distributions work I've done to my fork of dstats. I haven't submitted a PR yet because I had submitted some other small fixes and want to see how they go (and maybe adjust some of the work I've done in response). https://github.com/jmh530/dstats/tree/jmh530-addDistributions
Aug 30
prev sibling parent reply Jean-Louis Leroy <jl leroy.nyc> writes:
On Wednesday, 30 August 2017 at 22:30:12 UTC, data pulverizer 
wrote:
 On Wednesday, 30 August 2017 at 22:10:38 UTC, Jean-Louis Leroy 
 wrote:
 On Wednesday, 30 August 2017 at 21:30:29 UTC, data pulverizer 
 wrote:
 In the light of this I think your package just became more 
 interesting to me.
I think that your work and mine are complementary :-)
Here is one strange difference between inheriting from an interface and a class: ``` interface Animal{} class Dog: Animal{} class Cat: Animal{} void main() { Animal[] x; x ~= new Cat(); x ~= new Dog(); x ~= new Cat(); writeln(typeid(x[0])); // Gives Animal } ``` But if Animal is set to a class the typeid gives Cat, why does this happen? Does this mean that inheriting from an interface is not really polymorphism?
I noticed that too. Still scratching my head.
Aug 30
next sibling parent Rainer Schuetze <r.sagitario gmx.de> writes:
On 31.08.2017 01:34, Jean-Louis Leroy wrote:
 On Wednesday, 30 August 2017 at 22:30:12 UTC, data pulverizer wrote:
 On Wednesday, 30 August 2017 at 22:10:38 UTC, Jean-Louis Leroy wrote:
 On Wednesday, 30 August 2017 at 21:30:29 UTC, data pulverizer wrote:
 In the light of this I think your package just became more 
 interesting to me.
I think that your work and mine are complementary :-)
Here is one strange difference between inheriting from an interface and a class: ``` interface Animal{} class Dog: Animal{} class Cat: Animal{} void main() {     Animal[] x;     x ~= new Cat();     x ~= new Dog();     x ~= new Cat();     writeln(typeid(x[0])); // Gives Animal } ``` But if Animal is set to a class the typeid gives Cat, why does this happen? Does this mean that inheriting from an interface is not really polymorphism?
I noticed that too. Still scratching my head.
typeid(Interface) has been subject to a number of bugzilla reports, e.g. https://issues.dlang.org/show_bug.cgi?id=13833 and https://issues.dlang.org/show_bug.cgi?id=14612.
Aug 30
prev sibling parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Wednesday, 30 August 2017 at 23:34:10 UTC, Jean-Louis Leroy 
wrote:
 On Wednesday, 30 August 2017 at 22:30:12 UTC, data pulverizer 
 wrote:
 On Wednesday, 30 August 2017 at 22:10:38 UTC, Jean-Louis Leroy 
 wrote:
 On Wednesday, 30 August 2017 at 21:30:29 UTC, data pulverizer 
 wrote:
 In the light of this I think your package just became more 
 interesting to me.
I think that your work and mine are complementary :-)
Here is one strange difference between inheriting from an interface and a class: ``` interface Animal{} class Dog: Animal{} class Cat: Animal{} void main() { Animal[] x; x ~= new Cat(); x ~= new Dog(); x ~= new Cat(); writeln(typeid(x[0])); // Gives Animal } ``` But if Animal is set to a class the typeid gives Cat, why does this happen? Does this mean that inheriting from an interface is not really polymorphism?
I noticed that too. Still scratching my head.
The workaround is to cast to Object before getting the typeid. The cause for this behavior is that if you have an interface reference to an object it points to the interface vtbl and not to the Object base class vtbl. https://run.dlang.io/is/3IMrin
Aug 30
parent Jean-Louis Leroy <jl leroy.nyc> writes:
On Thursday, 31 August 2017 at 06:58:53 UTC, Petar Kirov 
[ZombineDev] wrote:
 The workaround is to cast to Object before getting the typeid. 
 The cause for this behavior is that if you have an interface 
 reference to an object it points to the interface vtbl and not 
 to the Object base class vtbl.
Yeah I know. And in my openmethods lib I simply follow the pointers, without relying on typeid. It does look to me like a bug though. D has all the info it needs to implement the process you describe.
Aug 31
prev sibling parent Jean-Louis Leroy <jl leroy.nyc> writes:
On Wednesday, 30 August 2017 at 16:45:19 UTC, data pulverizer 
wrote:
 One thing that confused me was examples like this ...

  method
 Matrix _plus(DiagonalMatrix a, DiagonalMatrix b)
 {
   // just add the elements on diagonals
   // return a DiagonalMatrix
 }

 Which is marked as returning a DiagonalMatrix rather than a 
 Matrix by polymorphism however the function is marked Matrix 
 return type.
Indeed returning a DiagonalMatrix would work, and is marginally more useful (in case you want to call the specialization directly). I'll update the example. Thanks.
Aug 30
prev sibling parent Jean-Louis Leroy <jl leroy.nyc> writes:
On Thursday, 24 August 2017 at 23:50:21 UTC, data pulverizer 
wrote:
 I find OOP-polymorphic types ultimately unsatisfying, but I 
 don't know of anyway to write, compile and load a D script with 
 new types and methods on the fly into the same session.
That is why binding membership and polymorphism together is a historical wrong turn. CLOS had it right but the world followed the Simula/Smalltalk path because of a nice metaphor (objects sending messages to each other). My openmethods library allows you to add methods "from outside" and also supports dynamic loading: you can add new methods to existing classes and new classes to hierarchies that have methods. See the blog post that just came up.
Aug 28