www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Mir Algorithm preview: the new ndslice, dlang Iterators, fast generics

reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
Hi all,

This is early preview before the announce of Mir Algorithm and 
the new ndslice. Ndslice is a dlang package for multidimensional 
and numeric worlds.

The new ndslice has three kinds:

1. Universal:
     numpy-like, similar to the old ndslice,

2. Canonical:
     BLAS-like, raw stride is assumed to be 1

3. Contiguous:
     Contiguous in memory, no strides are required.

New ndslice is 90% reworked.

Mir Algorithm has its own map, retro, stride, reduce, naryFun, 
bitwise, to and others stuff from Phobos. But they are 
implemented differently, may have different API (like iota), and 
may boost your program and compilation speed few times.

Ah, ndslice is based on iterators! Dlang iterators! Dlang 
iterators are only random access, other kinds of iterators can be 
replaced with D Ranges. There are strong reasons why iterators 
are the best for random access and multidimensional worlds. They 
will be described in the future this year. Do not worry, we do 
not need to write C++-like code, iterators are used internally by 
ndslices :-)

We have full backward compatibility with Phobos Range API, so we 
can mix Phobos and Mir code. Iterators are useful if we want to 
implement custom and fast ndslices. Big collection of predefined 
ndslices can be found at mir.ndslice.topology .

Mir Algorithm is already used in Tamedia's lincount (for bitwise 
and accelerated bit count), Mir main repository. A PR with update 
for the D Computer Vision library is 85% ready.

Docs
http://docs.algorithm.dlang.io
======================

Github
https://github.com/libmir/mir-algorithm
======================

Dub
http://code.dlang.org/packages/mir-algorithm
======================

Feedback is welcome!

For commercial support: ilyayaroshenko at gmail dot com

Best regards,
Ilya
Feb 07
next sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Tuesday, 7 February 2017 at 16:18:20 UTC, Ilya Yaroshenko 
wrote:
 Feedback is welcome!
I don't have time right now to play around with it, but I looked through the documentation. Here are my comments: 1) I would consider http://docs.algorithm.dlang.io/latest/mir_ndslice.html to be the first thing that anyone looks at when they think about mir ndslice. I think it's pretty good, but there's always room for some improvements, such as a) The extent of the compatibility between D's standard library and mir (such as what you wrote in the post above) b) Some of your high level explanation for the motivation for recent changes, such as iterators, universal/canonical/contigous. I.e., something along the lines of ndslice is so powerful it can do all these different things. 2) The explanation of the difference between universal, canonical, and contiguous is in http://docs.algorithm.dlang.io/latest/mir_ndslice_slice.html#.Slice It should be referenced in functions like Universal/Canonical/Contigous/universal/canonical. 3) mir.ndslice.topology has universal, canonical, assumeCanonical, and assumeContiguous. Why no contiguous or assumeUniversal? 4) It looks like some of the functions from std.algorithm, like map, are now in mir.ndslice.topology instead of mir.ndslice.algorithm. You might include more detailed explanations at the top of the documentation of topology and algorithm for why the modules are split up this way. Keep up the good work.
Feb 08
next sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Wednesday, 8 February 2017 at 15:33:25 UTC, jmh530 wrote:
 I don't have time right now to play around with it, but I 
 looked through the documentation. Here are my comments:
Also on #2, you might beef that link up with some of the comments from the post up top.
Feb 08
prev sibling parent Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Wednesday, 8 February 2017 at 15:33:25 UTC, jmh530 wrote:
 On Tuesday, 7 February 2017 at 16:18:20 UTC, Ilya Yaroshenko 
 wrote:
 Feedback is welcome!
I don't have time right now to play around with it, but I looked through the documentation. Here are my comments: 1) I would consider http://docs.algorithm.dlang.io/latest/mir_ndslice.html to be the first thing that anyone looks at when they think about mir ndslice. I think it's pretty good, but there's always room for some improvements, such as a) The extent of the compatibility between D's standard library and mir (such as what you wrote in the post above) b) Some of your high level explanation for the motivation for recent changes, such as iterators, universal/canonical/contigous. I.e., something along the lines of ndslice is so powerful it can do all these different things. 2) The explanation of the difference between universal, canonical, and contiguous is in http://docs.algorithm.dlang.io/latest/mir_ndslice_slice.html#.Slice It should be referenced in functions like Universal/Canonical/Contigous/universal/canonical. 3) mir.ndslice.topology has universal, canonical, assumeCanonical, and assumeContiguous. Why no contiguous or assumeUniversal? 4) It looks like some of the functions from std.algorithm, like map, are now in mir.ndslice.topology instead of mir.ndslice.algorithm. You might include more detailed explanations at the top of the documentation of topology and algorithm for why the modules are split up this way. Keep up the good work.
1, 2, 4 Agreed. I wonder if ndslice (any) is used by companies, if so, i can invest more my time to it. Will post a questionary in announce. 3) contiguous and canonical can be always safely converted to universal. Plus, contiguous can be always safely converted to canonical. Safe: contiguous -> canonical -> universal Unsafe (has assume prefix) universal -> canonical -> contiguous The ndslice composes all kind of random access ranges you may have in real life with one interface, one declaration. And this is faster to develop (2-10 times less LOC), compile, and execute because of implementation techniques. In addition, it is multidimensional. bitwise, zip, map are good examples to compare Mir and Phobos.
Feb 08
prev sibling parent Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
 2. Canonical:
     BLAS-like, raw stride is assumed to be 1
EDIT: row stride
Feb 08