## digitalmars.D - Matrix API support - start with formats?

- Andrei Alexandrescu (10/10) Aug 14 2015 I stumbled upon https://software.intel.com/en-us/node/471374 which gives...
- Rikki Cattermole (2/12) Aug 14 2015 What would we gain other gl3n in Phobos?
- ponce (5/16) Aug 14 2015 Are sparse matrices a common scenario?
- Andrei Alexandrescu (4/20) Aug 14 2015 In machine learning, yes. But order is not as important as the basic
- Tofu Ninja (2/22) Aug 14 2015 +1
- David Nadlinger (9/12) Aug 14 2015 Yes. They tend to pop up in virtually all "serious" numerical
- Tofu Ninja (4/17) Aug 14 2015 Personally I would prefer small fixed sized vectors & matrices
- Manu via Digitalmars-d (3/26) Aug 20 2015 I think gamedev's would unanimously agree on this.
- Xavier Bigand (10/38) Aug 21 2015 Those kind of maths aren't used only for games, their is also editing
- jmh530 (20/27) Aug 14 2015 While I'm interested in working with big (500,000+) dense (and
- ponce (3/15) Aug 14 2015 One single API would be great in that case. If someone can pull
- bachmeier (11/22) Aug 14 2015 One concern I have is the choice of MKL, which due to cost and
- jmh530 (7/16) Aug 14 2015 I agree. MKL is expensive. OpenBLAS is supposed have comparable
- jmh530 (99/110) Aug 14 2015 I am very excited that there is a greater focus on this.
- Shachar Shemesh (7/16) Aug 15 2015 Please please please make them templated on the type, with clear
- Daniel Shapero (25/33) Aug 15 2015 For comparison, some other sparse linear algebra libraries worth
- dominik (39/39) Aug 21 2015 coming from a computer graphics and image processing background
- dominik (2/4) Aug 21 2015 vec2, vec3, vec4, mat2, mat3, mat4 was what i ment obviously
- ZombineDev (12/23) Sep 01 2015 One thing that will make D really shine is to implement something
- Laeeth Isharc (4/31) Sep 02 2015 I was looking at blaze the other day and wondering just the same.
- jmh530 (5/8) Sep 02 2015 Thanks for posting that. It provides a nuanced discussion of

I stumbled upon https://software.intel.com/en-us/node/471374 which gives good detail on Intel's Math Kernel Library's data formats for sparse matrices. No doubt other popular linear algebra libraries have similar documentation. I was thinking we could start with adding these layouts to std, along with a few simple primitives (construction, element/slice access, stride etc). Then, people may just use those as they are or link with the linalg libraries for specific computations. Thoughts? Andrei

Aug 14 2015

On 15/08/2015 2:57 a.m., Andrei Alexandrescu wrote:I stumbled upon https://software.intel.com/en-us/node/471374 which gives good detail on Intel's Math Kernel Library's data formats for sparse matrices. No doubt other popular linear algebra libraries have similar documentation. I was thinking we could start with adding these layouts to std, along with a few simple primitives (construction, element/slice access, stride etc). Then, people may just use those as they are or link with the linalg libraries for specific computations. Thoughts? AndreiWhat would we gain other gl3n in Phobos?

Aug 14 2015

On Friday, 14 August 2015 at 14:57:19 UTC, Andrei Alexandrescu wrote:I stumbled upon https://software.intel.com/en-us/node/471374 which gives good detail on Intel's Math Kernel Library's data formats for sparse matrices. No doubt other popular linear algebra libraries have similar documentation. I was thinking we could start with adding these layouts to std, along with a few simple primitives (construction, element/slice access, stride etc). Then, people may just use those as they are or link with the linalg libraries for specific computations. Thoughts? AndreiAre sparse matrices a common scenario? If anything I think small vectors, non-sparse matrixes and rectangles/AABB could be part of Phobos before that.

Aug 14 2015

On 8/14/15 11:11 AM, ponce wrote:On Friday, 14 August 2015 at 14:57:19 UTC, Andrei Alexandrescu wrote:In machine learning, yes. But order is not as important as the basic strategy: layouts + simple/naive primitives in Phobos, elaborate primitives in specialized external libraries. -- Andrei

Aug 14 2015

On Friday, 14 August 2015 at 15:11:39 UTC, ponce wrote:On Friday, 14 August 2015 at 14:57:19 UTC, Andrei Alexandrescu wrote:+1

Aug 14 2015

On Friday, 14 August 2015 at 15:11:39 UTC, ponce wrote:Are sparse matrices a common scenario?Yes. They tend to pop up in virtually all "serious" numerical problems in science and engineering, particularly when partial differential equations are involved.If anything I think small vectors, non-sparse matrixes and rectangles/AABB could be part of Phobos before that.If you just had a go at it from the CG/gamedev perspective, you'd probably end up with an API that is entirely unsuitable for larger problems. This might not be a big issue (just have two separate APIs), but we'd need to make it a conscious decision. — David

Aug 14 2015

On Friday, 14 August 2015 at 18:51:51 UTC, David Nadlinger wrote:On Friday, 14 August 2015 at 15:11:39 UTC, ponce wrote:Personally I would prefer small fixed sized vectors & matrices with game dev focused api be kept separate from a big/sparse matrix api.Are sparse matrices a common scenario?Yes. They tend to pop up in virtually all "serious" numerical problems in science and engineering, particularly when partial differential equations are involved.If anything I think small vectors, non-sparse matrixes and rectangles/AABB could be part of Phobos before that.If you just had a go at it from the CG/gamedev perspective, you'd probably end up with an API that is entirely unsuitable for larger problems. This might not be a big issue (just have two separate APIs), but we'd need to make it a conscious decision. — David

Aug 14 2015

On 15 August 2015 at 05:11, Tofu Ninja via Digitalmars-d <digitalmars-d puremagic.com> wrote:On Friday, 14 August 2015 at 18:51:51 UTC, David Nadlinger wrote:I think gamedev's would unanimously agree on this.On Friday, 14 August 2015 at 15:11:39 UTC, ponce wrote:Personally I would prefer small fixed sized vectors & matrices with game dev focused api be kept separate from a big/sparse matrix api.Are sparse matrices a common scenario?Yes. They tend to pop up in virtually all "serious" numerical problems in science and engineering, particularly when partial differential equations are involved.If anything I think small vectors, non-sparse matrixes and rectangles/AABB could be part of Phobos before that.If you just had a go at it from the CG/gamedev perspective, you'd probably end up with an API that is entirely unsuitable for larger problems. This might not be a big issue (just have two separate APIs), but we'd need to make it a conscious decision. — David

Aug 20 2015

Le 21/08/2015 02:30, Manu via Digitalmars-d a écrit :On 15 August 2015 at 05:11, Tofu Ninja via Digitalmars-d <digitalmars-d puremagic.com> wrote:Those kind of maths aren't used only for games, their is also editing software,... Their is also spacial relations that can be interesting to have in a geometry module. Boost do it and respect those rules : http://edndoc.esri.com/arcsde/9.0/general_topics/understand_spatial_relations.htm For my job it help us to solve a lot of precision issues. Having this in the standard library is important IMO cause sadly their is often a lot of bugs/issues when reimplemented handly. A lot of articles on Internet are just wrong.On Friday, 14 August 2015 at 18:51:51 UTC, David Nadlinger wrote:I think gamedev's would unanimously agree on this.Are sparse matrices a common scenario?Yes. They tend to pop up in virtually all "serious" numerical problems in science and engineering, particularly when partial differential equations are involved.

Aug 21 2015

On Friday, 14 August 2015 at 18:51:51 UTC, David Nadlinger wrote:While I'm interested in working with big (500,000+) dense (and sometimes smaller sparse) rectangular matrices, my sense of the D forum is that there are more CG/gamedev people using D. Nevertheless, I think it is important to get the general structure right in a way that can be specialized to work with small matrices, rectangular matrices, strided matrices, or sparse matrices. I don't see an issue with prioritizing what users want in terms of fleshing out the APIs, so long as the different ways people use matrix libraries is in the back of the designers minds when they design it. I was looking over some of the OpenGL matrix libraries recently. They have types like mat4 or vec3. I couldn't help but think that template constraints (at least for static arrays) would probably make it much easier. For instance, if you have a function that takes a vec, you can specialize it to do different things depending on if the array is small or large (small, unroll loops, large use parallelism, maybe with the decision on when stop unrolling loops or starting to use parallelism determined by AEOS, as in ATLAS).

Aug 14 2015

On Friday, 14 August 2015 at 18:51:51 UTC, David Nadlinger wrote:On Friday, 14 August 2015 at 15:11:39 UTC, ponce wrote:One single API would be great in that case. If someone can pull it off.Are sparse matrices a common scenario?Yes. They tend to pop up in virtually all "serious" numerical problems in science and engineering, particularly when partial differential equations are involved.

Aug 14 2015

On Friday, 14 August 2015 at 14:57:19 UTC, Andrei Alexandrescu wrote:

Aug 14 2015

On Friday, 14 August 2015 at 19:19:24 UTC, bachmeier wrote:One concern I have is the choice of MKL, which due to cost and license reasons, many developers will not have on all (or even any) of their machines. I don't work with sparse matrices often so I do not know which libraries are most popular, but at a minimum I think it is necessary to say you can use it with a popular open source library. Given the importance of MKL, it would be a bad idea to not offer a compatible format, but it would be equally bad to focus on only MKL.I agree. MKL is expensive. OpenBLAS is supposed have comparable performance, more or less, and it is free. Alternately, ATLAS can be used to build BLAS on many different systems. I would also distinguish between the low level API like BLAS and OpenBLAS and higher level APIs. Good higher level APIs allow drop-in replacement of lower level APIs (e.g. Armadillo).

Aug 14 2015

On Friday, 14 August 2015 at 20:23:00 UTC, jmh530 wrote:I agree. MKL is expensive. OpenBLAS is supposed have comparable performance, more or less, and it is free. Alternately, ATLAS can be used to build BLAS on many different systems. I would also distinguish between the low level API like BLAS and OpenBLAS and higher level APIs. Good higher level APIs allow drop-in replacement of lower level APIs (e.g. Armadillo).It looks like AMD has open-sourced some of their OpenCL math libraries as well. https://github.com/clMathLibraries/clBLAS https://github.com/clMathLibraries/clRNG https://github.com/clMathLibraries/clSPARSE https://github.com/clMathLibraries/clFFT They also made some available through https://github.com/flame

Aug 20 2015

On Thursday, 20 August 2015 at 21:28:10 UTC, jmh530 wrote:https://github.com/clMathLibraries/clBLAS https://github.com/clMathLibraries/clRNG https://github.com/clMathLibraries/clSPARSE https://github.com/clMathLibraries/clFFT They also made some available through https://github.com/flameI probably should have also highlighted the BLIS project in the FLAME repo. https://code.google.com/p/blis/

Aug 20 2015

On Friday, 14 August 2015 at 14:57:19 UTC, Andrei Alexandrescu wrote:

Aug 14 2015

On 14/08/15 17:57, Andrei Alexandrescu wrote:I stumbled upon https://software.intel.com/en-us/node/471374 which gives good detail on Intel's Math Kernel Library's data formats for sparse matrices. No doubt other popular linear algebra libraries have similar documentation. I was thinking we could start with adding these layouts to std, along with a few simple primitives (construction, element/slice access, stride etc). Then, people may just use those as they are or link with the linalg libraries for specific computations. Thoughts?Please please please make them templated on the type, with clear instructions what the type needs to support in order to work. I've recently tried to find an implementation of matrix inversion over a Z2 field, and found that that NONE of the standard implementations supports that. Shachar

Aug 15 2015

On Friday, 14 August 2015 at 14:57:19 UTC, Andrei Alexandrescu wrote:No doubt other popular linear algebra libraries have similar documentation. I was thinking we could start with adding these layouts to std, along with a few simple primitives (construction, element/slice access, stride etc). Then, people may just use those as they are or link with the linalg libraries for specific computations. Thoughts? AndreiFor comparison, some other sparse linear algebra libraries worth looking at are: * Eigen (http://eigen.tuxfamily.org/) * PETSc (http://www.mcs.anl.gov/petsc/) * the Epetra sub-package of Trilinos (https://trilinos.org/) * OSKI, which aims to do for sparse matrices what ATLAS does for dense (http://bebop.cs.berkeley.edu/oski/) MKL is missing some important sparse matrix formats -- the ellpack and jagged diagonal formats, which are very well suited for SIMD processors, and the MSR/MSC format, which makes multigrid smoothing faster. I don't know enough to weigh in on whether sparse matrix algebra is best left to the individual libraries or put in the language's standard library. It's common practice whenever one needs sparse matrices to write wrapper classes that can use either PETSc or Trilinos, depending on what the users have installed on their systems. For example, this is the approach taken in the finite element library deal.II (http://dealii.org/). Moreover, the big sparse matrix libraries can all use each other; PETSc can link to the solvers in Trilinos, both PETSc and Trilinos can link to OSKI, etc. If there were some support in the standard library for this functionality, it could obviate the need for some of the glue code in C/C++.

Aug 15 2015

coming from a computer graphics and image processing background i'd love to have a standard way to deal with matrices. But matrices come in a lot of different flavors, which need different implementations. - computer graphics applications need mostly 1d-4d vectors and matrices. encapsulation of transformation types: rigid/similarity/affine/linear/projective give the opportunity to choose simpler algorithms for e.g. inversion of a matrix - images are usually represented as dense 2d matrix (or 3d if multiple colors channels are present) - equation systems use dense and sparse matrices, and use lots of different data structures to be optimal for different algorithms. yale dok lil coo to name some sparse formats. mathematically they have different properties like square/rectangular positive/negative definit diagonal - matlab style matrices for convenience it should be possible to resize, and concatenate matrices in different dimensions, e.g. to compose images or to augment equation systems on top of that possibilities to solve those equation systems would be handy: with c++ i mostly use eigen, opennl or ceres this was just a list of things i thought should be kept in mind when implementing a matrix api - the matrix api doesnt have to implement all this imho the matrix api could provide some basic types and a general interface to access such matrices. for start a simple dense and sparse n-dimensional matrix format with the possibility to construct and change matrices. slicing in n-dimensions as simple as in matlab would be awesome. such an api would also be a great base for writing graph based algorithms like markov random fields ... hopes and wishes :) best dominik

Aug 21 2015

On Friday, 21 August 2015 at 11:31:16 UTC, dominik wrote:- computer graphics applications need mostly 1d-4d vectors and matrices.vec2, vec3, vec4, mat2, mat3, mat4 was what i ment obviously

Aug 21 2015

On Friday, 14 August 2015 at 14:57:19 UTC, Andrei Alexandrescu wrote:

Sep 01 2015

On Tuesday, 1 September 2015 at 14:00:52 UTC, ZombineDev wrote:On Friday, 14 August 2015 at 14:57:19 UTC, Andrei Alexandrescu wrote:I was looking at blaze the other day and wondering just the same. I haven't used it, and don't have a great feeling for what would be involved. Maybe John Colvin does.

Sep 02 2015

On Tuesday, 1 September 2015 at 14:00:52 UTC, ZombineDev wrote:One thing that will make D really shine is to implement something like this: https://www.youtube.com/watch?v=hfn0BVOegacThanks for posting that. It provides a nuanced discussion of expression templates. The final conclusion is that C++ should be very careful in choosing a linear algebra library for the standard. I think that applies to D just as well.

Sep 02 2015