www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Scientific computing using D

reply ahmat <amthamdan gmail.com> writes:
Hi everyone,
I use mainly Python for scientific computing and I want to switch 
to D but I can't find good libraries as replacement for pandas, 
matplotlib, scipy, ...
Are there plans to make D better in this area?
Mar 16 2020
next sibling parent reply 9il <ilyayaroshenko gmail.com> writes:
On Tuesday, 17 March 2020 at 00:48:24 UTC, ahmat wrote:

 Hi everyone,
 I use mainly Python for scientific computing and I want to 
 switch to D but I can't find good libraries as replacement for 
 pandas, matplotlib, scipy, ...
 Are there plans to make D better in this area?


https://github.com/libmir/mir-algorithm
https://github.com/libmir/mir-random
https://github.com/libmir/mir-blas
https://github.com/libmir/mir-lapack
https://github.com/libmir/mir-optim
https://github.com/libmir/mir-intergral

Mir is not the libraries for scientific research on a PC as well 
as the whole D isn't the best choice for that. Python is better 
for this kind of stuff. If you just like D, it isn't a good 
reason to use it. Use D if you don't have other choices because 
of technical reasons. D offers what other languages can't and 
sometimes it is the only good choice an engineer has.

Ilya
Mar 16 2020
next sibling parent 9il <ilyayaroshenko gmail.com> writes:
On Tuesday, 17 March 2020 at 06:02:25 UTC, 9il wrote:

 https://github.com/libmir/mir-intergral


https://github.com/libmir/mir-integral
Mar 16 2020
prev sibling next sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Tuesday, 17 March 2020 at 06:02:25 UTC, 9il wrote:

 [snip]

 Mir is not the libraries for scientific research on a PC as 
 well as the whole D isn't the best choice for that. Python is 
 better for this kind of stuff. If you just like D, it isn't a 
 good reason to use it. Use D if you don't have other choices 
 because of technical reasons. D offers what other languages 
 can't and sometimes it is the only good choice an engineer has.

 Ilya


I think it is useful for anyone to think about their use case and 
whether the programming language is the best tool to use.

I am often more productive in Python/Matlab/R for small projects 
that do not take long to write and where the code does not need 
to run for a long time. Knowing nothing about what the OP is 
doing, I would say start there.

However, if they are doing something that starts to run into the 
limitations of those languages, maybe they are doing something 
that uses a lot of memory or the code takes a long time to run, 
then they should start thinking about what the bottlenecks are. 
Sometimes with python, you might find the the bottleneck is 
actually some C code that numpy is calling. In which case, the 
improvement in performance may not be so large by using D 
instead. Other times, you can use something like cython to 
improve performance. However, then you start mixing C and Python 
in your code and then you might wonder why not do it in C first?

I considered using D because I was writing code that took several 
hours to run and sometimes ran into memory problems. I also 
didn't want to use C, and I couldn't stand the template system in 
C++. It has been good to learn D. I think I've gotten a better 
handle on many other programming languages because of it. 
However, I'm not sure I've saved as much time as I expected.
Mar 17 2020
parent Ahmat <amthamdan gmail.com> writes:
On Tuesday, 17 March 2020 at 10:28:00 UTC, jmh530 wrote:

 On Tuesday, 17 March 2020 at 06:02:25 UTC, 9il wrote:

 [...]


 I think it is useful for anyone to think about their use case 
 and whether the programming language is the best tool to use.

 [...]


Python is easy to use but difficult to debug, Cython is a 
nightmare, C++ have many good libraries but it’s kind of complex. 
I am still a beginner in D, but if writing high performance code 
in D is easy i am definitely going to drop Python.
Mar 17 2020
prev sibling parent reply Ahmat <amthamdan gmail.com> writes:
On Tuesday, 17 March 2020 at 06:02:25 UTC, 9il wrote:

 On Tuesday, 17 March 2020 at 00:48:24 UTC, ahmat wrote:

 [...]


 https://github.com/libmir/mir-algorithm
 https://github.com/libmir/mir-random
 https://github.com/libmir/mir-blas
 https://github.com/libmir/mir-lapack
 https://github.com/libmir/mir-optim
 https://github.com/libmir/mir-intergral

 Mir is not the libraries for scientific research on a PC as 
 well as the whole D isn't the best choice for that. Python is 
 better for this kind of stuff. If you just like D, it isn't a 
 good reason to use it. Use D if you don't have other choices 
 because of technical reasons. D offers what other languages 
 can't and sometimes it is the only good choice an engineer has.

 Ilya


Thanks you, How is Libmir in term of performance ?
Mar 17 2020
parent reply 9il <ilyayaroshenko gmail.com> writes:
On Tuesday, 17 March 2020 at 11:50:26 UTC, Ahmat wrote:

 On Tuesday, 17 March 2020 at 06:02:25 UTC, 9il wrote:

 On Tuesday, 17 March 2020 at 00:48:24 UTC, ahmat wrote:

 [...]


 https://github.com/libmir/mir-algorithm
 https://github.com/libmir/mir-random
 https://github.com/libmir/mir-blas
 https://github.com/libmir/mir-lapack
 https://github.com/libmir/mir-optim
 https://github.com/libmir/mir-intergral

 Mir is not the libraries for scientific research on a PC as 
 well as the whole D isn't the best choice for that. Python is 
 better for this kind of stuff. If you just like D, it isn't a 
 good reason to use it. Use D if you don't have other choices 
 because of technical reasons. D offers what other languages 
 can't and sometimes it is the only good choice an engineer has.

 Ilya


 Thanks you, How is Libmir in term of performance ?


When compiled with LDC Mir has the same performance as C/C++ code 
compiled with Clang/Clang++. The linear algebra performance 
depends on BLAS/Lapack library that is linked with.
Mar 17 2020
parent Dominikus Dittes Scherkl <dominikus.scherkl continental-corporation.com> writes:
On Tuesday, 17 March 2020 at 12:10:24 UTC, 9il wrote:


 When compiled with LDC Mir has the same performance as C/C++ 
 code compiled with Clang/Clang++. The linear algebra 
 performance depends on BLAS/Lapack library that is linked with.


What he is trying to say is: The performance is about the best 
mankind has developed so far, sometimes + some small epsilon 
which he it about to minimize.
Mar 17 2020
prev sibling next sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Tuesday, 17 March 2020 at 00:48:24 UTC, ahmat wrote:

 Hi everyone,
 I use mainly Python for scientific computing and I want to 
 switch to D but I can't find good libraries as replacement for 
 pandas, matplotlib, scipy, ...
 Are there plans to make D better in this area?



Matplotlib-d:
https://github.com/koji-kojiro/matplotlib-d

Numir:
https://github.com/libmir/numir

There is not currently something as polished as pandas, but there 
has been some progress made on magpie that covers some of the 
functionality of that and the tidyverse:
https://github.com/Kriyszig/magpie
Mar 17 2020
prev sibling next sibling parent reply maarten van damme <maartenvd1994 gmail.com> writes:
Have you considered julia? I like D and even tried to sneak it into as many
uni assignments as possible but the package ecosystem isn't really there.
You will spend a considerable amount of time on non-scientific-computing
tasks such as writing interfaces for other c libraries.

Op di 17 mrt. 2020 om 01:50 schreef ahmat via Digitalmars-d <
digitalmars-d puremagic.com>:


 Hi everyone,
 I use mainly Python for scientific computing and I want to switch
 to D but I can't find good libraries as replacement for pandas,
 matplotlib, scipy, ...
 Are there plans to make D better in this area?
Mar 17 2020
parent Ahmat <amthamdan gmail.com> writes:
On Tuesday, 17 March 2020 at 11:45:20 UTC, maarten van damme 
wrote:

 Have you considered julia? I like D and even tried to sneak it 
 into as many uni assignments as possible but the package 
 ecosystem isn't really there. You will spend a considerable 
 amount of time on non-scientific-computing tasks such as 
 writing interfaces for other c libraries.

 Op di 17 mrt. 2020 om 01:50 schreef ahmat via Digitalmars-d < 
 digitalmars-d puremagic.com>:


 Hi everyone,
 I use mainly Python for scientific computing and I want to 
 switch
 to D but I can't find good libraries as replacement for pandas,
 matplotlib, scipy, ...
 Are there plans to make D better in this area?




I have tried Julia and it’s great for machine learning (Flux.jl) 
and automatic differentiation. I am and undergrad but I will 
consider using it in the future for R&D.
Mar 17 2020
prev sibling parent reply bachmeier <no spam.net> writes:
On Tuesday, 17 March 2020 at 00:48:24 UTC, ahmat wrote:

 Hi everyone,
 I use mainly Python for scientific computing and I want to 
 switch to D but I can't find good libraries as replacement for 
 pandas, matplotlib, scipy, ...
 Are there plans to make D better in this area?


Do you mean writing D libraries that do these things? Probably 
not. That would take a lot of resources and would duplicate work 
already done.

I've done a fair amount of this type of work myself, and IMO D is 
great if you don't mind wrapping C libraries like GSL. That's 
really all Python did in the beginning - it was just a glue 
language. I don't find it all that time consuming due to D's 
great interoperability with other languages, and I'm a lot more 
productive in D than in those other languages. For me, the 
cost-benefit analysis works out in favor of D.

If you want something polished, something that "just works", 
you're better off using Julia. If you want to write libraries to 
make D as convenient to use as Python, it will be welcome. Don't 
hold your breath waiting for others to deliver something. I'd do 
it if I had the time...
Mar 17 2020
parent reply Ahmat <amthamdan gmail.com> writes:
On Tuesday, 17 March 2020 at 18:16:17 UTC, bachmeier wrote:

 On Tuesday, 17 March 2020 at 00:48:24 UTC, ahmat wrote:

 Hi everyone,
 I use mainly Python for scientific computing and I want to 
 switch to D but I can't find good libraries as replacement for 
 pandas, matplotlib, scipy, ...
 Are there plans to make D better in this area?


 Do you mean writing D libraries that do these things? Probably 
 not. That would take a lot of resources and would duplicate 
 work already done.

 I've done a fair amount of this type of work myself, and IMO D 
 is great if you don't mind wrapping C libraries like GSL. 
 That's really all Python did in the beginning - it was just a 
 glue language. I don't find it all that time consuming due to 
 D's great interoperability with other languages, and I'm a lot 
 more productive in D than in those other languages. For me, the 
 cost-benefit analysis works out in favor of D.

 If you want something polished, something that "just works", 
 you're better off using Julia. If you want to write libraries 
 to make D as convenient to use as Python, it will be welcome. 
 Don't hold your breath waiting for others to deliver something. 
 I'd do it if I had the time...


Python just work and that’s what a beginner need from a language. 
You can just run jupyter notebook, import pandas and load a csv 
dataset to a pandas DataFrame. But if you want to write high 
performance code or use advanced things it’s better to use 
another language like C++ and maybe D.
D is great for wrapping C libraries but many python libraries are 
binding of C++. How is the status of interoperability between D 
and C++ ? I am available to help if someone is interested in 
writing some useful libraries in D.

I think Julia is fast and one doesn’t need numpy and others 
libraries to write high performance. It’s gaining users in 
Academia and research even in deep learning.
Mar 17 2020
next sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Tuesday, 17 March 2020 at 18:57:33 UTC, Ahmat wrote:

 [snip]

 Python just work and that’s what a beginner need from a 
 language. You can just run jupyter notebook, import pandas and 
 load a csv dataset to a pandas DataFrame.


I generally find R's data frames easier to use than pandas.
Mar 17 2020
prev sibling next sibling parent reply bachmeier <no spam.net> writes:
On Tuesday, 17 March 2020 at 18:57:33 UTC, Ahmat wrote:


 D is great for wrapping C libraries but many python libraries 
 are binding of C++. How is the status of interoperability 
 between D and C++ ?


Getting better, but I don't use it. There's actually a "backdoor" 
to C++ numerical computing libraries that I use. Many C++ 
libraries can be called from R, but the R interface is nothing 
more than a C interface that can be called directly from D, and 
doesn't actually involve R in any way. So for scientific 
computing, I'd say it's pretty darn good.


 I think Julia is fast and one doesn’t need numpy and others 
 libraries to write high performance. It’s gaining users in 
 Academia and research even in deep learning.


Well, it has something D doesn't: manpower. Although I prefer D, 
there's certainly nothing wrong with Julia. I plan to work on 
calling Julia from D when I have a reason to do so.
Mar 17 2020
parent jmh530 <john.michael.hall gmail.com> writes:
On Tuesday, 17 March 2020 at 19:36:42 UTC, bachmeier wrote:

 [snip]

 Well, it has something D doesn't: manpower. Although I prefer 
 D, there's certainly nothing wrong with Julia. I plan to work 
 on calling Julia from D when I have a reason to do so.


In addition to dpp, symmetry has done some work on this here
https://github.com/symmetryinvestments/juliad
Mar 17 2020
prev sibling parent bachmeier <no spam.net> writes:
On Tuesday, 17 March 2020 at 18:57:33 UTC, Ahmat wrote:

[...]

One other thing I'll add is that my bigger goal is to move us 
away from encouraging people to choose a particular language. D's 
strength is that any code you write can be called from any other 
language. For me, that's far more important than the short-term 
goal of writing code that does what I need it to do this week. 
I've done something useful when I write code in D and others use 
it from their language of choice without ever having heard of D. 
If someone wants to use Python, R, Matlab, Octave, Fortran, or 
Cobol, I still want them to be able to take advantage of the code 
I've already written.
Mar 17 2020