www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Scientific computing using D

reply ahmat <amthamdan gmail.com> writes:
Hi everyone,
I use mainly Python for scientific computing and I want to switch 
to D but I can't find good libraries as replacement for pandas, 
matplotlib, scipy, ...
Are there plans to make D better in this area?
Mar 16
next sibling parent reply 9il <ilyayaroshenko gmail.com> writes:
On Tuesday, 17 March 2020 at 00:48:24 UTC, ahmat wrote:
 Hi everyone,
 I use mainly Python for scientific computing and I want to 
 switch to D but I can't find good libraries as replacement for 
 pandas, matplotlib, scipy, ...
 Are there plans to make D better in this area?
https://github.com/libmir/mir-algorithm https://github.com/libmir/mir-random https://github.com/libmir/mir-blas https://github.com/libmir/mir-lapack https://github.com/libmir/mir-optim https://github.com/libmir/mir-intergral Mir is not the libraries for scientific research on a PC as well as the whole D isn't the best choice for that. Python is better for this kind of stuff. If you just like D, it isn't a good reason to use it. Use D if you don't have other choices because of technical reasons. D offers what other languages can't and sometimes it is the only good choice an engineer has. Ilya
Mar 16
next sibling parent 9il <ilyayaroshenko gmail.com> writes:
On Tuesday, 17 March 2020 at 06:02:25 UTC, 9il wrote:
 https://github.com/libmir/mir-intergral
https://github.com/libmir/mir-integral
Mar 16
prev sibling next sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Tuesday, 17 March 2020 at 06:02:25 UTC, 9il wrote:
 [snip]

 Mir is not the libraries for scientific research on a PC as 
 well as the whole D isn't the best choice for that. Python is 
 better for this kind of stuff. If you just like D, it isn't a 
 good reason to use it. Use D if you don't have other choices 
 because of technical reasons. D offers what other languages 
 can't and sometimes it is the only good choice an engineer has.

 Ilya
I think it is useful for anyone to think about their use case and whether the programming language is the best tool to use. I am often more productive in Python/Matlab/R for small projects that do not take long to write and where the code does not need to run for a long time. Knowing nothing about what the OP is doing, I would say start there. However, if they are doing something that starts to run into the limitations of those languages, maybe they are doing something that uses a lot of memory or the code takes a long time to run, then they should start thinking about what the bottlenecks are. Sometimes with python, you might find the the bottleneck is actually some C code that numpy is calling. In which case, the improvement in performance may not be so large by using D instead. Other times, you can use something like cython to improve performance. However, then you start mixing C and Python in your code and then you might wonder why not do it in C first? I considered using D because I was writing code that took several hours to run and sometimes ran into memory problems. I also didn't want to use C, and I couldn't stand the template system in C++. It has been good to learn D. I think I've gotten a better handle on many other programming languages because of it. However, I'm not sure I've saved as much time as I expected.
Mar 17
parent Ahmat <amthamdan gmail.com> writes:
On Tuesday, 17 March 2020 at 10:28:00 UTC, jmh530 wrote:
 On Tuesday, 17 March 2020 at 06:02:25 UTC, 9il wrote:
 [...]
I think it is useful for anyone to think about their use case and whether the programming language is the best tool to use. [...]
Python is easy to use but difficult to debug, Cython is a nightmare, C++ have many good libraries but it’s kind of complex. I am still a beginner in D, but if writing high performance code in D is easy i am definitely going to drop Python.
Mar 17
prev sibling parent reply Ahmat <amthamdan gmail.com> writes:
On Tuesday, 17 March 2020 at 06:02:25 UTC, 9il wrote:
 On Tuesday, 17 March 2020 at 00:48:24 UTC, ahmat wrote:
 [...]
https://github.com/libmir/mir-algorithm https://github.com/libmir/mir-random https://github.com/libmir/mir-blas https://github.com/libmir/mir-lapack https://github.com/libmir/mir-optim https://github.com/libmir/mir-intergral Mir is not the libraries for scientific research on a PC as well as the whole D isn't the best choice for that. Python is better for this kind of stuff. If you just like D, it isn't a good reason to use it. Use D if you don't have other choices because of technical reasons. D offers what other languages can't and sometimes it is the only good choice an engineer has. Ilya
Thanks you, How is Libmir in term of performance ?
Mar 17
parent reply 9il <ilyayaroshenko gmail.com> writes:
On Tuesday, 17 March 2020 at 11:50:26 UTC, Ahmat wrote:
 On Tuesday, 17 March 2020 at 06:02:25 UTC, 9il wrote:
 On Tuesday, 17 March 2020 at 00:48:24 UTC, ahmat wrote:
 [...]
https://github.com/libmir/mir-algorithm https://github.com/libmir/mir-random https://github.com/libmir/mir-blas https://github.com/libmir/mir-lapack https://github.com/libmir/mir-optim https://github.com/libmir/mir-intergral Mir is not the libraries for scientific research on a PC as well as the whole D isn't the best choice for that. Python is better for this kind of stuff. If you just like D, it isn't a good reason to use it. Use D if you don't have other choices because of technical reasons. D offers what other languages can't and sometimes it is the only good choice an engineer has. Ilya
Thanks you, How is Libmir in term of performance ?
When compiled with LDC Mir has the same performance as C/C++ code compiled with Clang/Clang++. The linear algebra performance depends on BLAS/Lapack library that is linked with.
Mar 17
parent Dominikus Dittes Scherkl <dominikus.scherkl continental-corporation.com> writes:
On Tuesday, 17 March 2020 at 12:10:24 UTC, 9il wrote:

 When compiled with LDC Mir has the same performance as C/C++ 
 code compiled with Clang/Clang++. The linear algebra 
 performance depends on BLAS/Lapack library that is linked with.
What he is trying to say is: The performance is about the best mankind has developed so far, sometimes + some small epsilon which he it about to minimize.
Mar 17
prev sibling next sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Tuesday, 17 March 2020 at 00:48:24 UTC, ahmat wrote:
 Hi everyone,
 I use mainly Python for scientific computing and I want to 
 switch to D but I can't find good libraries as replacement for 
 pandas, matplotlib, scipy, ...
 Are there plans to make D better in this area?
Matplotlib-d: https://github.com/koji-kojiro/matplotlib-d Numir: https://github.com/libmir/numir There is not currently something as polished as pandas, but there has been some progress made on magpie that covers some of the functionality of that and the tidyverse: https://github.com/Kriyszig/magpie
Mar 17
prev sibling next sibling parent reply maarten van damme <maartenvd1994 gmail.com> writes:
Have you considered julia? I like D and even tried to sneak it into as many
uni assignments as possible but the package ecosystem isn't really there.
You will spend a considerable amount of time on non-scientific-computing
tasks such as writing interfaces for other c libraries.

Op di 17 mrt. 2020 om 01:50 schreef ahmat via Digitalmars-d <
digitalmars-d puremagic.com>:

 Hi everyone,
 I use mainly Python for scientific computing and I want to switch
 to D but I can't find good libraries as replacement for pandas,
 matplotlib, scipy, ...
 Are there plans to make D better in this area?
Mar 17
parent Ahmat <amthamdan gmail.com> writes:
On Tuesday, 17 March 2020 at 11:45:20 UTC, maarten van damme 
wrote:
 Have you considered julia? I like D and even tried to sneak it 
 into as many uni assignments as possible but the package 
 ecosystem isn't really there. You will spend a considerable 
 amount of time on non-scientific-computing tasks such as 
 writing interfaces for other c libraries.

 Op di 17 mrt. 2020 om 01:50 schreef ahmat via Digitalmars-d < 
 digitalmars-d puremagic.com>:

 Hi everyone,
 I use mainly Python for scientific computing and I want to 
 switch
 to D but I can't find good libraries as replacement for pandas,
 matplotlib, scipy, ...
 Are there plans to make D better in this area?
I have tried Julia and it’s great for machine learning (Flux.jl) and automatic differentiation. I am and undergrad but I will consider using it in the future for R&D.
Mar 17
prev sibling parent reply bachmeier <no spam.net> writes:
On Tuesday, 17 March 2020 at 00:48:24 UTC, ahmat wrote:
 Hi everyone,
 I use mainly Python for scientific computing and I want to 
 switch to D but I can't find good libraries as replacement for 
 pandas, matplotlib, scipy, ...
 Are there plans to make D better in this area?
Do you mean writing D libraries that do these things? Probably not. That would take a lot of resources and would duplicate work already done. I've done a fair amount of this type of work myself, and IMO D is great if you don't mind wrapping C libraries like GSL. That's really all Python did in the beginning - it was just a glue language. I don't find it all that time consuming due to D's great interoperability with other languages, and I'm a lot more productive in D than in those other languages. For me, the cost-benefit analysis works out in favor of D. If you want something polished, something that "just works", you're better off using Julia. If you want to write libraries to make D as convenient to use as Python, it will be welcome. Don't hold your breath waiting for others to deliver something. I'd do it if I had the time...
Mar 17
parent reply Ahmat <amthamdan gmail.com> writes:
On Tuesday, 17 March 2020 at 18:16:17 UTC, bachmeier wrote:
 On Tuesday, 17 March 2020 at 00:48:24 UTC, ahmat wrote:
 Hi everyone,
 I use mainly Python for scientific computing and I want to 
 switch to D but I can't find good libraries as replacement for 
 pandas, matplotlib, scipy, ...
 Are there plans to make D better in this area?
Do you mean writing D libraries that do these things? Probably not. That would take a lot of resources and would duplicate work already done. I've done a fair amount of this type of work myself, and IMO D is great if you don't mind wrapping C libraries like GSL. That's really all Python did in the beginning - it was just a glue language. I don't find it all that time consuming due to D's great interoperability with other languages, and I'm a lot more productive in D than in those other languages. For me, the cost-benefit analysis works out in favor of D. If you want something polished, something that "just works", you're better off using Julia. If you want to write libraries to make D as convenient to use as Python, it will be welcome. Don't hold your breath waiting for others to deliver something. I'd do it if I had the time...
Python just work and that’s what a beginner need from a language. You can just run jupyter notebook, import pandas and load a csv dataset to a pandas DataFrame. But if you want to write high performance code or use advanced things it’s better to use another language like C++ and maybe D. D is great for wrapping C libraries but many python libraries are binding of C++. How is the status of interoperability between D and C++ ? I am available to help if someone is interested in writing some useful libraries in D. I think Julia is fast and one doesn’t need numpy and others libraries to write high performance. It’s gaining users in Academia and research even in deep learning.
Mar 17
next sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Tuesday, 17 March 2020 at 18:57:33 UTC, Ahmat wrote:
 [snip]

 Python just work and that’s what a beginner need from a 
 language. You can just run jupyter notebook, import pandas and 
 load a csv dataset to a pandas DataFrame.
I generally find R's data frames easier to use than pandas.
Mar 17
prev sibling next sibling parent reply bachmeier <no spam.net> writes:
On Tuesday, 17 March 2020 at 18:57:33 UTC, Ahmat wrote:

 D is great for wrapping C libraries but many python libraries 
 are binding of C++. How is the status of interoperability 
 between D and C++ ?
Getting better, but I don't use it. There's actually a "backdoor" to C++ numerical computing libraries that I use. Many C++ libraries can be called from R, but the R interface is nothing more than a C interface that can be called directly from D, and doesn't actually involve R in any way. So for scientific computing, I'd say it's pretty darn good.
 I think Julia is fast and one doesn’t need numpy and others 
 libraries to write high performance. It’s gaining users in 
 Academia and research even in deep learning.
Well, it has something D doesn't: manpower. Although I prefer D, there's certainly nothing wrong with Julia. I plan to work on calling Julia from D when I have a reason to do so.
Mar 17
parent jmh530 <john.michael.hall gmail.com> writes:
On Tuesday, 17 March 2020 at 19:36:42 UTC, bachmeier wrote:
 [snip]

 Well, it has something D doesn't: manpower. Although I prefer 
 D, there's certainly nothing wrong with Julia. I plan to work 
 on calling Julia from D when I have a reason to do so.
In addition to dpp, symmetry has done some work on this here https://github.com/symmetryinvestments/juliad
Mar 17
prev sibling parent bachmeier <no spam.net> writes:
On Tuesday, 17 March 2020 at 18:57:33 UTC, Ahmat wrote:

[...]

One other thing I'll add is that my bigger goal is to move us 
away from encouraging people to choose a particular language. D's 
strength is that any code you write can be called from any other 
language. For me, that's far more important than the short-term 
goal of writing code that does what I need it to do this week. 
I've done something useful when I write code in D and others use 
it from their language of choice without ever having heard of D. 
If someone wants to use Python, R, Matlab, Octave, Fortran, or 
Cobol, I still want them to be able to take advantage of the code 
I've already written.
Mar 17