www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Strategy for Traction

reply Russel Winder <russel winder.org.uk> writes:
As some of you will know Python is big in bioinformatics, HEP, finance,
signal processing and many other computational areas. Originally (and
still to some extent) Python was (and still is) used for coordination of
Fortran and C++ computations, and for rendering the visualizations.
Mathematica, R, Julia, Maple, etc. also have roles in this game.

Those in the Python side of the game disliked C and C++ co much they
created abstractions, ending up with NumPy, Cython, ShedSkin, Numba,
SciPy and also PyPy to replace CPython. All attempts to write Python and
gain native code speed of computation. C is definitely not the right
tool for this. C++14 however may give a new impetus especially using
Boost.Python. The question is whether the anti-C/C++ mindset, pro
NumPy/Cython midset is now so embedded there is no other alternative.

It would have been really nice to have had a D option since writing D
makes writing C++ look like a labour of Sisyphus.

I think it might be a very good idea to ensure that D is a really good
tool for native code sub-systems within a Python system. Basically to
try and remove the NumPy component, and also the Cython, ShedSkin and
Numba ones.

As we know from recent little experiments and email list threads, D can
create C linkage shared libraries which is exactly what is needed for
CPython usage (as PyPy sort of), and there is PyD for create CPython
extensions. With a little tidying via "annotations" all the runtime
initialization can be handled for C linkage shared libraries, PyD
already handles all that for extensions.

In a sense all that is needed is some good examples and thence
marketing.

However I think std.parallelism needs some work: the data parallelism
offered is not yet low enough in overhead to really offer argument-free
competition to NumPy. The goal needs to be for D + std.parallelism to be
as fast of execution as C++ + TBB. Currently it is far from that.

Also we need good ARM support so we can run D on RaspberryPis. OK so the
GPU of a RaspberryPi is great and the CPU dreadful, but they are the
platform of fashion just now. Python is the language of choice, except
for those using Java or C and C++.

So I suggest two directions to go:

1. Create a NumPy replacement.

2. Ensure D can be used on Raspberry Pis.

If there is interest and resource for this then last year was the time
to implement, so time is of the essence ;-)
 
-- 
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Feb 23 2014
next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Russel Winder:

 Those in the Python side of the game disliked C and C++ co much 
 they created abstractions, ending up with NumPy, Cython, 
 ShedSkin,

Unfortunately ShedSkin is now essentially dead. And Julia could replace Python in some usages, because of its performance. Bye, bearophile
Feb 23 2014
parent Paulo Pinto <pjmlp progtools.org> writes:
Am 23.02.2014 20:53, schrieb bearophile:
 Russel Winder:

 Those in the Python side of the game disliked C and C++ co much they
 created abstractions, ending up with NumPy, Cython, ShedSkin,

Unfortunately ShedSkin is now essentially dead. And Julia could replace Python in some usages, because of its performance. Bye, bearophile

And they keep improving it, http://julialang.org/blog/2013/03/efficient-aggregates/
Feb 23 2014
prev sibling next sibling parent Russel Winder <russel winder.org.uk> writes:
On Sun, 2014-02-23 at 19:53 +0000, bearophile wrote:
 Russel Winder:
 
 Those in the Python side of the game disliked C and C++ co much 
 they created abstractions, ending up with NumPy, Cython, 
 ShedSkin,

Unfortunately ShedSkin is now essentially dead. And Julia could replace Python in some usages, because of its performance.

ShedSkin certainly isn't keeping up with the programme and working with Python 3 so I have never tried it. Julia is a system I keep up with, it is splendid. And there is even now a London Julia user group. User group activity is likely a measure of involvement: Java and JVM technology groups have no problem having meetings with attenders. Go went from 20 → 120 attenders per month in a year. Python has activity. Julia has a new user group. I suspect D has no user group activity because it is a 10 year old language very few people have heard about. It needs a marketing reboot so as to be new and exciting and attract people to attend meetings. -- Russel. ============================================================================= Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.net 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Feb 23 2014
prev sibling next sibling parent "francesco cattoglio" <francesco.cattoglio gmail.com> writes:
On Monday, 24 February 2014 at 06:36:33 UTC, Russel Winder wrote:

 I suspect D has no user group activity because it is a 10 year 
 old
 language very few people have heard about. It needs a marketing 
 reboot

Let's make 2 or 3 breaking changes, call it D 3.0, and start spamming it around!:P
Feb 24 2014
prev sibling next sibling parent "Abdulhaq" <alynch4047 gmail.com> writes:
On Monday, 24 February 2014 at 06:36:33 UTC, Russel Winder wrote:
 On Sun, 2014-02-23 at 19:53 +0000, bearophile wrote:
 Russel Winder:
 
 Those in the Python side of the game disliked C and C++ co 
 much they created abstractions, ending up with NumPy, 
 Cython, ShedSkin,

Unfortunately ShedSkin is now essentially dead. And Julia could replace Python in some usages, because of its performance.

ShedSkin certainly isn't keeping up with the programme and working with Python 3 so I have never tried it. Julia is a system I keep up with, it is splendid. And there is even now a London Julia user group. User group activity is likely a measure of involvement: Java and JVM technology groups have no problem having meetings with attenders. Go went from 20 → 120 attenders per month in a year. Python has activity. Julia has a new user group. I suspect D has no user group activity because it is a 10 year old language very few people have heard about. It needs a marketing reboot so as to be new and exciting and attract people to attend meetings.

I've been mainly working with python at work for the last ten years, in aeroengineering. We've built some very large and interesting GUI and non-GUI engineering tools. On the back of that experience I've looked around for languages that address the problems we found (mainly startup speed and lack of static typing with tooling around that) and I ended up here with D. We use numpy, scipy, vtk, Qt and CAD kernel software extensively. From my personal perspective I've noticed a really heavy focus here on C++ refugees - that's just an observation, not a criticism. For instance, there are sometimes questions about what to use as a Set. The answer revolves around how only an incompetent doesn't pay attention to exactly what sort of set they need. However, from a python programmer's perspective we just want a Set to hold a dozen objects, I don't care if it takes 1 nanosecond or 1.5 nanoseconds to run. Set also reads better that RedBlackTree etc. So, I think that D has a lot to offer those wanting to move on from python. With that in mind I'd like to see the equivalent of numpy/scipy etc. for D. At the moment that's just wishful thinking though (IMHO) because it's the people, drive and experience to actually build those tools that is required, and that's really hard to come by.
Feb 24 2014
prev sibling next sibling parent Russel Winder <russel winder.org.uk> writes:
On Sun, 2014-02-23 at 19:53 +0000, bearophile wrote:
[…]
 Unfortunately ShedSkin is now essentially dead. And Julia could 
 replace Python in some usages, because of its performance.

I have it on good authority, i.e. from a person writing a book who has the ShedSkin author as a reviewer, that ShedSkin never took off. It is not dead. This same author may pick up and run with ShedSkin over the summer. Of course there is also Pythran. Same author confirms that there is a lot of leaving Java for Python, not to mention IPython now connects to Julia as well as SciPy. Definitely an ideal time for D to have a high profile as a computational language that is easy to program with and connects easily to Python. -- Russel. ============================================================================= Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.net 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Feb 24 2014
prev sibling next sibling parent "Abdulhaq" <alynch4047 gmail.com> writes:
On Monday, 24 February 2014 at 10:21:52 UTC, Russel Winder wrote:
 On Sun, 2014-02-23 at 19:53 +0000, bearophile wrote:
 […]
 Unfortunately ShedSkin is now essentially dead. And Julia 
 could replace Python in some usages, because of its 
 performance.

I have it on good authority, i.e. from a person writing a book who has the ShedSkin author as a reviewer, that ShedSkin never took off. It is not dead. This same author may pick up and run with ShedSkin over the summer. Of course there is also Pythran. Same author confirms that there is a lot of leaving Java for Python, not to mention IPython now connects to Julia as well as SciPy. Definitely an ideal time for D to have a high profile as a computational language that is easy to program with and connects easily to Python.

ShedSkin never took off but I benched it a few years ago and psyco was just as fast and was much easier to use. It seems to me that the make-python-go-fast community are mainly focussing on pypy which to me is much more promising. Converting python to C++ (like shedskin does) seems doomed to me, given the dynamic typing, JIT seems far better suited.
Feb 24 2014
prev sibling next sibling parent "Rikki Cattermole" <alphaglosined gmail.com> writes:
On Monday, 24 February 2014 at 10:17:53 UTC, Abdulhaq wrote:
 From my personal perspective I've noticed a really heavy focus 
 here on C++ refugees - that's just an observation, not a 
 criticism. For instance, there are sometimes questions about 
 what to use as a Set. The answer revolves around how only an 
 incompetent doesn't pay attention to exactly what sort of set 
 they need. However, from a python programmer's perspective we 
 just want a Set to hold a dozen objects, I don't care if it 
 takes 1 nanosecond or 1.5 nanoseconds to run. Set also reads 
 better that RedBlackTree etc.

 So, I think that D has a lot to offer those wanting to move on 
 from python. With that in mind I'd like to see the equivalent 
 of numpy/scipy etc. for D. At the moment that's just wishful 
 thinking though (IMHO) because it's the people, drive and 
 experience to actually build those tools that is required, and 
 that's really hard to come by.

This semester I'm having to learn Python as part of my degree. Unfortunately. From a period of 5 hours today I have already gleamed this: Python community doesn't really care about 64 bit much; 50 ways to do something? sure but lets not make it really really amazing. Over complicated? not really, needs more in fact. From coming from the D perspective it can be quite challenging for me. I would rather as a community we focus on projects together (which we do quite well already) than split off and do our own thing. The other thing is, making things just work. In as many use cases as possible. I usually go in the deep end when I start learning a language so this probably doesn't reflect the python community completely.
Feb 24 2014
prev sibling next sibling parent "Abdulhaq" <alynch4047 gmail.com> writes:
On Monday, 24 February 2014 at 11:25:18 UTC, Rikki Cattermole 
wrote:
 This semester I'm having to learn Python as part of my degree. 
 Unfortunately.
 From a period of 5 hours today I have already gleamed this:

You youngsters, you learn it all so fast! (Just kidding :-)))
 Python community doesn't really care about 64 bit much;

OK maybe
50 ways to do something? sure

But the python principle is "there's only one way to do it"
 but lets not make it really really amazing.

Perfect is the enemy of good enough :-) Building very large systems is best done like laying out a tableau of cards, you do the first 7, then another 7 on top etc. If you try to get it perfect first time you'll get it wrong and probably never finish it anyway.
 Over complicated? not really, needs more in fact.

Not sure what you mean?
 From coming from the D perspective it can be quite challenging 
 for me. I would rather as a community we focus on projects 
 together (which we do quite well already) than split off and do 
 our own thing.
 The other thing is, making things just work. In as many use 
 cases as possible.

With open source projects each person scratches their own itch - it's really hard to find people who will do what other people want, for free. Having said that, I'm not sure what you mean, can you elaborate?
 I usually go in the deep end when I start learning a language 
 so this probably doesn't reflect the python community 
 completely.

It's a very large heterogeneous community so hard to typecast... I'm very interested in how you get on so keep posting...
Feb 24 2014
prev sibling parent Russel Winder <russel winder.org.uk> writes:
On Mon, 2014-02-24 at 11:25 +0000, Rikki Cattermole wrote:
[…]
  From a period of 5 hours today I have already gleamed this:
 Python community doesn't really care about 64 bit much; 50 ways 
 to do something? sure but lets not make it really really amazing. 
 Over complicated? not really, needs more in fact.

What makes you say Python community don't care about 64-bit? I bet you'll find most of the Python community either don't care about 32-bit/64-bit because they have no need to, or they are using 64-bit already. I'm not sure what the rest of the paragraph is trying to say. Sorry probably my fault.
  From coming from the D perspective it can be quite challenging 
 for me. I would rather as a community we focus on projects 
 together (which we do quite well already) than split off and do 
 our own thing.
 The other thing is, making things just work. In as many use cases 
 as possible.

Historically the Python community has been fiercely libertarian despite the "batteries included" approach of the distribution early on. This leads to many variants for new ideas but this generally sorts itself out.
 I usually go in the deep end when I start learning a language so 
 this probably doesn't reflect the python community completely.

Get involved with the Python community and find out :-) -- Russel. ============================================================================= Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.net 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Feb 24 2014