www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Plotting Using PLPlot

reply dsimcha <dsimcha yahoo.com> writes:
As the scientific computing libraries for D improve, I find myself wanting
more and more to be able to plot stuff straight from D without having to rely
on kludges like writing data out to a file and then calling Python or Matlab
or something.  I've noticed that PLPlot has D bindings.  Its license is also
reasonably permissive (LGPL).  This is certainly an improvement over nothing,
but the API kind of sucks because it was written in C.  For example, instead
of ranges or D arrays of arbitrary type, you pass data in as a double* and a
number of data points.

On the other hand, all the nitty-gritty, low-level, probably
platform-specific, stuff needed for a plotting library is (I guess) pretty
good.  This led me to the following idea for how to get a good D plotting lib
for relatively few man-hours:  Take the low-level stuff from PLPlot, and
reimplement the higher level stuff on top of it in pure D, using the full
power of templates, ranges, builtin arrays, etc.  I'm considering making this
my next hobby project, and I'm interested in some suggestions on how it should
be done (what a good API would be, etc.), as well as getting an idea of how
many people are interested in something like this.
May 09 2009
next sibling parent BCS <none anon.com> writes:
Hello dsimcha,

 I'm
 interested in some suggestions on how it should be done (what a good
 API would be, etc.), as well as getting an idea of how many people are
 interested in something like this.
 

I'm interested. Make simple stuff simple once you have the data. Ploting n lines should be about 2 + n lines of code. ploting a polygon should be a single line. A cool feature would be a templated front end that lest you process data from any data type that you can tell it how to get X/Y data out of.
May 09 2009
prev sibling next sibling parent Don <nospam nospam.com> writes:
dsimcha wrote:
 As the scientific computing libraries for D improve, I find myself wanting
 more and more to be able to plot stuff straight from D without having to rely
 on kludges like writing data out to a file and then calling Python or Matlab
 or something.  I've noticed that PLPlot has D bindings.  Its license is also
 reasonably permissive (LGPL).  This is certainly an improvement over nothing,
 but the API kind of sucks because it was written in C.  For example, instead
 of ranges or D arrays of arbitrary type, you pass data in as a double* and a
 number of data points.
 
 On the other hand, all the nitty-gritty, low-level, probably
 platform-specific, stuff needed for a plotting library is (I guess) pretty
 good.  This led me to the following idea for how to get a good D plotting lib
 for relatively few man-hours:  Take the low-level stuff from PLPlot, and
 reimplement the higher level stuff on top of it in pure D, using the full
 power of templates, ranges, builtin arrays, etc.  I'm considering making this
 my next hobby project, and I'm interested in some suggestions on how it should
 be done (what a good API would be, etc.), as well as getting an idea of how
 many people are interested in something like this.

Definitely. I was considering doing vaguely the same thing (replacing my hacky WinAPI plot library which I've used up to now with a PLPlot binding). It's probably the biggest thing lacking in D's support for scientific computing.
May 09 2009
prev sibling next sibling parent Frank Benoit <keinfarbton googlemail.com> writes:
What is needed to add a dwt device driver?
May 09 2009
prev sibling parent reply Fawzi Mohamed <fmohamed mac.com> writes:
On 2009-05-10 05:19:53 +0200, dsimcha <dsimcha yahoo.com> said:

 As the scientific computing libraries for D improve, I find myself wanting
 more and more to be able to plot stuff straight from D without having to rely
 on kludges like writing data out to a file and then calling Python or Matlab
 or something.  I've noticed that PLPlot has D bindings.  Its license is also
 reasonably permissive (LGPL).  This is certainly an improvement over nothing,
 but the API kind of sucks because it was written in C.  For example, instead
 of ranges or D arrays of arbitrary type, you pass data in as a double* and a
 number of data points.
 
 On the other hand, all the nitty-gritty, low-level, probably
 platform-specific, stuff needed for a plotting library is (I guess) pretty
 good.  This led me to the following idea for how to get a good D plotting lib
 for relatively few man-hours:  Take the low-level stuff from PLPlot, and
 reimplement the higher level stuff on top of it in pure D, using the full
 power of templates, ranges, builtin arrays, etc.  I'm considering making this
 my next hobby project, and I'm interested in some suggestions on how it should
 be done (what a good API would be, etc.), as well as getting an idea of how
 many people are interested in something like this.

This is definitely very interesting, having an integrated plot would be very nice Fawzi
May 10 2009
parent reply Bill Baxter <wbaxter gmail.com> writes:
A D-ish wrapper around PLPlot's low-level D-to-C bindings sounds great
to me too.
I frequently use the D -> data file -> Python matplotlib  route
myself.  Something more direct would be great.

--bb

On Sun, May 10, 2009 at 5:51 AM, Fawzi Mohamed <fmohamed mac.com> wrote:
 On 2009-05-10 05:19:53 +0200, dsimcha <dsimcha yahoo.com> said:

 As the scientific computing libraries for D improve, I find myself wanti=


 more and more to be able to plot stuff straight from D without having to
 rely
 on kludges like writing data out to a file and then calling Python or
 Matlab
 or something. =A0I've noticed that PLPlot has D bindings. =A0Its license=


 also
 reasonably permissive (LGPL). =A0This is certainly an improvement over
 nothing,
 but the API kind of sucks because it was written in C. =A0For example,
 instead
 of ranges or D arrays of arbitrary type, you pass data in as a double* a=


 a
 number of data points.

 On the other hand, all the nitty-gritty, low-level, probably
 platform-specific, stuff needed for a plotting library is (I guess) pret=


 good. =A0This led me to the following idea for how to get a good D plott=


 lib
 for relatively few man-hours: =A0Take the low-level stuff from PLPlot, a=


 reimplement the higher level stuff on top of it in pure D, using the ful=


 power of templates, ranges, builtin arrays, etc. =A0I'm considering maki=


 this
 my next hobby project, and I'm interested in some suggestions on how it
 should
 be done (what a good API would be, etc.), as well as getting an idea of
 how
 many people are interested in something like this.

This is definitely very interesting, having an integrated plot would be v=

 nice

 Fawzi

May 10 2009
parent reply dsimcha <dsimcha yahoo.com> writes:
It seems like there's substantial interest in this.  Please give me some use
cases, i.e. what would you personally use this for, and what do you foresee
others
using it for, so I can start thinking about what the API should be.  I need a
wide
variety of use cases because, if I design the API based only on my personal use
cases, it will end up being geared entirely toward histograms, scatter plots,
and
line graphs because that's what I use regularly.

Besides use cases, here are some specific questions:
1.  Is there any need for an OO-based API, or should I just use free functions?
2.  Does anyone have any use cases where plotting is performance critical, or
should I just keep things simple/stupid in terms of the performance/simplicity
tradeoff?
May 10 2009
next sibling parent Walter Bright <newshound1 digitalmars.com> writes:
dsimcha wrote:
 Besides use cases, here are some specific questions:
 1.  Is there any need for an OO-based API, or should I just use free functions?
 2.  Does anyone have any use cases where plotting is performance critical, or
 should I just keep things simple/stupid in terms of the performance/simplicity
 tradeoff?

I'd stick with (2) for now.
May 10 2009
prev sibling next sibling parent reply Fawzi Mohamed <fmohamed mac.com> writes:
On 2009-05-10 21:23:48 +0200, dsimcha <dsimcha yahoo.com> said:

 It seems like there's substantial interest in this.  Please give me some use
 cases, i.e. what would you personally use this for, and what do you 
 foresee others
 using it for, so I can start thinking about what the API should be.  I 
 need a wide
 variety of use cases because, if I design the API based only on my personal use
 cases, it will end up being geared entirely toward histograms, scatter 
 plots, and
 line graphs because that's what I use regularly.

yep me too, well 3d surface plots would also be nice to have, but I can live without.
 Besides use cases, here are some specific questions:
 1.  Is there any need for an OO-based API, or should I just use free functions?

I would use an OO API where one window/image/output graph is represented by an object, and then you have functions to
 2.  Does anyone have any use cases where plotting is performance critical, or
 should I just keep things simple/stupid in terms of the performance/simplicity
 tradeoff?

keep it simple I would just send dense arrays to it (which are close to the C api), and then have utility functions that convert ranges,... to dense arrays, but maybe I am biased because I have a good library to handle dense arrays. I would say that a reasonable goal is that the library could cope directly to plot of 1'000s of points at least for the simple 1D plot types. Fawzi
May 10 2009
parent reply dsimcha <dsimcha yahoo.com> writes:
== Quote from Fawzi Mohamed (fmohamed mac.com)'s article
 On 2009-05-10 21:23:48 +0200, dsimcha <dsimcha yahoo.com> said:
 It seems like there's substantial interest in this.  Please give me some use
 cases, i.e. what would you personally use this for, and what do you
 foresee others
 using it for, so I can start thinking about what the API should be.  I
 need a wide
 variety of use cases because, if I design the API based only on my personal use
 cases, it will end up being geared entirely toward histograms, scatter
 plots, and
 line graphs because that's what I use regularly.

live without.
 Besides use cases, here are some specific questions:
 1.  Is there any need for an OO-based API, or should I just use free functions?

represented by an object, and then you have functions to
 2.  Does anyone have any use cases where plotting is performance critical, or
 should I just keep things simple/stupid in terms of the performance/simplicity
 tradeoff?

the C api), and then have utility functions that convert ranges,... to dense arrays, but maybe I am biased because I have a good library to handle dense arrays. I would say that a reasonable goal is that the library could cope directly to plot of 1'000s of points at least for the simple 1D plot types.

Ok, this is way less than I had in mind. When I said high performance, I was thinking like, either plotting stuff under realtime constraints like if you're some Wall Street bigwig plotting zillions of charts to figure out what stocks to buy or, when doing summary stuff like histograms, handling billions of points read as a range from a file, i.e. more data than you have address space. I personally would not consider anything that couldn't gracefully handle at least a few million data points for histograms and a few 10s of thousands for scatter plots to be good enough.
May 10 2009
parent reply Robert Fraser <fraserofthenight gmail.com> writes:
dsimcha wrote:
 == Quote from Fawzi Mohamed (fmohamed mac.com)'s article
 On 2009-05-10 21:23:48 +0200, dsimcha <dsimcha yahoo.com> said:
 It seems like there's substantial interest in this.  Please give me some use
 cases, i.e. what would you personally use this for, and what do you
 foresee others
 using it for, so I can start thinking about what the API should be.  I
 need a wide
 variety of use cases because, if I design the API based only on my personal use
 cases, it will end up being geared entirely toward histograms, scatter
 plots, and
 line graphs because that's what I use regularly.

live without.
 Besides use cases, here are some specific questions:
 1.  Is there any need for an OO-based API, or should I just use free functions?

represented by an object, and then you have functions to
 2.  Does anyone have any use cases where plotting is performance critical, or
 should I just keep things simple/stupid in terms of the performance/simplicity
 tradeoff?

the C api), and then have utility functions that convert ranges,... to dense arrays, but maybe I am biased because I have a good library to handle dense arrays. I would say that a reasonable goal is that the library could cope directly to plot of 1'000s of points at least for the simple 1D plot types.

Ok, this is way less than I had in mind. When I said high performance, I was thinking like, either plotting stuff under realtime constraints like if you're some Wall Street bigwig plotting zillions of charts to figure out what stocks to buy or, when doing summary stuff like histograms, handling billions of points read as a range from a file, i.e. more data than you have address space. I personally would not consider anything that couldn't gracefully handle at least a few million data points for histograms and a few 10s of thousands for scatter plots to be good enough.

Having plots that update in realtime would be kind of awesome for monitoring, but the ones I was thinking of wouldn't be more than a few thousand data points in each sliding window, if that.
May 10 2009
parent Fawzi Mohamed <fmohamed mac.com> writes:
On 2009-05-11 02:05:51 +0200, Robert Fraser <fraserofthenight gmail.com> said:

 dsimcha wrote:
 == Quote from Fawzi Mohamed (fmohamed mac.com)'s article
 On 2009-05-10 21:23:48 +0200, dsimcha <dsimcha yahoo.com> said:
 It seems like there's substantial interest in this.  Please give me some use
 cases, i.e. what would you personally use this for, and what do you
 foresee others
 using it for, so I can start thinking about what the API should be.  I
 need a wide
 variety of use cases because, if I design the API based only on my personal use
 cases, it will end up being geared entirely toward histograms, scatter
 plots, and
 line graphs because that's what I use regularly.

live without.
 Besides use cases, here are some specific questions:
 1.  Is there any need for an OO-based API, or should I just use free functions?

represented by an object, and then you have functions to
 2.  Does anyone have any use cases where plotting is performance critical, or
 should I just keep things simple/stupid in terms of the performance/simplicity
 tradeoff?

the C api), and then have utility functions that convert ranges,... to dense arrays, but maybe I am biased because I have a good library to handle dense arrays. I would say that a reasonable goal is that the library could cope directly to plot of 1'000s of points at least for the simple 1D plot types.

Ok, this is way less than I had in mind. When I said high performance, I was thinking like, either plotting stuff under realtime constraints like if you're some Wall Street bigwig plotting zillions of charts to figure out what stocks to buy or, when doing summary stuff like histograms, handling billions of points read as a range from a file, i.e. more data than you have address space. I personally would not consider anything that couldn't gracefully handle at least a few million data points for histograms and a few 10s of thousands for scatter plots to be good enough.

Having plots that update in realtime would be kind of awesome for monitoring, but the ones I was thinking of wouldn't be more than a few thousand data points in each sliding window, if that.

my answer was along the keep it simple lines, you cannot really expect to represent more than some 1000s of points, if you have more you should do some transformation to represent them. Histogram for example reduce them, some cluster or spread analysis and represent fewer discrete objects,... All those things can be built later, the only thing needed is a basic lib that supports few 1000s of simple objects well, and less of complex objects. even realtime update an animations can be done if the library is fast for its basic use. Keep it simple, the fancy stuff can be built on the top of it later.
May 10 2009
prev sibling next sibling parent "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
dsimcha wrote:
 It seems like there's substantial interest in this.  Please give me some use
 cases, i.e. what would you personally use this for, and what do you foresee
others
 using it for, so I can start thinking about what the API should be.  I need a
wide
 variety of use cases because, if I design the API based only on my personal use
 cases, it will end up being geared entirely toward histograms, scatter plots,
and
 line graphs because that's what I use regularly.
 
 Besides use cases, here are some specific questions:
 1.  Is there any need for an OO-based API, or should I just use free functions?
 2.  Does anyone have any use cases where plotting is performance critical, or
 should I just keep things simple/stupid in terms of the performance/simplicity
 tradeoff?

I would be *very* interested in a plotting library for D. (Currently I output my data to text files, which I then process in Gnuplot.) You've already mentioned line graphs. The other thing I use most are 3D plots, i.e. z as a function of x and y -- preferably with color/gradient mapping. In such plots one should be able to specify the viewpoint from which the graph is seen. A special case should be the top-down view, which is essentially a 2d plot where the z axis value is represented solely by color/brightness. I think the functions should be able to work with both data sets and functions, i.e. both plot (real[] x, real[] y) and plot (real function(real) f, real xLeft, real xRight) should be available. Regarding the API, I say keep it as simple as possible -- at least to begin with. I would love it if plotting my results could be done almost as simply as writefln'ing them. :) -Lars
May 10 2009
prev sibling next sibling parent Jason House <jason.james.house gmail.com> writes:
dsimcha Wrote:

 It seems like there's substantial interest in this.  Please give me some use
 cases, i.e. what would you personally use this for, and what do you foresee
others
 using it for, so I can start thinking about what the API should be.  I need a
wide
 variety of use cases because, if I design the API based only on my personal use
 cases, it will end up being geared entirely toward histograms, scatter plots,
and
 line graphs because that's what I use regularly.
 
 Besides use cases, here are some specific questions:
 1.  Is there any need for an OO-based API, or should I just use free functions?
 2.  Does anyone have any use cases where plotting is performance critical, or
 should I just keep things simple/stupid in terms of the performance/simplicity
 tradeoff?

All my plotting would be real time monitoring of program operation. While I'm performance-sensitive, I would not expect a plotting library to be terribly efficient. A seamless way to allow plotting data over a socket would be awesome. My current uses would involve two kinds of plots: • line graphs that where I append data for the most recent timestamp. • bar graphs or maybe points with error bars (x axis is has labels, not numbers)
May 10 2009
prev sibling parent BLS <windevguy hotmail.de> writes:
dsimcha wrote:
 It seems like there's substantial interest in this.  Please give me some use
 cases, i.e. what would you personally use this for, and what do you foresee
others
 using it for, so I can start thinking about what the API should be.  I need a
wide
 variety of use cases because, if I design the API based only on my personal use
 cases, it will end up being geared entirely toward histograms, scatter plots,
and
 line graphs because that's what I use regularly.
 
 Besides use cases, here are some specific questions:
 1.  Is there any need for an OO-based API, or should I just use free functions?
 2.  Does anyone have any use cases where plotting is performance critical, or
 should I just keep things simple/stupid in terms of the performance/simplicity
 tradeoff?

I vote for simple free functions. 1) It will enable us to create a DLL. 2) An OOP framework can be added on top of free functions. (probably by using mixin templates) Björn
May 11 2009