www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - SciD: the humble beginning

reply "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
I've just created a new dsource project and uploaded my collection of 
numerical routines. You can find it here:

   http://www.dsource.org/projects/scid

Hopefully others will find it useful as well. Most of the code is 
written with a "get it to work - NOW" philoshophy, and as such there is 
a lot of potential for improvement and polish, not least when it comes 
to performance. I am aware of this, so there is really no need to 
benchmark it against established scientific libraries quite yet. ;) More 
routines will be added as I write them (which is usually when I need 
them for work), and the existing ones will be polished whenever I have 
time to do so.

Huge thanks to Bill Baxter for letting me use his BLAS and LAPACK bindings!

Requirements:
   DMD 2.037
   BLAS and LAPACK libraries

-Lars
Dec 13 2009
next sibling parent reply dsimcha <dsimcha yahoo.com> writes:
== Quote from Lars T. Kyllingstad (public kyllingen.NOSPAMnet)'s article
 I've just created a new dsource project and uploaded my collection of
 numerical routines. You can find it here:
    http://www.dsource.org/projects/scid
 Hopefully others will find it useful as well. Most of the code is
 written with a "get it to work - NOW" philoshophy, and as such there is
 a lot of potential for improvement and polish, not least when it comes
 to performance. I am aware of this, so there is really no need to
 benchmark it against established scientific libraries quite yet. ;) More
 routines will be added as I write them (which is usually when I need
 them for work), and the existing ones will be polished whenever I have
 time to do so.
 Huge thanks to Bill Baxter for letting me use his BLAS and LAPACK bindings!
 Requirements:
    DMD 2.037
    BLAS and LAPACK libraries
 -Lars

This looks absolutely terrific given that it's a work in progress and all. I agree that we shouldn't be too obsessed with performance yet because: 1. **Make it work**, make it right, make it fast. We can always optimize it later provided that there aren't any ridiculously bad design decisions. 2. At least some of the time people (myself included) would rather a nice API and decent performance than an ugly API (read: a straight Fortran or C API) and lightening quick performance. We've got the beginnings of a good matrix/numerics lib in your work. I believe that my dstats lib would provide just about all the statistical functionality a good scientific lib would need, and I'm getting close to declaring it beta-quality. I'd say based on a quick look at what scipy has that we also need the following for a credible full-fledged scientific lib: Machine learning, i.e. classification and clustering. (I'd probably be qualified to write that and was halfway thinking of breaking ground on it over Christmas break. However, I'm not committing to this yet, so if someone else already has a work in progress, let me know. Also, there's a Java machine learning library called Weka that I've used before. It's very complete but the API is painful and it's GPL, so IDK if it'd be worth it to port to D.) Optimization. (A few people have tried but IDK if they've actually gotten far off the ground with it.) Basic image I/O and processing. Plotting. I've considered doing this a few times, but I've decided it needs to be put off until D2 is stable and the GUI toolkits for it are reasonably stable. One layer of instability (D2 itself) is workable, but two layers (D2 and the GUI libs) is not.
Dec 13 2009
next sibling parent "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
dsimcha wrote:
 == Quote from Lars T. Kyllingstad (public kyllingen.NOSPAMnet)'s article
 I've just created a new dsource project and uploaded my collection of
 numerical routines. You can find it here:
    http://www.dsource.org/projects/scid
 Hopefully others will find it useful as well. Most of the code is
 written with a "get it to work - NOW" philoshophy, and as such there is
 a lot of potential for improvement and polish, not least when it comes
 to performance. I am aware of this, so there is really no need to
 benchmark it against established scientific libraries quite yet. ;) More
 routines will be added as I write them (which is usually when I need
 them for work), and the existing ones will be polished whenever I have
 time to do so.
 Huge thanks to Bill Baxter for letting me use his BLAS and LAPACK bindings!
 Requirements:
    DMD 2.037
    BLAS and LAPACK libraries
 -Lars

This looks absolutely terrific given that it's a work in progress and all. I agree that we shouldn't be too obsessed with performance yet because: 1. **Make it work**, make it right, make it fast. We can always optimize it later provided that there aren't any ridiculously bad design decisions.

A lot of the code is ported from 20-30 year old (i.e. extremely well tested) FORTRAN code, so any performance gains would be a bonus provided by the D language itself, and most likely not in the algorithms. Also, if you spot any ridiculously bad design decisions, let me know. ;)
 2.  At least some of the time people (myself included) would rather a nice API
and
 decent performance than an ugly API (read:  a straight Fortran or C API) and
 lightening quick performance.

Yeah, that's what I think too. And, the way it is now, people who for some reason want to use the old-style api can just access the scid.ports functions directly.
 We've got the beginnings of a good matrix/numerics lib in your work.  I believe
 that my dstats lib would provide just about all the statistical functionality a
 good scientific lib would need, and I'm getting close to declaring it
 beta-quality.  I'd say based on a quick look at what scipy has that we also
need
 the following for a credible full-fledged scientific lib:
 
 Machine learning, i.e. classification and clustering.  (I'd probably be
qualified
 to write that and was halfway thinking of breaking ground on it over Christmas
 break.  However, I'm not committing to this yet, so if someone else already
has a
 work in progress, let me know.  Also, there's a Java machine learning library
 called Weka that I've used before.  It's very complete but the API is painful
and
 it's GPL, so IDK if it'd be worth it to port to D.)
 
 Optimization.  (A few people have tried but IDK if they've actually gotten far
off
 the ground with it.)
 
 Basic image I/O and processing.
 
 Plotting.  I've considered doing this a few times, but I've decided it needs
to be
 put off until D2 is stable and the GUI toolkits for it are reasonably stable. 
One
 layer of instability (D2 itself) is workable, but two layers (D2 and the GUI
libs)
 is not.

I agree with all of this. Here's a piece of advice to anyone who wants to try their hand at these and other things: There are a LOT of established and *extremely* well-tested numerical libraries lying around on the internet, and many of them are in the public domain. Instead of spending a lot of time re-inventing the wheel (I've tried it), port some old code and try to improve on it when you get it working. Here are a few starting points: http://www.netlib.org/ http://gams.nist.gov/ And I forgot to say in my original announcement: I welcome contributors with open arms. I have no intention of making this a one-man show. If you want to contribute or participate in some way, let me know. -Lars
Dec 13 2009
prev sibling parent "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
dsimcha wrote:
 We've got the beginnings of a good matrix/numerics lib in your work.  I believe
 that my dstats lib would provide just about all the statistical functionality a
 good scientific lib would need, and I'm getting close to declaring it
 beta-quality.  I'd say based on a quick look at what scipy has that we also
need
 the following for a credible full-fledged scientific lib:

dstats has actually been kind of an inspiration for me. I've never used it, because I don't do any statistics, but I've looked through the API docs and the examples, and I really like the interface.
 Machine learning, i.e. classification and clustering.  (I'd probably be
qualified
 to write that and was halfway thinking of breaking ground on it over Christmas
 break.  However, I'm not committing to this yet, so if someone else already
has a
 work in progress, let me know.  Also, there's a Java machine learning library
 called Weka that I've used before.  It's very complete but the API is painful
and
 it's GPL, so IDK if it'd be worth it to port to D.)
 
 Optimization.  (A few people have tried but IDK if they've actually gotten far
off
 the ground with it.)
 
 Basic image I/O and processing.
 
 Plotting.  I've considered doing this a few times, but I've decided it needs
to be
 put off until D2 is stable and the GUI toolkits for it are reasonably stable. 
One
 layer of instability (D2 itself) is workable, but two layers (D2 and the GUI
libs)
 is not.

A few things that can be added to the list: - special functions - Fourier transforms - Monte Carlo methods - series - least-squares fitting - differential and integral equations There is a good chance that I will need the latter for work just after the holidays, in which case I'll write it myself. I'll add the above lists to the project home page. -Lars
Dec 14 2009
prev sibling parent reply Georg Wrede <georg nospam.org> writes:
Lars T. Kyllingstad wrote:
 Most of the code is 
 written with a "get it to work - NOW" philoshophy, and as such there is 
 a lot of potential for improvement and polish, not least when it comes 
 to performance. I am aware of this, so there is really no need to 
 benchmark it against established scientific libraries quite yet.

I wish more of software were written with this attitude. Too many a promising project is abandoned because it is not "proper", "comprehensive", "neat", "polished", "cool", "impressive", "performant", "unique", "impressive", "trail-blazing", ... And the only one who loses is US. WE lose. This is tantamount to quenching creativity, the very property that differentiates us from the aboriginals. But observe: not because they'd be inferior, but simply because such a big part of their existence is consumed by the very acts of finding food and shelter, that we take for granted, while we have the time to indulge in quests of remote rewards. The difference between "the Nike philosophy" (just do it), and doing it "the day AFTER tomorrow, after proper design and deliberation", is not the same as between "Bill Gates' get it out now, who cares how crappy", and "wait 'till it's perfect".
Dec 21 2009
parent torhu <no spam.invalid> writes:
On 22.12.2009 1:44, Georg Wrede wrote:
 Lars T. Kyllingstad wrote:
  Most of the code is
  written with a "get it to work - NOW" philoshophy, and as such there is
  a lot of potential for improvement and polish, not least when it comes
  to performance. I am aware of this, so there is really no need to
  benchmark it against established scientific libraries quite yet.

I wish more of software were written with this attitude. Too many a promising project is abandoned because it is not "proper", "comprehensive", "neat", "polished", "cool", "impressive", "performant", "unique", "impressive", "trail-blazing", ... And the only one who loses is US. WE lose. This is tantamount to quenching creativity, the very property that differentiates us from the aboriginals. But observe: not because they'd be inferior, but simply because such a big part of their existence is consumed by the very acts of finding food and shelter, that we take for granted, while we have the time to indulge in quests of remote rewards. The difference between "the Nike philosophy" (just do it), and doing it "the day AFTER tomorrow, after proper design and deliberation", is not the same as between "Bill Gates' get it out now, who cares how crappy", and "wait 'till it's perfect".

LOL WUT Sorry, someone had to say it. :)
Dec 21 2009