www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Implementing Half Floats in D

reply Walter Bright <newshound2 digitalmars.com> writes:
http://www.drdobbs.com/cpp/implementing-half-floats-in-d/240146674

Anyone care to do the reddit honors?
Jan 28 2013
next sibling parent reply "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Monday, 28 January 2013 at 23:11:11 UTC, Walter Bright wrote:
 http://www.drdobbs.com/cpp/implementing-half-floats-in-d/240146674

 Anyone care to do the reddit honors?

[quote] and crushed back down to 16 bytes for storage. [/quote] Should be bits. Otherwise it looks really well done.
Jan 28 2013
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/28/2013 3:30 PM, Era Scarecrow wrote:
 On Monday, 28 January 2013 at 23:11:11 UTC, Walter Bright wrote:
 http://www.drdobbs.com/cpp/implementing-half-floats-in-d/240146674

 Anyone care to do the reddit honors?

[quote] and crushed back down to 16 bytes for storage. [/quote] Should be bits. Otherwise it looks really well done.

thank you. Sorry that didn't get caught in review!
Jan 28 2013
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 01/29/2013 11:40 AM, TommiT wrote:
 On Tuesday, 29 January 2013 at 08:48:24 UTC, Simen Kjaeraas wrote:
 Except that's not a literal. 1.3.hf would be a function call, while
 hf!1.3f
 is a template instantiation.

Can't help but point out, that some time ago, I made a suggestion to add a new function attribute to the language that would mean "evaluate any call to this function at compile time whenever it's possible". That would enable literals like: 1.3.hf and ComplexToConstruct(2.5, 7.5) Given: property HalfFloat hf(float v) aggressive_ctfe; and... struct ComplexToConstruct { this(float x, float y) aggressive_ctfe; }

I'd call it constfold.
Jan 29 2013
prev sibling next sibling parent reply Justin Whear <justin economicmodeling.com> writes:
On Mon, 28 Jan 2013 15:11:06 -0800, Walter Bright wrote:

 http://www.drdobbs.com/cpp/implementing-half-floats-in-d/240146674
 
 Anyone care to do the reddit honors?

Posted to /r/d_language, perhaps a x-post to /r/programming would be in order? http://www.reddit.com/r/d_language/comments/17gkrk/ implementing_half_floats_in_d_dr_dobbs/
Jan 28 2013
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/28/13 6:40 PM, Justin Whear wrote:
 On Mon, 28 Jan 2013 15:11:06 -0800, Walter Bright wrote:

 http://www.drdobbs.com/cpp/implementing-half-floats-in-d/240146674

 Anyone care to do the reddit honors?

Posted to /r/d_language, perhaps a x-post to /r/programming would be in order? http://www.reddit.com/r/d_language/comments/17gkrk/ implementing_half_floats_in_d_dr_dobbs/

Not sure what's going on, I tried to submit and got "already submitted" and then got redirected to http://www.reddit.com/r/programming/comments/16ycws/implementing_half_floats_in_d/ Hoever, I can't find the post in http://www.reddit.com/r/programming/new/ Andrei
Jan 28 2013
parent Walter Bright <newshound2 digitalmars.com> writes:
On 1/28/2013 3:50 PM, Andrei Alexandrescu wrote:
 Not sure what's going on, I tried to submit and got "already submitted" and
then
 got redirected to
 http://www.reddit.com/r/programming/comments/16ycws/implementing_half_floats_in_d/

8 days old, durn, it's sunsetted.
Jan 28 2013
prev sibling parent Justin Whear <justin economicmodeling.com> writes:
On Mon, 28 Jan 2013 18:50:22 -0500, Andrei Alexandrescu wrote:

 On 1/28/13 6:40 PM, Justin Whear wrote:
 On Mon, 28 Jan 2013 15:11:06 -0800, Walter Bright wrote:

 http://www.drdobbs.com/cpp/implementing-half-floats-in-d/240146674

 Anyone care to do the reddit honors?

Posted to /r/d_language, perhaps a x-post to /r/programming would be in order? http://www.reddit.com/r/d_language/comments/17gkrk/ implementing_half_floats_in_d_dr_dobbs/

Not sure what's going on, I tried to submit and got "already submitted" and then got redirected to http://www.reddit.com/r/programming/comments/16ycws/

 
 Hoever, I can't find the post in
 http://www.reddit.com/r/programming/new/
 
 
 Andrei

Possibly this: http://code.reddit.com/wiki/help/ faq#Whyisntmysubmissioncommentshowingup Walter is an admin; maybe he can check on it?
Jan 28 2013
prev sibling next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Walter Bright:

 http://www.drdobbs.com/cpp/implementing-half-floats-in-d/240146674

From the article:
Built-in types tend to run faster because the optimizer can take 
advantage of mathematical identities,<

Generally in D we like the compiler to verify user-applied annotations like pure and const. For the compiler of a normal language today it's probably too much difficult to prove the theorems coming from those identities on user defined types. But if the compiler assumes the programmer to be right on this (because such types are usually defined in well reviewed libraries), then I think it's not too much hard to invent a small set of annotations that attached to a user defined struct tell the compiler to allow some optimizations typical of integral numbers, floating point numbers, complex numbers, gaussian integers, quaternions, octonions and few others, according to them having associative, commutative, etc, properties, or not having them. Bye, bearophile
Jan 28 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Jan 29, 2013 at 01:44:20AM +0100, bearophile wrote:
 Walter Bright:
 
http://www.drdobbs.com/cpp/implementing-half-floats-in-d/240146674

From the article:
Built-in types tend to run faster because the optimizer can take
advantage of mathematical identities,<

Generally in D we like the compiler to verify user-applied annotations like pure and const. For the compiler of a normal language today it's probably too much difficult to prove the theorems coming from those identities on user defined types. But if the compiler assumes the programmer to be right on this (because such types are usually defined in well reviewed libraries), then I think it's not too much hard to invent a small set of annotations that attached to a user defined struct tell the compiler to allow some optimizations typical of integral numbers, floating point numbers, complex numbers, gaussian integers, quaternions, octonions and few others, according to them having associative, commutative, etc, properties, or not having them.

+1. I've always wanted a language capable of user-specified optimizations for custom types. This would be very useful on things like matrix types, which would otherwise require heavy trickery with expression templates in order to produce efficient code. T -- May you live all the days of your life. -- Jonathan Swift
Jan 28 2013
prev sibling next sibling parent "Tove" <tove fransson.se> writes:
On Monday, 28 January 2013 at 23:58:40 UTC, Walter Bright wrote:
 On 1/28/2013 3:30 PM, Era Scarecrow wrote:
 On Monday, 28 January 2013 at 23:11:11 UTC, Walter Bright 
 wrote:
 http://www.drdobbs.com/cpp/implementing-half-floats-in-d/240146674

 Anyone care to do the reddit honors?

[quote] and crushed back down to 16 bytes for storage. [/quote] Should be bits. Otherwise it looks really well done.

thank you. Sorry that didn't get caught in review!

"HalfFloat h = hf!1.3f;" Maybe you could also demonstrate that it's possible to implement another literal syntax? HalfFloat h = 1.3.hf; some people will prefer that for sure.
Jan 28 2013
prev sibling next sibling parent "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
On 2013-49-29 08:01, Tove <tove fransson.se> wrote:

 On Monday, 28 January 2013 at 23:58:40 UTC, Walter Bright wrote:
 On 1/28/2013 3:30 PM, Era Scarecrow wrote:
 On Monday, 28 January 2013 at 23:11:11 UTC, Walter Bright wrote:
 http://www.drdobbs.com/cpp/implementing-half-floats-in-d/240146674

 Anyone care to do the reddit honors?

[quote] and crushed back down to 16 bytes for storage. [/quote] Should be bits. Otherwise it looks really well done.

thank you. Sorry that didn't get caught in review!

"HalfFloat h = hf!1.3f;" Maybe you could also demonstrate that it's possible to implement another literal syntax? HalfFloat h = 1.3.hf; some people will prefer that for sure.

Except that's not a literal. 1.3.hf would be a function call, while hf!1.3f is a template instantiation. -- Simen
Jan 29 2013
prev sibling next sibling parent "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Tuesday, 29 January 2013 at 08:48:24 UTC, Simen Kjaeraas wrote:
 On 2013-49-29 08:01, Tove <tove fransson.se> wrote:
 Maybe you could also demonstrate that it's possible to 
 implement another literal syntax?
 HalfFloat h = 1.3.hf;

 some people will prefer that for sure.

Except that's not a literal. 1.3.hf would be a function call, while hf!1.3f is a template instantiation.

Unless it's an enum or something that requires it to know it during compile-time. enum h = 1.3.hf; //potentially correct But I doubt it would be that useful. More likely during optimization static values would be equal to it.
Jan 29 2013
prev sibling next sibling parent "TommiT" <tommitissari hotmail.com> writes:
On Tuesday, 29 January 2013 at 08:48:24 UTC, Simen Kjaeraas wrote:
 Except that's not a literal. 1.3.hf would be a function call, 
 while hf!1.3f
 is a template instantiation.

Can't help but point out, that some time ago, I made a suggestion to add a new function attribute to the language that would mean "evaluate any call to this function at compile time whenever it's possible". That would enable literals like: 1.3.hf and ComplexToConstruct(2.5, 7.5) Given: property HalfFloat hf(float v) aggressive_ctfe; and... struct ComplexToConstruct { this(float x, float y) aggressive_ctfe; }
Jan 29 2013
prev sibling next sibling parent reply "Stephan" <stephan_schiffels mac.com> writes:
On Monday, 28 January 2013 at 23:11:11 UTC, Walter Bright wrote:
 http://www.drdobbs.com/cpp/implementing-half-floats-in-d/240146674

 Anyone care to do the reddit honors?

Very nice. I learnt quite a bit there. btw: "In order to restrict it to only accepting floats, turn it into a template and check the type with static if:" - you actually use a static assert. But while we're at it: Why *don't* you use a signature constraint with if? I feel it would be more fitting to constrain the constructor usage. Stephan
Jan 29 2013
parent Walter Bright <newshound2 digitalmars.com> writes:
On 1/29/2013 2:43 AM, Stephan wrote:
 Why *don't* you use a signature constraint with if?

To get a custom error message.
Jan 29 2013
prev sibling next sibling parent "TommiT" <tommitissari hotmail.com> writes:
On Tuesday, 29 January 2013 at 11:26:00 UTC, Timon Gehr wrote:
 I'd call it  constfold.

That's a good name. Descriptive and short. I'm not going to try to push this suggestion any harder, since it got no support on the first time I tried it. But I'd like to mention just one thing that came to mind: The generic solution without any extra function attributes: template ct(alias expr) { enum ct = expr; } Literal of type TypeWithNonTrivialConstructor: ct!(TypeWithNonTrivialConstructor(2.5, 7.5)) The above solution throws the responsibility of the correct use of TypeWithNonTrivialConstructor on the end-user. Whereas a constfold attribute on the constructor of TypeWithNonTrivialConstructor automates the correct behaviour.
Jan 29 2013
prev sibling next sibling parent reply "Don" <don nospam.com> writes:
On Monday, 28 January 2013 at 23:11:11 UTC, Walter Bright wrote:
 http://www.drdobbs.com/cpp/implementing-half-floats-in-d/240146674

Since it got lost in the old thread on this topic, I'll repost my versions of floatToshort and shortToFloat, which are extremely fast (no unpredictable branches, no lookup tables) and respect the current rounding mode: ----------------- float shortToFloat(ushort s) { // note this is a signed shift, so sign bit gets smeared all the way into the int bit! uint u = ((cast(int)cast(short)s) << 13); if ( (s & EXPMASK) == 0 ) { // Subnormal or 0. // The simple conversion is wrong in two ways: // (1) it's added an implicit bit. This has value 0x1p-15. // (2) the mantissa bits got shifted along to make space for the hidden bit. // So we need to multiply the result by 2. // Note that 0x3880_0000 means 0x1p-14. uint v = (u & 0x0FFF_FFFF ) + 0x3880_0000; float f = *cast(float *)&v - 0x1p-14; u = (u & 0x8000_0000) | *cast(uint *)&f; return *cast(float *)&u; } u = (u & 0x8FFF_FFFF) + 0x3800_0000; if ( (s & EXPMASK) == EXPMASK ) { // infinity or NaN u |= FEXPMASK; } return *cast(float *)&u; } ----------------- NOTE: this only works on 64-bit runtime, for 32bit or CTFE with 80-bit intermediates, the constants need to be changed. Unfortunately I don't know of a nice way to detect the size of the intermediates. ----- ushort floatToShort(float f) { // Remember the sign uint x = *cast(uint *)&f; ushort sgn = (x >> 16) & 0x8000; // Need to treat NaN and Inf specially, otherwise the // mantissa shortening step would generate a new NaN. if ( (x & FEXPMASK) == FEXPMASK) return ( (x >> 13) & 0x7FFF) | sgn; // Shorten the mantissa, rounding it according to the current rounding mode f = (f * (1.0f + 0x1p-13f) -f) * 0x1p13f; // Force large numbers to overflow my moving near float.max f *= 0x1p112f; f *= 0x1p-112f; // Then undo it // Force small numbers to underflow, and shift into position f *= 0x1p-112f; uint u = *cast(uint *)&f; return ((u>>13) & 0x7FFF) | sgn; }
Jan 29 2013
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/29/2013 4:31 AM, Don wrote:
 Since it got lost in the old thread on this topic, I'll repost my versions of
 floatToshort and shortToFloat, which are extremely fast (no unpredictable
 branches, no lookup tables) and respect the current rounding mode:

I was waiting on incorporating your improvements until after std.halffloat was merged into Phobos. This is because the old ones were easier to understand, and I wanted them in the repository history.
Jan 29 2013
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-01-29 20:06, Walter Bright wrote:

 I was waiting on incorporating your improvements until after
 std.halffloat was merged into Phobos. This is because the old ones were
 easier to understand, and I wanted them in the repository history.

Oh, please no, not another module at the "std" level. -- /Jacob Carlborg
Jan 29 2013
parent reply Dejan Lekic <dejan.lekic gmail.com> writes:
Jacob Carlborg wrote:

 On 2013-01-29 20:06, Walter Bright wrote:
 
 I was waiting on incorporating your improvements until after
 std.halffloat was merged into Phobos. This is because the old ones were
 easier to understand, and I wanted them in the repository history.

Oh, please no, not another module at the "std" level.

+11111111 I can't agree more to this, Jacob... Seriously people... "std.halffloat" ??? -- Dejan Lekic dejan.lekic (a) gmail.com http://dejan.lekic.org
Jan 29 2013
parent 1100110 <0b1100110 gmail.com> writes:
On 01/30/2013 05:27 PM, Era Scarecrow wrote:
 On Tuesday, 29 January 2013 at 22:17:30 UTC, Dejan Lekic wrote:
 Jacob Carlborg wrote:
 Oh, please no, not another module at the "std" level.

+11111111 I can't agree more to this, Jacob... Seriously people... "std.halffloat" ???

I'll agree too. HalfFloats are more a special case with certain circles of programmers that need those. The majority of programmers (95%+) won't care about half floats, nor would it be needed; It would be akin to making std.bitop instad of core.bitop. You could HalfFloats to std.math; Right next to cos, sin and tan. std.mathspecial doesn't seem like it would be closely enough tied to make it an option. Alternatively make a std.customtypes for related misc types like this as they are added to phobos.

std.typecons? Seriously, std.type.cons, tuple, halffloat, bigint, etc...
Jan 30 2013
prev sibling next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 1/29/2013 2:27 PM, Iain Buclaw wrote:
 I'm sure this was mentioned before, but what does halffloat have over
 std.numeric.CustomFloat?

There was a long discussion about it, I don't remember if it was in the ng or on github.
Jan 29 2013
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/31/13 5:18 AM, Don wrote:
 std.numeric is not superficially flawed, it's fundamentally flawed. What
 is it for? What is its theme? The problem is, std.numeric is one of the
 few good names which are left as a possible package name, after C
 insulted the mathematical community by creating a module called 'math'.

Guilty as charged. I've put stuff in std.numeric as I was working on my thesis. I recall you added some stuff there too. As I'm sure you remember the state of D in 2007 was rather different than that of today. Overall no need to get agitated here, we're all on the same boat and aiming for the same shore. Let's see what we have there: entropy CustomFloat kullbackLeiblerDivergence Fft gapWeightedSimilarityIncremental gapWeightedSimilarity gapWeightedSimilarityNormalized FPTemporary findRoot euclideanDistance dotProduct cosineSimilarity gcd jensenShannonDivergence normalize secantMethod The general theme is obvious - numeric algorithms and data structures. Many are obvious and with obvious utility to one interested in numerics: entropy, various distance and similarity measures. I think you wrote findRoot. The gapWeightedSimilarity algorithms are string kernels. They are somewhat niche but quite powerful to anyone interested in string similarity (technically they are string edit distance on steroids). They might belong in std.string but I figured they have enough numeric algorithm flavor to put them in there. So let's itemize the grievances and see how we can sort this out. Thanks, Andrei
Jan 31 2013
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/31/13 10:38 AM, Don wrote:
 The basic problem is that there are hundreds of potential numeric
 algorithms and data structures of equal importance to these ones. In
 fact, the total number of mathematical algorithms is probably a
 substantial fraction of the total algorithms in computer science!

 Even a module which contained only FFT, could be quite large, once it
 included all the important related transforms.

A possible solution would be to make std.numeric a knot for several others, or convert it to a package.
 I'm not sure that we can solve this without addressing the high-level
 question: What is the scope of Phobos?

 How big will it eventually get? Twice its current size? Ten times? A
 hundred times?

I think the current trend is to go for large libraries.
 Both SmallPhobos and LargePhobos are reasonable, but we do have to pick
 one. Currently we have aspects of both approaches, but they aren't
 compatible.

LargePhobos. Agreed that we need to package-ize some of the current modules. Andrei
Jan 31 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-02-01 20:15, Paul D. Anderson wrote:

 Another consideration is the process for a package to become "standard".
 Current practice is to implement a package and go through a review in
 this forum. That's a good thing, but I'd like to see a more interactive
 approach, where the community was involved sooner:

 1. A package is proposed for future inclusion in the library. Presumably
 this is done by someone with an interest in the subject, ideally this is
 someone who wants to spearhead the effort and may already have a draft
 implementation.

 2. If there's enough interest to move forward, community members could
 bicker about all the bikeshed stuff: structures, methods,
 implementations, API, etc. until there's enough agreement about what the
 result should look like.

 3. Interested parties could collaborate on the implementation and release.

 4. There would have to be time limits. Many (most?) D projects start
 with a bang but then fizzle out before they're complete. There would
 need to be (monthly?) progress reviews. If the discussion or the
 implementation has dwindled, a reconsideration of the project could
 occur. If unsuccessful, the project would be moved off the active list.
 If there is renewed interest or if somebody else wants to take a crack
 at it the previous work can be a resource, but the project should
 probably go back to step 1.

 You all see the problem with this: the very short attention span of the
 D community. But we've got a good start between the DIP process and the
 review queue. The only thing really different would be the progress
 reviews, which would hopefully prevent projects falling of the end of
 the earth.

 This is just one possibility and there are probably better ways to do
 this. But I think some sort of maintenance of active projects is needed.

Isn't this what the whole community is about and what we're already doing. Just not very formal or official. -- /Jacob Carlborg
Feb 01 2013
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-02-02 14:42, Joseph Rushton Wakeling wrote:

 Speaking as a researcher, I've on occasion had cause to call on stuff
 from such "2nd party" libraries, and it's not necessarily a happy
 experience.

 A while ago I had cause to call on a package in CRAN, the contributed
 package archive for the R statistical programming language.  It wasn't
 one of the modules that's packaged for Debian et al., so I had to pull
 things in directly from R's own package management and build and install
 locally.

 In and of itself, R makes this very easy, but what I wasn't prepared for
 was that in installing the one package I was interested in, it would
 pull in and install dozens of other CRAN packages.  And then, when I
 looked inside the code, actually finding out what it was doing was
 nightmarish, because the module of interest was built on top of several
 other contributions by the same authors. Understanding the code involved
 a massive wild goose chase through all those other contributed modules
 to find what functions were being called and what they did.

 I don't know if this is typical of CRAN packages, because I tend to use
 R infrequently, but my strong impression was of code that had been
 built, thrown over the wall and then built on top of without any
 attention to design, integration or performance (when I re-implemented
 the algorithms using Octave/MATLAB they were much faster, and I doubt
 this was down to the superiority of the language or interpreter).

 And this code wasn't built by stupid people -- they were very good
 statisticians.  In their defence, I suspect the reason they built
 higgledy-piggledy as they did was because they knew their earlier
 modules _worked_ and reliability was the most important thing for them.

 That's the cost of 2nd-party libraries -- they are very hit-and-miss in
 terms of design, sustainability and hence, reliability.

 Now, that said, I think a "2nd-party" repository for D could be a great
 project, but what I _wouldn't_ like to see was that repository being
 considered an adequate replacement for carefully designed standard
 library functionality.  One of the things I love about D is precisely
 the breadth of Phobos' support, and it feels like the solution to
 Phobos' problems is a better design and review process rather than
 ringfencing a too-small set of core functionality.

 Of course, a 2nd-party repository could be part of the prototyping and
 experimentation behind new standard-library work, just as Boost is for C++.

I can tell you this about Ruby. Its package manager, RubyGems, is one its greatest assets. Just because a library has many dependencies doesn't mean it's badly designed. It could just mean that it's flexible and modular. When Ruby on Rails 3.0 (a web framework for Ruby) was released it had a lot more dependencies than Rails 2.0. One reason was it was more modular designed. They extracted package that could be used without the rest of Rails, like ActiveRecord (database access), ActiveSupport (a general utility package) and so on. Do you rather prefer how most C and C++ library works. They all implement everything from scratch. String classes, containers, algorithms and so on. Or they bundle a third party library within their own library forcing me to use it even though I already have it installed out of the box. -- /Jacob Carlborg
Feb 02 2013
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/2/13 11:50 AM, Jacob Carlborg wrote:
 I can tell you this about Ruby. Its package manager, RubyGems, is one
 its greatest assets.

 Just because a library has many dependencies doesn't mean it's badly
 designed. It could just mean that it's flexible and modular.

 When Ruby on Rails 3.0 (a web framework for Ruby) was released it had a
 lot more dependencies than Rails 2.0. One reason was it was more modular
 designed. They extracted package that could be used without the rest of
 Rails, like ActiveRecord (database access), ActiveSupport (a general
 utility package) and so on.

 Do you rather prefer how most C and C++ library works. They all
 implement everything from scratch. String classes, containers,
 algorithms and so on. Or they bundle a third party library within their
 own library forcing me to use it even though I already have it installed
 out of the box.

I agree with all of the above. While knowing very little about design of package managers, I really appreciate using such (either at Facebook or for other languages). It would be really meaningful to the future of D if a strongly motivated expert in package management would want to propose what it takes to get us where we should be in that regard. As a simple question - what would be an existing design that would be most appropriate for us to take inspiration from? Andrei
Feb 02 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Jan 31, 2013 at 07:53:58PM +0100, Zach the Mystic wrote:
 On Thursday, 31 January 2013 at 15:38:04 UTC, Don wrote:
I'm not sure that we can solve this without addressing the
high-level question: What is the scope of Phobos?

How big will it eventually get? Twice its current size? Ten times?
A hundred times?

Purely by chance I read an article on the blog of Jarrett Billingsley about this topic: http://www.jfbillingsley.com/blog/?p=206

Interesting. I think Phobos can be easily divided into two parts, with core stuff like std.algorithm, std.range, std.stdio, etc., in one, and less common but still widely applicable stuff like numeric algorithms in a "2nd party" library (or libraries). Even if we don't actually split Phobos up in that way, this may help us reorganize Phobos in a better way. The current flat std hierarchy is slowly approaching the point where it's starting to tear at the seams. We should consider starting a 3rd level of hierarchy sooner rather than later. T -- Help a man when he is in trouble and he will remember you when he is in trouble again.
Jan 31 2013
prev sibling next sibling parent "Zach the Mystic" <reachBUTMINUSTHISzach gOOGLYmail.com> writes:
On Thursday, 31 January 2013 at 19:04:05 UTC, H. S. Teoh wrote:
 On Thu, Jan 31, 2013 at 07:53:58PM +0100, Zach the Mystic wrote:
 On Thursday, 31 January 2013 at 15:38:04 UTC, Don wrote:
I'm not sure that we can solve this without addressing the
high-level question: What is the scope of Phobos?

How big will it eventually get? Twice its current size? Ten 
times?
A hundred times?

Purely by chance I read an article on the blog of Jarrett Billingsley about this topic: http://www.jfbillingsley.com/blog/?p=206

Interesting. I think Phobos can be easily divided into two parts, with core stuff like std.algorithm, std.range, std.stdio, etc., in one, and less common but still widely applicable stuff like numeric algorithms in a "2nd party" library (or libraries). Even if we don't actually split Phobos up in that way, this may help us reorganize Phobos in a better way. The current flat std hierarchy is slowly approaching the point where it's starting to tear at the seams. We should consider starting a 3rd level of hierarchy sooner rather than later. T

I don't know if this is relevant, but I find Jarret's writings to be quite brilliant. He designed a Lua-like language called miniD, based partly on D, but renamed Croc after he left the D community. I feel like I've learned a lot about programming just by reading the spec for his language.
Feb 01 2013
prev sibling next sibling parent "Zach the Mystic" <reachBUTMINUSTHISzach gOOGLYmail.com> writes:
On Thursday, 31 January 2013 at 19:04:05 UTC, H. S. Teoh wrote:
 Interesting. I think Phobos can be easily divided into two 
 parts, with
 core stuff like std.algorithm, std.range, std.stdio, etc., in 
 one, and
 less common but still widely applicable stuff like numeric 
 algorithms in
 a "2nd party" library (or libraries).

 Even if we don't actually split Phobos up in that way, this may 
 help us
 reorganize Phobos in a better way. The current flat std 
 hierarchy is
 slowly approaching the point where it's starting to tear at the 
 seams.
 We should consider starting a 3rd level of hierarchy sooner 
 rather than
 later.


 T

Whoops. Forgot to actually give the site: http://www.jfbillingsley.com/croc/
Feb 01 2013
prev sibling next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 01/31/2013 08:02 PM, H. S. Teoh wrote:
 Interesting. I think Phobos can be easily divided into two parts, with
 core stuff like std.algorithm, std.range, std.stdio, etc., in one, and
 less common but still widely applicable stuff like numeric algorithms in
 a "2nd party" library (or libraries).

Speaking as a researcher, I've on occasion had cause to call on stuff from such "2nd party" libraries, and it's not necessarily a happy experience. A while ago I had cause to call on a package in CRAN, the contributed package archive for the R statistical programming language. It wasn't one of the modules that's packaged for Debian et al., so I had to pull things in directly from R's own package management and build and install locally. In and of itself, R makes this very easy, but what I wasn't prepared for was that in installing the one package I was interested in, it would pull in and install dozens of other CRAN packages. And then, when I looked inside the code, actually finding out what it was doing was nightmarish, because the module of interest was built on top of several other contributions by the same authors. Understanding the code involved a massive wild goose chase through all those other contributed modules to find what functions were being called and what they did. I don't know if this is typical of CRAN packages, because I tend to use R infrequently, but my strong impression was of code that had been built, thrown over the wall and then built on top of without any attention to design, integration or performance (when I re-implemented the algorithms using Octave/MATLAB they were much faster, and I doubt this was down to the superiority of the language or interpreter). And this code wasn't built by stupid people -- they were very good statisticians. In their defence, I suspect the reason they built higgledy-piggledy as they did was because they knew their earlier modules _worked_ and reliability was the most important thing for them. That's the cost of 2nd-party libraries -- they are very hit-and-miss in terms of design, sustainability and hence, reliability. Now, that said, I think a "2nd-party" repository for D could be a great project, but what I _wouldn't_ like to see was that repository being considered an adequate replacement for carefully designed standard library functionality. One of the things I love about D is precisely the breadth of Phobos' support, and it feels like the solution to Phobos' problems is a better design and review process rather than ringfencing a too-small set of core functionality. Of course, a 2nd-party repository could be part of the prototyping and experimentation behind new standard-library work, just as Boost is for C++.
Feb 02 2013
prev sibling next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 02/02/2013 02:42 PM, Joseph Rushton Wakeling wrote:
 I've on occasion had cause to call on ....

 A while ago I had cause to call on ....

Oh bugger. Overly verbose _and_ repetitive. :-P
Feb 02 2013
prev sibling next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 02/02/2013 05:50 PM, Jacob Carlborg wrote:
 I can tell you this about Ruby. Its package manager, RubyGems, is one its
 greatest assets.

Don't get me wrong. I'm not speaking out against a package library per se, I think that would be an excellent idea. I'm just against the idea of narrowing Phobos' scope too strongly in favour of various things being provided by contributed libraries.
 Just because a library has many dependencies doesn't mean it's badly designed.
 It could just mean that it's flexible and modular.

I agree, but the example I was talking about really was just frustrating and opaque.
 Do you rather prefer how most C and C++ library works. They all implement
 everything from scratch. String classes, containers, algorithms and so on. Or
 they bundle a third party library within their own library forcing me to use it
 even though I already have it installed out of the box.

No, I don't like the typical C/C++ library situation -- that's why I'd like to see as much functionality as possible in a well-designed and supported standard library implementation.
Feb 02 2013
prev sibling parent "Peter Sommerfeld" <noreply rubrica.at> writes:
Joseph Rushton Wakeling <joseph.wakeling webdrake.net> wrote:

 No, I don't like the typical C/C++ library situation -- that's
 why I'd like to see as much functionality as possible in a well-designed  
 and supported standard library implementation.

As d-newcomer I want to support this! A large standard library is an attractive argument for the language because it makes sure that its features will be supported as the language evolves. Its organization is currently rather confusing and I suggest to seriously think about a deeper nested structure like it has been suggested for std.numeric: std.numeric.~, std.concurrent.~, std.io.~ etc. The pain of transformation is once, to keep it, it is recurring. Peter
Feb 02 2013
prev sibling next sibling parent "TommiT" <tommitissari hotmail.com> writes:
On Monday, 28 January 2013 at 23:11:11 UTC, Walter Bright wrote:
 http://www.drdobbs.com/cpp/implementing-half-floats-in-d/240146674

In the heat of coding, I think, the end-user is quite likely to make the honest mistake of writing: HalfFloat h = HalfFloat(1.3f); ...when he should be writing: HalfFloat h = hf!1.3f;
Jan 29 2013
prev sibling next sibling parent "David Nadlinger" <see klickverbot.at> writes:
On Tuesday, 29 January 2013 at 19:51:03 UTC, Jacob Carlborg wrote:
 On 2013-01-29 20:06, Walter Bright wrote:

 I was waiting on incorporating your improvements until after
 std.halffloat was merged into Phobos. This is because the old 
 ones were
 easier to understand, and I wanted them in the repository 
 history.

Oh, please no, not another module at the "std" level.

+1 David
Jan 29 2013
prev sibling next sibling parent Iain Buclaw <ibuclaw ubuntu.com> writes:
--e89a8f921a1ee9ed0204d474e4ef
Content-Type: text/plain; charset=ISO-8859-1

On 29 January 2013 19:06, Walter Bright <newshound2 digitalmars.com> wrote:

 On 1/29/2013 4:31 AM, Don wrote:

 Since it got lost in the old thread on this topic, I'll repost my
 versions of
 floatToshort and shortToFloat, which are extremely fast (no unpredictable
 branches, no lookup tables) and respect the current rounding mode:

I was waiting on incorporating your improvements until after std.halffloat was merged into Phobos. This is because the old ones were easier to understand, and I wanted them in the repository history.

I'm sure this was mentioned before, but what does halffloat have over std.numeric.CustomFloat? Thanks -- Iain Buclaw *(p < e ? p++ : p) = (c & 0x0f) + '0'; --e89a8f921a1ee9ed0204d474e4ef Content-Type: text/html; charset=ISO-8859-1 <div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On 29 January 2013 19:06, Walter Bright <span dir="ltr">&lt;<a href="mailto:newshound2 digitalmars.com" target="_blank">newshound2 digitalmars.com</a>&gt;</span> wrote:<br> <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">On 1/29/2013 4:31 AM, Don wrote:<br> <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> Since it got lost in the old thread on this topic, I&#39;ll repost my versions of<br> floatToshort and shortToFloat, which are extremely fast (no unpredictable<br> branches, no lookup tables) and respect the current rounding mode:<br> </blockquote> <br></div> I was waiting on incorporating your improvements until after std.halffloat was merged into Phobos. This is because the old ones were easier to understand, and I wanted them in the repository history.<br> <br> </blockquote></div><br><br></div><div class="gmail_extra">I&#39;m sure this was mentioned before, but what does halffloat have over std.numeric.CustomFloat?<br></div><div class="gmail_extra"><br><br clear="all"></div><div class="gmail_extra"> Thanks<br></div><div class="gmail_extra">-- <br>Iain Buclaw<br><br>*(p &lt; e ? p++ : p) = (c &amp; 0x0f) + &#39;0&#39;; </div></div> --e89a8f921a1ee9ed0204d474e4ef--
Jan 29 2013
prev sibling next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 01/29/2013 11:27 PM, Iain Buclaw wrote:
 I'm sure this was mentioned before, but what does halffloat have over
 std.numeric.CustomFloat?

"These formats are for storage only; all operations on them are performed by first implicitly extracting them to real first." ... whereas HalfFloat extracts to float, and so should be faster, which is probably one of the main goals here. But since std.numeric was mentioned, why not add HalfFloat there rather than its own separate module?
Jan 29 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Jan 29, 2013 at 04:36:48PM -0800, Walter Bright wrote:
 On 1/29/2013 2:27 PM, Iain Buclaw wrote:
I'm sure this was mentioned before, but what does halffloat have over
std.numeric.CustomFloat?

There was a long discussion about it, I don't remember if it was in the ng or on github.

It was on github. But IIRC no conclusion was ever reached. T -- Живёшь только однажды.
Jan 29 2013
prev sibling next sibling parent "Mehrdad" <wfunction hotmail.com> writes:
On Tuesday, 29 January 2013 at 22:49:02 UTC, Joseph Rushton 
Wakeling wrote:
 But since std.numeric was mentioned, why not add HalfFloat 
 there rather than its own separate module?

+1
Jan 29 2013
prev sibling next sibling parent "Don" <don nospam.com> writes:
On Tuesday, 29 January 2013 at 22:49:02 UTC, Joseph Rushton 
Wakeling wrote:
 On 01/29/2013 11:27 PM, Iain Buclaw wrote:
 I'm sure this was mentioned before, but what does halffloat 
 have over
 std.numeric.CustomFloat?

"These formats are for storage only; all operations on them are performed by first implicitly extracting them to real first." ... whereas HalfFloat extracts to float, and so should be faster, which is probably one of the main goals here. But since std.numeric was mentioned, why not add HalfFloat there rather than its own separate module?

The discussion we had on github agreed that std.halffloat isn't a good place. But OTOH std.numeric needs a complete overhaul, it's a mess. It would be a mistake to throw it in there.
Jan 30 2013
prev sibling next sibling parent "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
On 2013-01-30, 09:26, Don wrote:

 The discussion we had on github agreed that std.halffloat isn't a good  
 place.
 But OTOH std.numeric needs a complete overhaul, it's a mess.
 It would be a mistake to throw it in there.

So put it somewhere else for the moment, and move it to std.numeric after a while? Boy, does that ever sound stupid. I say stuff it in std.numeric, and announce properly that someone should probably have look at fixing std.numeric, and that HalfFloat in there may be the only thing salvageable (not that I know if it is, but that's how you make it sound). -- Simen
Jan 30 2013
prev sibling next sibling parent "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Tuesday, 29 January 2013 at 22:17:30 UTC, Dejan Lekic wrote:
 Jacob Carlborg wrote:
 Oh, please no, not another module at the "std" level.

+11111111 I can't agree more to this, Jacob... Seriously people... "std.halffloat" ???

I'll agree too. HalfFloats are more a special case with certain circles of programmers that need those. The majority of programmers (95%+) won't care about half floats, nor would it be needed; It would be akin to making std.bitop instad of core.bitop. You could HalfFloats to std.math; Right next to cos, sin and tan. std.mathspecial doesn't seem like it would be closely enough tied to make it an option. Alternatively make a std.customtypes for related misc types like this as they are added to phobos.
Jan 30 2013
prev sibling next sibling parent "Don" <don nospam.com> writes:
On Wednesday, 30 January 2013 at 12:51:18 UTC, Simen Kjaeraas 
wrote:
 On 2013-01-30, 09:26, Don wrote:

 The discussion we had on github agreed that std.halffloat 
 isn't a good place.
 But OTOH std.numeric needs a complete overhaul, it's a mess.
 It would be a mistake to throw it in there.

So put it somewhere else for the moment, and move it to std.numeric after a while? Boy, does that ever sound stupid.

Yes, of course that would be stupid. It will never be in std.numeric.
 I say stuff it in std.numeric, and announce properly that 
 someone should
 probably have look at fixing std.numeric, and that HalfFloat in 
 there may
 be the only thing salvageable (not that I know if it is, but 
 that's how
 you make it sound).

std.numeric is not superficially flawed, it's fundamentally flawed. What is it for? What is its theme? The problem is, std.numeric is one of the few good names which are left as a possible package name, after C insulted the mathematical community by creating a module called 'math'.
Jan 31 2013
prev sibling next sibling parent "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
On 2013-18-31 11:01, Don <don nospam.com> wrote:

 std.numeric is not superficially flawed, it's fundamentally flawed. What  
 is it for? What is its theme? The problem is, std.numeric is one of the  
 few good names which are left as a possible package name, after C  
 insulted the mathematical community by creating a module called 'math'.

Ah. Now that makes sense. Suppose we do scrap std.numeric as it is, and find new homes for the contents that can be salvaged (I sorta like 1100110's suggestion of std.type.halffloat, std.type.bigint, etc). What would you want in a refurbished std.numeric? -- Simen
Jan 31 2013
prev sibling next sibling parent Jens Mueller <jens.k.mueller gmx.de> writes:
Don wrote:
 On Wednesday, 30 January 2013 at 12:51:18 UTC, Simen Kjaeraas wrote:
On 2013-01-30, 09:26, Don wrote:

The discussion we had on github agreed that std.halffloat isn't
a good place.
But OTOH std.numeric needs a complete overhaul, it's a mess.
It would be a mistake to throw it in there.

So put it somewhere else for the moment, and move it to std.numeric after a while? Boy, does that ever sound stupid.

Yes, of course that would be stupid. It will never be in std.numeric.
I say stuff it in std.numeric, and announce properly that someone
should
probably have look at fixing std.numeric, and that HalfFloat in
there may
be the only thing salvageable (not that I know if it is, but
that's how
you make it sound).

std.numeric is not superficially flawed, it's fundamentally flawed. What is it for? What is its theme? The problem is, std.numeric is one of the few good names which are left as a possible package name, after C insulted the mathematical community by creating a module called 'math'.

Dimitry said this once. Why not start from scratch with package called std.numerics? // for numeric types std.numerics.halffloat std.numerics.complex std.numerics.quaternion ... // algorithms working with numeric types std.numerics.algorithms // even std.math could make sense or something similar. Wouldn't this work? Jens
Jan 31 2013
prev sibling next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 01/31/2013 11:18 AM, Don wrote:
 std.numeric is not superficially flawed, it's fundamentally flawed. What is it
 for? What is its theme? The problem is, std.numeric is one of the few good
names
 which are left as a possible package name

std.numerical ... ?
Jan 31 2013
prev sibling next sibling parent "Don" <don nospam.com> writes:
On Thursday, 31 January 2013 at 13:41:13 UTC, Andrei Alexandrescu 
wrote:
 On 1/31/13 5:18 AM, Don wrote:
 std.numeric is not superficially flawed, it's fundamentally 
 flawed. What
 is it for? What is its theme? The problem is, std.numeric is 
 one of the
 few good names which are left as a possible package name, 
 after C
 insulted the mathematical community by creating a module 
 called 'math'.

Guilty as charged. I've put stuff in std.numeric as I was working on my thesis. I recall you added some stuff there too. As I'm sure you remember the state of D in 2007 was rather different than that of today. Overall no need to get agitated here, we're all on the same boat and aiming for the same shore.

Sorry if that came across as agitated, it wasn't intended to be. As you noted, I have code in there as well. It's just one of those old modules that needs to be cleaned up, though it reveals a deeper issue - see below.
 Let's see what we have there:

 entropy
 CustomFloat
 kullbackLeiblerDivergence
 Fft
 gapWeightedSimilarityIncremental
 gapWeightedSimilarity
 gapWeightedSimilarityNormalized
 FPTemporary
 findRoot
 euclideanDistance
 dotProduct
 cosineSimilarity
 gcd
 jensenShannonDivergence
 normalize
 secantMethod

 The general theme is obvious - numeric algorithms and data 
 structures. Many are obvious and with obvious utility to one 
 interested in numerics: entropy, various distance and 
 similarity measures. I think you wrote findRoot.

Yes. The basic problem is that there are hundreds of potential numeric algorithms and data structures of equal importance to these ones. In fact, the total number of mathematical algorithms is probably a substantial fraction of the total algorithms in computer science! Even a module which contained only FFT, could be quite large, once it included all the important related transforms.
 The gapWeightedSimilarity algorithms are string kernels. They 
 are somewhat niche but quite powerful to anyone interested in 
 string similarity (technically they are string edit distance on 
 steroids). They might belong in std.string but I figured they 
 have enough numeric algorithm flavor to put them in there.

 So let's itemize the grievances and see how we can sort this 
 out.

I'm not sure that we can solve this without addressing the high-level question: What is the scope of Phobos? How big will it eventually get? Twice its current size? Ten times? A hundred times? Both SmallPhobos and LargePhobos are reasonable, but we do have to pick one. Currently we have aspects of both approaches, but they aren't compatible. The current approach of putting everything directly into a single level in std doesn't scale very far -- it will look very clumsy once it gets more than (say) three times larger. This argues for SmallPhobos. But if it doesn't get to be at least ten times larger, some of this niche stuff shouldn't be in there, they are functions from LargePhobos. If we go with SmallPhobos then we need to move the niche stuff somewhere else.
Jan 31 2013
prev sibling next sibling parent Sean Kelly <sean invisibleduck.org> writes:
On Jan 31, 2013, at 7:38 AM, "Don" <don nospam.com> wrote:

 But if it doesn't get to be at least ten times larger, some of this niche s=

th SmallPhobos then we need to move the niche stuff somewhere else. If D had a package distribution system, the solution would be obvious. I'd a= rgue for SmallPhobos plus that. Everything in Phobos proper has to be mainta= ined as a part of Phobos, after all.=20=
Jan 31 2013
prev sibling next sibling parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Thursday, 31 January 2013 at 15:38:04 UTC, Don wrote:
 On Thursday, 31 January 2013 at 13:41:13 UTC, Andrei 
 Alexandrescu wrote:
 On 1/31/13 5:18 AM, Don wrote:
 std.numeric is not superficially flawed, it's fundamentally 
 flawed. What
 is it for? What is its theme? The problem is, std.numeric is 
 one of the
 few good names which are left as a possible package name, 
 after C
 insulted the mathematical community by creating a module 
 called 'math'.

Guilty as charged. I've put stuff in std.numeric as I was working on my thesis. I recall you added some stuff there too. As I'm sure you remember the state of D in 2007 was rather different than that of today. Overall no need to get agitated here, we're all on the same boat and aiming for the same shore.

Sorry if that came across as agitated, it wasn't intended to be. As you noted, I have code in there as well. It's just one of those old modules that needs to be cleaned up, though it reveals a deeper issue - see below.
 Let's see what we have there:

 entropy
 CustomFloat
 kullbackLeiblerDivergence
 Fft
 gapWeightedSimilarityIncremental
 gapWeightedSimilarity
 gapWeightedSimilarityNormalized
 FPTemporary
 findRoot
 euclideanDistance
 dotProduct
 cosineSimilarity
 gcd
 jensenShannonDivergence
 normalize
 secantMethod

 The general theme is obvious - numeric algorithms and data 
 structures. Many are obvious and with obvious utility to one 
 interested in numerics: entropy, various distance and 
 similarity measures. I think you wrote findRoot.

Yes. The basic problem is that there are hundreds of potential numeric algorithms and data structures of equal importance to these ones. In fact, the total number of mathematical algorithms is probably a substantial fraction of the total algorithms in computer science! Even a module which contained only FFT, could be quite large, once it included all the important related transforms.
 The gapWeightedSimilarity algorithms are string kernels. They 
 are somewhat niche but quite powerful to anyone interested in 
 string similarity (technically they are string edit distance 
 on steroids). They might belong in std.string but I figured 
 they have enough numeric algorithm flavor to put them in there.

 So let's itemize the grievances and see how we can sort this 
 out.

I'm not sure that we can solve this without addressing the high-level question: What is the scope of Phobos? How big will it eventually get? Twice its current size? Ten times? A hundred times? Both SmallPhobos and LargePhobos are reasonable, but we do have to pick one. Currently we have aspects of both approaches, but they aren't compatible. The current approach of putting everything directly into a single level in std doesn't scale very far -- it will look very clumsy once it gets more than (say) three times larger. This argues for SmallPhobos. But if it doesn't get to be at least ten times larger, some of this niche stuff shouldn't be in there, they are functions from LargePhobos. If we go with SmallPhobos then we need to move the niche stuff somewhere else.

I think having a large standard library inspires confidence in developers. Rightly or wrongly, code in a standard library has an appearance of permanence, as opposed to being someone's personal project that may or may not disappear/cease to be maintained tomorrow.
Jan 31 2013
prev sibling next sibling parent "Zach the Mystic" <reachBUTMINUSTHISzach gOOGLYmail.com> writes:
On Thursday, 31 January 2013 at 15:38:04 UTC, Don wrote:
 I'm not sure that we can solve this without addressing the 
 high-level question: What is the scope of Phobos?

 How big will it eventually get? Twice its current size? Ten 
 times? A hundred times?

Purely by chance I read an article on the blog of Jarrett Billingsley about this topic: http://www.jfbillingsley.com/blog/?p=206
Jan 31 2013
prev sibling next sibling parent "Paul D. Anderson" <paul.d.removethis.anderson comcast.andthis.net> writes:
I agree that we need a distinct package for numerics, both 
numbers and algorithms.

I proposed a birds-of-a-feather discussion on this topic for 
DConf2013, but haven't heard back yet. If not officially, we can 
still discuss this there, perhaps?

Paul D. Anderson
Feb 01 2013
prev sibling next sibling parent "Paul D. Anderson" <paul.d.removethis.anderson comcast.andthis.net> writes:
On Thursday, 31 January 2013 at 17:05:50 UTC, Andrei Alexandrescu 
wrote:
 On 1/31/13 10:38 AM, Don wrote:
 The basic problem is that there are hundreds of potential 
 numeric
 algorithms and data structures of equal importance to these 
 ones. In
 fact, the total number of mathematical algorithms is probably a
 substantial fraction of the total algorithms in computer 
 science!

 Even a module which contained only FFT, could be quite large, 
 once it
 included all the important related transforms.

A possible solution would be to make std.numeric a knot for several others, or convert it to a package.
 I'm not sure that we can solve this without addressing the 
 high-level
 question: What is the scope of Phobos?

 How big will it eventually get? Twice its current size? Ten 
 times? A
 hundred times?

I think the current trend is to go for large libraries.
 Both SmallPhobos and LargePhobos are reasonable, but we do 
 have to pick
 one. Currently we have aspects of both approaches, but they 
 aren't
 compatible.

LargePhobos. Agreed that we need to package-ize some of the current modules. Andrei

Another consideration is the process for a package to become "standard". Current practice is to implement a package and go through a review in this forum. That's a good thing, but I'd like to see a more interactive approach, where the community was involved sooner: 1. A package is proposed for future inclusion in the library. Presumably this is done by someone with an interest in the subject, ideally this is someone who wants to spearhead the effort and may already have a draft implementation. 2. If there's enough interest to move forward, community members could bicker about all the bikeshed stuff: structures, methods, implementations, API, etc. until there's enough agreement about what the result should look like. 3. Interested parties could collaborate on the implementation and release. 4. There would have to be time limits. Many (most?) D projects start with a bang but then fizzle out before they're complete. There would need to be (monthly?) progress reviews. If the discussion or the implementation has dwindled, a reconsideration of the project could occur. If unsuccessful, the project would be moved off the active list. If there is renewed interest or if somebody else wants to take a crack at it the previous work can be a resource, but the project should probably go back to step 1. You all see the problem with this: the very short attention span of the D community. But we've got a good start between the DIP process and the review queue. The only thing really different would be the progress reviews, which would hopefully prevent projects falling of the end of the earth. This is just one possibility and there are probably better ways to do this. But I think some sort of maintenance of active projects is needed. Paul
Feb 01 2013
prev sibling next sibling parent "Paul D. Anderson" <paul.d.removethis.anderson comcast.andthis.net> writes:
On Friday, 1 February 2013 at 20:04:47 UTC, Jacob Carlborg wrote:
 Isn't this what the whole community is about and what we're 
 already doing. Just not very formal or official.

You're right, but what I was looking for was something that actually was more formal. I think the only thing that's missing is someone to keep track of and check up on development efforts. So I'll revise my proposal: I'll volunteer to try to gather information on the projects that are in-work and see what I can do to find their status and gauge their progress. If doing this turns out to be too difficult to manage or of little value I'll announce that and shut up. But if it works, I thik it will add value to our efforts. Paul
Feb 01 2013
prev sibling next sibling parent "SomeDude" <lovelydear mailmetrash.com> writes:
On Thursday, 31 January 2013 at 15:38:04 UTC, Don wrote:
 The current approach of putting everything directly into a 
 single level in std doesn't scale very far -- it will look very 
 clumsy once it gets more than (say) three times larger. This 
 argues for SmallPhobos.

 But if it doesn't get to be at least ten times larger, some of 
 this niche stuff shouldn't be in there, they are functions from 
 LargePhobos. If we go with SmallPhobos then we need to move the 
 niche stuff somewhere else.

I am all for a "small" Phobos. And specialized libraries in Deimos, which should get the same level of attention as Phobos.
Feb 02 2013
prev sibling parent "David Nadlinger" <see klickverbot.at> writes:
On Sunday, 3 February 2013 at 07:43:57 UTC, SomeDude wrote:
 I am all for a "small" Phobos. And specialized libraries in 
 Deimos, which should get the same level of attention as Phobos.

Huh? Deimos is specifically for C headers translated to D. David
Feb 03 2013