www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Experimental Phobos modules?

reply =?ISO-8859-1?Q?Alex_R=F8nne_Petersen?= <alex lycus.org> writes:
Hi,

I think we need a semi-formal process for experimental Phobos modules. 
That is, modules that are slated for the Phobos review process and/or 
eventual inclusion in Phobos but needs actual field testing to work out 
a proper interface or implementation.

(I'm assuming most people in the community think an experimental package 
namespace in Phobos is a good idea.)

Some people see the "etc" package as being this but I think it's too 
poorly named. Something like "experimental" (or just "exp") would be 
more sensible IMO.

Does anyone have any thoughts on how we might do this? Any bikeshedding? 
I'm not really sure what the best way to go about this is, so really, 
any input is very welcome.

-- 
Alex Rønne Petersen
alex lycus.org
http://lycus.org
Dec 05 2012
next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday, December 05, 2012 09:27:39 Alex R=C3=B8nne Petersen wrote=
:
 Hi,
=20
 I think we need a semi-formal process for experimental Phobos modules=

 That is, modules that are slated for the Phobos review process and/or=

 eventual inclusion in Phobos but needs actual field testing to work o=

 a proper interface or implementation.
=20
 (I'm assuming most people in the community think an experimental pack=

 namespace in Phobos is a good idea.)
=20
 Some people see the "etc" package as being this but I think it's too
 poorly named. Something like "experimental" (or just "exp") would be
 more sensible IMO.
=20
 Does anyone have any thoughts on how we might do this? Any bikesheddi=

 I'm not really sure what the best way to go about this is, so really,=

 any input is very welcome.

Given the discussions of stability of late, I _have_ wondered if we wou= ld be=20 better off if instead of directly merging new modules into Phobos, we p= ut them=20 in a package (e.g. experimental) which was known to be temporary so tha= t=20 people could mess around with it before its API was more or less set in= stone.=20 Only after some time in there would they actually move into Phobos prop= er. The=20 review process helps a great deal, but the modules don't always get muc= h use=20 before they're actually merged in, and actual use can definitely reveal= flaws=20 that even a solid review won't. Whether we want some kind of experimental set of modules in Phobos whic= h=20 _could_ be reviewed, I don't know. I'm inclined to think that that's a = bad=20 idea. However, it might be a good idea if we had a separate project (ak= in to=20 Boost perhaps) with a lower standard for inclusion and minimal long ter= m=20 stability so that more modules could be tested and tried out before goi= ng=20 through the actual review process. It would also potentially make it ea= sier=20 for programmers to take other people's code and make it Phobos ready wh= en the=20 original programmers are willing to write it in the first place but not= got the=20 extra mile to make it Phobos ready, as they could put into that incubat= or=20 project to be used, abused, and improved by others. Regardless, I think that we need to look at finding ways to make sure t= hat=20 major inclusions to Phobos get more actual field testing before being p= ut into=20 Phobos in their final form. The review process is great, but I don't th= ink that=20 it's always enough to make sure that a module's API is really going to = work=20 long term. And if we want long term stability, we need to make sure tha= t new=20 modules are really ready to have their APIs essentially frozen when the= y make=20 it into Phobos proper (allowing further additions but avoiding breaking= =20 compatibility with any changes that are made). - Jonathan M Davis
Dec 05 2012
prev sibling next sibling parent "Pragma Tix" <bizprac orange.fr> writes:
On Wednesday, 5 December 2012 at 08:27:39 UTC, Alex Rønne 
Petersen wrote:
 Hi,

 I think we need a semi-formal process for experimental Phobos 
 modules. That is, modules that are slated for the Phobos review 
 process and/or eventual inclusion in Phobos but needs actual 
 field testing to work out a proper interface or implementation.

 (I'm assuming most people in the community think an 
 experimental package namespace in Phobos is a good idea.)

 Some people see the "etc" package as being this but I think 
 it's too poorly named. Something like "experimental" (or just 
 "exp") would be more sensible IMO.

 Does anyone have any thoughts on how we might do this? Any 
 bikeshedding? I'm not really sure what the best way to go about 
 this is, so really, any input is very welcome.

Years ago I've suggested to introduce an incubator project.
Dec 05 2012
prev sibling next sibling parent reply "Pragma Tix" <bizprac orange.fr> writes:
On Wednesday, 5 December 2012 at 08:27:39 UTC, Alex Rønne 
Petersen wrote:
 Hi,

 I think we need a semi-formal process for experimental Phobos 
 modules. That is, modules that are slated for the Phobos review 
 process and/or eventual inclusion in Phobos but needs actual 
 field testing to work out a proper interface or implementation.

 (I'm assuming most people in the community think an 
 experimental package namespace in Phobos is a good idea.)

 Some people see the "etc" package as being this but I think 
 it's too poorly named. Something like "experimental" (or just 
 "exp") would be more sensible IMO.

 Does anyone have any thoughts on how we might do this? Any 
 bikeshedding? I'm not really sure what the best way to go about 
 this is, so really, any input is very welcome.

Years ago I have suggested to establish an incubator project.
Dec 05 2012
parent reply =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:
On 05-12-2012 17:10, Jonathan M Davis wrote:
 On Wednesday, December 05, 2012 15:10:59 Pragma Tix wrote:
 Years ago I have suggested to establish an incubator project.

It's been suggested a few times, but no one ever does it. As with a lot of things around here, unless someone steps up and champions it, it isn't going to happen. - Jonathan M Davis

In this case, though, there isn't really much to *do*. People just need to send pull requests when they have a module they feel is ready for field testing. -- Alex Rønne Petersen alex lycus.org http://lycus.org
Dec 05 2012
parent reply =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:
On 05-12-2012 22:05, Jonathan M Davis wrote:
 On Wednesday, December 05, 2012 21:32:37 Alex Rønne Petersen wrote:
 On 05-12-2012 17:10, Jonathan M Davis wrote:
 On Wednesday, December 05, 2012 15:10:59 Pragma Tix wrote:
 Years ago I have suggested to establish an incubator project.

It's been suggested a few times, but no one ever does it. As with a lot of things around here, unless someone steps up and champions it, it isn't going to happen. - Jonathan M Davis

In this case, though, there isn't really much to *do*. People just need to send pull requests when they have a module they feel is ready for field testing.

Yes. But I think that that's completely inappropriate for putting into Phobos. We would actually need a separate project for that. And someone would need to set up and manage that project. Given that they wouldn't be doing code reviews or maintaining the code or whatnot, it's not at all the same as getting someone to take the time to produce a large piece of functionality for Phobos and might ultimately end up not taking much time from the person leading the project. But someone still needs to step up and do it. And no one has done that. Several have suggested it, and a number of us have agreed that it's a good idea, but as long as no one actually takes the initiative and actually does it, it's just a nice idea. - Jonathan M Davis

Separate project? That kind of defeats the entire purpose: Making the to-be-tested module readily available to everyone... -- Alex Rønne Petersen alex lycus.org http://lycus.org
Dec 05 2012
next sibling parent =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:
On 05-12-2012 23:42, Jonathan M Davis wrote:
 On Wednesday, December 05, 2012 22:52:09 Alex Rønne Petersen wrote:
 Separate project? That kind of defeats the entire purpose: Making the
 to-be-tested module readily available to everyone...

It makes sense to me to have a separate package in Phobos for stuff that has already been reviewed and is planned to be put into Phobos but we want to make sure that it's 100% solid before committing to its API permanently. It does not make sense to me to other stuff in there. That would be endorsing stuff that isn't necessarily even close to a good enough state to go into Phobos. And how would you even decide what went in there? I really don't think that any kind of incubator project should go into Phobos-proper. - Jonathan M Davis

How do we decide, today, what goes into Phobos proper? 1. The community expresses some kind of common need. 2. There is universal agreement that the feature should be in Phobos. 3. Somebody takes the time to write a module for inclusion. 4. The module is submitted for review. 5. If review passes, the module is included; otherwise, it is refined and goto 4. It's usually the community that drives the decisions about what's appropriate for Phobos and what isn't (i.e. at point 2). If we reach stage 3 in the aforementioned process, the module can be included in the staging package in Phobos proper for field testing if this is necessary before the initial review (this will obviously vary on a case-by-case basis). If, after the staging period (say, 3 months AKA a release cycle), there is agreement that the module doesn't fit into Phobos after all (for whatever reason), it can simply be removed from the staging package. Otherwise, lessons learned from the staging period can be taken into account, the module refined, and submitted for review (point 4). This seems like a perfectly sane model to me. -- Alex Rønne Petersen alex lycus.org http://lycus.org
Dec 05 2012
prev sibling parent Brad Roberts <braddr slice-2.puremagic.com> writes:
On Wed, 5 Dec 2012, Alex R?nne Petersen wrote:
 On 05-12-2012 22:05, Jonathan M Davis wrote:
 On Wednesday, December 05, 2012 21:32:37 Alex R?nne Petersen wrote:
 On 05-12-2012 17:10, Jonathan M Davis wrote:
 On Wednesday, December 05, 2012 15:10:59 Pragma Tix wrote:
 Years ago I have suggested to establish an incubator project.

It's been suggested a few times, but no one ever does it. As with a lot of things around here, unless someone steps up and champions it, it isn't going to happen. - Jonathan M Davis

In this case, though, there isn't really much to *do*. People just need to send pull requests when they have a module they feel is ready for field testing.

Yes. But I think that that's completely inappropriate for putting into Phobos. We would actually need a separate project for that. And someone would need to set up and manage that project. Given that they wouldn't be doing code reviews or maintaining the code or whatnot, it's not at all the same as getting someone to take the time to produce a large piece of functionality for Phobos and might ultimately end up not taking much time from the person leading the project. But someone still needs to step up and do it. And no one has done that. Several have suggested it, and a number of us have agreed that it's a good idea, but as long as no one actually takes the initiative and actually does it, it's just a nice idea. - Jonathan M Davis

Separate project? That kind of defeats the entire purpose: Making the to-be-tested module readily available to everyone...

IMHO it must be separate to gain the speed and evolution that experimental suggests. It should not be tied to the phobos release schedule. Additionally, the set of people should be separate (though could overlap). There's more downsides to being in the same repo and same release process as there are upsides, as far as I see it. My 2 cents, Brad
Dec 05 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday, December 05, 2012 15:10:59 Pragma Tix wrote:
 Years ago I have suggested to establish an incubator project.

It's been suggested a few times, but no one ever does it. As with a lot of things around here, unless someone steps up and champions it, it isn't going to happen. - Jonathan M Davis
Dec 05 2012
prev sibling next sibling parent "angel" <andrey.gelman gmail.com> writes:
In the Linux community it is called 'staging', rather than 
'experimental'.
A module has to be of some quality level to be accepted to 
staging, where it may be reviewed and tested by the more 
adventurous people.
 From staging, the module proceeds to the mainline, or back to 
hell.
In Linux this approach pretty much proves itself.
Dec 05 2012
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 5 December 2012 at 08:43:39 UTC, Jonathan M Davis 
wrote:
 Whether we want some kind of experimental set of modules in 
 Phobos which
 _could_ be reviewed, I don't know. I'm inclined to think that 
 that's a bad
 idea. However, it might be a good idea if we had a separate 
 project (akin to
 Boost perhaps) with a lower standard for inclusion and minimal 
 long term
 stability so that more modules could be tested and tried out 
 before going
 through the actual review process. It would also potentially 
 make it easier
 for programmers to take other people's code and make it Phobos 
 ready when the
 original programmers are willing to write it in the first place 
 but not got the
 extra mile to make it Phobos ready, as they could put into that 
 incubator
 project to be used, abused, and improved by others.

 Regardless, I think that we need to look at finding ways to 
 make sure that
 major inclusions to Phobos get more actual field testing before 
 being put into
 Phobos in their final form. The review process is great, but I 
 don't think that
 it's always enough to make sure that a module's API is really 
 going to work
 long term. And if we want long term stability, we need to make 
 sure that new
 modules are really ready to have their APIs essentially frozen 
 when they make
 it into Phobos proper (allowing further additions but avoiding 
 breaking
 compatibility with any changes that are made).

I think D isn't big enough to get that into another project. Many people wont use that just because you need a setup when they would if they don't need to, while knowing they use something that isn't casted in stone. experimental seems the right name to me : - It is long. - It is explicit.
Dec 05 2012
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Dec 05, 2012 at 06:47:38PM +0100, angel wrote:
 In the Linux community it is called 'staging', rather than
 'experimental'.
 A module has to be of some quality level to be accepted to staging,
 where it may be reviewed and tested by the more adventurous people.
 From staging, the module proceeds to the mainline, or back to hell.
 In Linux this approach pretty much proves itself.

I like the name 'staging'. Spelling it out in full is also good, so that people who use it only do so deliberately. Like staging.io for the prospective new std.io, or staging.graph for a hypothetical graph algorithms module destined for std.graph, it's immediately clear that these are experimental/staging modules, not yet official. It can even be provided as part of Phobos (or rather, part of the Phobos package, under the staging namespace) so that it's convenient for people to test things out. To maximize test coverage by actual-use code, I don't think it's a good idea to make people jump yet another hurdle to install a staging package. Bundle it with Phobos and let everyone have ready access to it, with the understanding that APIs from staging will not be stable, and things may change drastically without warning. Requiring people to download, install, and configure yet another package only raises the barrier, and we'd probably get few or no testers at all, which defeats the purpose of staging. T -- Ruby is essentially Perl minus Wall.
Dec 05 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, December 05, 2012 21:32:37 Alex Rønne Petersen wrote:
 On 05-12-2012 17:10, Jonathan M Davis wrote:
 On Wednesday, December 05, 2012 15:10:59 Pragma Tix wrote:
 Years ago I have suggested to establish an incubator project.

It's been suggested a few times, but no one ever does it. As with a lot of things around here, unless someone steps up and champions it, it isn't going to happen. - Jonathan M Davis

In this case, though, there isn't really much to *do*. People just need to send pull requests when they have a module they feel is ready for field testing.

Yes. But I think that that's completely inappropriate for putting into Phobos. We would actually need a separate project for that. And someone would need to set up and manage that project. Given that they wouldn't be doing code reviews or maintaining the code or whatnot, it's not at all the same as getting someone to take the time to produce a large piece of functionality for Phobos and might ultimately end up not taking much time from the person leading the project. But someone still needs to step up and do it. And no one has done that. Several have suggested it, and a number of us have agreed that it's a good idea, but as long as no one actually takes the initiative and actually does it, it's just a nice idea. - Jonathan M Davis
Dec 05 2012
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Dec 05, 2012 at 10:52:09PM +0100, Alex Rønne Petersen wrote:
 On 05-12-2012 22:05, Jonathan M Davis wrote:
On Wednesday, December 05, 2012 21:32:37 Alex Rønne Petersen wrote:


In this case, though, there isn't really much to *do*. People just
need to send pull requests when they have a module they feel is
ready for field testing.

Yes. But I think that that's completely inappropriate for putting into Phobos. We would actually need a separate project for that. And someone would need to set up and manage that project. Given that they wouldn't be doing code reviews or maintaining the code or whatnot, it's not at all the same as getting someone to take the time to produce a large piece of functionality for Phobos and might ultimately end up not taking much time from the person leading the project. But someone still needs to step up and do it. And no one has done that. Several have suggested it, and a number of us have agreed that it's a good idea, but as long as no one actually takes the initiative and actually does it, it's just a nice idea.


 Separate project? That kind of defeats the entire purpose: Making
 the to-be-tested module readily available to everyone...

Yeah, doing a separate project pretty much defeats the entire purpose. It will have limited availability (not everyone will know where to look for it) and limited incentive to be used (why install another package manually, much less an *experimental* one?). I say we should just create a staging/ directory in Phobos alongside std/, and put the experimental/staging stuff there. With a big fat notice that everything in staging/ is fair game for incompatible API changes and breakages without any warning. This will (1) encourage more users to test the new code, since it's already sitting right there alongside the official part of Phobos, and (2) when it comes time to accept it into std/, it's a trivial rename rather than a possibly messy code merge from a foreign repo. T -- To provoke is to call someone stupid; to argue is to call each other stupid.
Dec 05 2012
prev sibling next sibling parent "Joshua Niehus" <jm.niehus gmail.com> writes:
On Wednesday, 5 December 2012 at 21:05:30 UTC, Jonathan M Davis 
wrote:
 But I think that that's completely inappropriate for putting 
 into Phobos.

I'd love to try out the new stuff if its convient enough... I dont want to go jumping around github, pick up the "staging" material, put it into my current dev system and then play. It would be *much* nicer if its part of Phobos
Dec 05 2012
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Wednesday, 5 December 2012 at 22:32:14 UTC, Joshua Niehus 
wrote:
 It would be *much* nicer if its part of Phobos

We could always make non-phobos dev projects included in the dmd distribution zip... then the dev processes remain whatever and people still get them easily enough.
Dec 05 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, December 05, 2012 22:52:09 Alex Rønne Petersen wrote:
 Separate project? That kind of defeats the entire purpose: Making the
 to-be-tested module readily available to everyone...

It makes sense to me to have a separate package in Phobos for stuff that has already been reviewed and is planned to be put into Phobos but we want to make sure that it's 100% solid before committing to its API permanently. It does not make sense to me to other stuff in there. That would be endorsing stuff that isn't necessarily even close to a good enough state to go into Phobos. And how would you even decide what went in there? I really don't think that any kind of incubator project should go into Phobos-proper. - Jonathan M Davis
Dec 05 2012
prev sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 5 December 2012 at 22:42:59 UTC, Jonathan M Davis 
wrote:
 On Wednesday, December 05, 2012 22:52:09 Alex Rønne Petersen 
 wrote:
 Separate project? That kind of defeats the entire purpose: 
 Making the
 to-be-tested module readily available to everyone...

It makes sense to me to have a separate package in Phobos for stuff that has already been reviewed and is planned to be put into Phobos but we want to make sure that it's 100% solid before committing to its API permanently. It does not make sense to me to other stuff in there. That would be endorsing stuff that isn't necessarily even close to a good enough state to go into Phobos. And how would you even decide what went in there? I really don't think that any kind of incubator project should go into Phobos-proper. - Jonathan M Davis

That make sense to me. Isn't it what we are talking about here ?
Dec 05 2012