www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Next focus: PROCESS

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Hello all,


Walter and I have had a long discussion following his trip to Australia. 
Following the current sprint for Win64 (which we all, I think, agree was 
long overdue and had to be done), the main area we need to work on (as 
I'm sure many would agree) is improving our process, in particular the 
approach to releasing the compiler and standard library.

Walter has delegated me to lead this effort, and agreed to obey with 
whatever process comes forward as long as it's reasonably clear and simple.

In turn, I'll be collecting thoughts and opinions in this thread, 
distilled from previous discussions. We should develop a few simple git 
scripts (such as git-start-new-feature or git-start-bugfix etc) 
supported by an equally simple piece of documentation describing how to 
start a new release, fix a bug, experiment with a new feature, and such.

(One piece that has been brought forward is 
http://nvie.com/posts/a-successful-git-branching-model/ - something to 
keep in mind.)

Please chime in with ideas and thoughts.


Thanks,

Andrei
Dec 10 2012
next sibling parent "Brad Anderson" <eco gnuk.net> writes:
On Monday, 10 December 2012 at 23:41:25 UTC, Andrei Alexandrescu 
wrote:
 Hello all,


 Walter and I have had a long discussion following his trip to 
 Australia. Following the current sprint for Win64 (which we 
 all, I think, agree was long overdue and had to be done), the 
 main area we need to work on (as I'm sure many would agree) is 
 improving our process, in particular the approach to releasing 
 the compiler and standard library.

 Walter has delegated me to lead this effort, and agreed to obey 
 with whatever process comes forward as long as it's reasonably 
 clear and simple.

 In turn, I'll be collecting thoughts and opinions in this 
 thread, distilled from previous discussions. We should develop 
 a few simple git scripts (such as git-start-new-feature or 
 git-start-bugfix etc) supported by an equally simple piece of 
 documentation describing how to start a new release, fix a bug, 
 experiment with a new feature, and such.

 (One piece that has been brought forward is 
 http://nvie.com/posts/a-successful-git-branching-model/ - 
 something to keep in mind.)

As someone who uses the git workflow described in this link every single day let me say I'm a big fan. It's so well documented (and widely used) that it should be pretty easy for everyone to adopt and it provides a lot of real benefits to a project. From the look of it most contributors already use feature branches for their pull requests so this should be a pretty simple transition.
 Please chime in with ideas and thoughts.

I like the idea of having a release manager (that changes to whoever volunteers for each release). It'll take some of the release time pressure off of Walter (at least for the times he isn't the release manager) and raise D's bus factor (which is very close to being above 1 these days) as the release process is documented and shared among several individuals. I imagine the role of release manager being fairly laid back though (it is a volunteer job still). Their responsibilities would be rolling the betas and actual releases and just being the point-man who goes through and makes sure all the pull requests that are ready to merge are merged and last minute regressions are acknowledged. BA
Dec 10 2012
prev sibling next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Dec 10, 2012 at 06:41:25PM -0500, Andrei Alexandrescu wrote:
 Hello all,
 
 Walter and I have had a long discussion following his trip to
 Australia. Following the current sprint for Win64 (which we all, I
 think, agree was long overdue and had to be done), the main area we
 need to work on (as I'm sure many would agree) is improving our
 process, in particular the approach to releasing the compiler and
 standard library.

*applause* It's about time we did this right.
 Walter has delegated me to lead this effort, and agreed to obey with
 whatever process comes forward as long as it's reasonably clear and
 simple.

Good to know.
 In turn, I'll be collecting thoughts and opinions in this thread,
 distilled from previous discussions. We should develop a few simple
 git scripts (such as git-start-new-feature or git-start-bugfix etc)
 supported by an equally simple piece of documentation describing how
 to start a new release, fix a bug, experiment with a new feature,
 and such.

Sounds OK to me, though it seems unnecessarily complicated. What's wrong with just doing things like: # New feature git checkout develop # latest dev branch git branch -b brand_new_feature # implement said feature git commit -a # Merge feature into dev branch git checkout develop git merge brand_new_feature # New release candidate git checkout develop git branch -b 2.062-rc1 # prepare code for release git commit -a # Release git checkout master git merge 2.062-rc1 Seems to me these are all straightforward everyday git commands. The whole selling point of git is that branching/merging is so easy that you could do it multiple times a day within a single implementation effort. For example, I frequently find myself doing things like this: # I want to implement feature X today git checkout master # Implementation attempt #1 git checkout -b feature_X_1 vim feature.d # code code code # While coding feature.d, I got inspired by an idea of an # alternative implementation. So, put current code aside and # test idea on new branch. git commit -a git checkout -b feature_X_alt_impl vim feature.d # implement alternative approach # Turns out, alternate approach doesn't work. Go back to # original approach. (But leave alt_impl in case I decide later # to go back to it.) git checkout feature_X_1 vim feature.d # more code # I hit a difficult spot in the code. Put it aside and work on # other stuff first. git commit -a git checkout master vim main.d # do some routine maintenance, bugfixes, etc. git commit -a # Take afternoon nap. While resting I got another idea for # feature X. git checkout feature_X_1 vim feature.d # try out new idea # Turns out, it needs a big change. So don't commit to # feature_X_1, but use a separate branch. git checkout -b feature_X_new_idea vim feature.d # more code git commit -a # I decide that feature_X_new_idea is the way to go. So fold # into feature X branch. git checkout -b feature_X_1 git merge feature_X_new_idea # I don't need the new idea branch anymore now that it's been # adopted as the "official" way to implement X. Also, the # previous alternative approach was a dead-end, so junk that as # well. git branch -d feature_X_new_idea git branch -d feature_X_alt_impl # Continue working on X vim feature.d git commit -a # I decide that it's now stable enough to merge into the # mainline code. git checkout master git merge feature_X_1 This can happen several times a day when I'm doing a lot of coding.
 (One piece that has been brought forward is
 http://nvie.com/posts/a-successful-git-branching-model/ - something
 to keep in mind.)

Glanced at it briefly. I think the basic ideas behind it are sound. Whether or not we should adopt all of the specifics is up for discussion, but I think we should definitely incorporate the main ideas: (1) have a release-ready branch, (2) a separate dev branch, and (3) always use a separate branch for new features so that if things go wrong or it takes too long to finish implementation, development work on other stuff can continue without being held up by bugs in the new feature. Furthermore, in a git environment it's perfectly normal to have multiple "exploratory" or "sub-feature" branches within a single feature branch, like I illustrated above in exploring how to implement feature X. These are usually just local branches, but if the feature is complex enough, it can warrant pushing to github so that other devs can also play around with the alternative implementation approaches. Eventually the best approach can be merged back to the feature branch and the others discarded. Personally, I find that this approach is very profitable, because you don't have to worry that a particular breaking change may turn out to be a dead-end; do the experiment in a sub-branch and throw it away if it doesn't work. The cheapness of doing this allows one to explore ambitious or risky approaches without affecting the stable "traditional" implementation, and so opens up the possibility of discovering new and novel ways of doing things that are usually left unexplored. T -- It said to install Windows 2000 or better, so I installed Linux instead.
Dec 10 2012
next sibling parent =?ISO-8859-1?Q?Alex_R=F8nne_Petersen?= <alex lycus.org> writes:
On 11-12-2012 01:19, H. S. Teoh wrote:
 On Mon, Dec 10, 2012 at 06:41:25PM -0500, Andrei Alexandrescu wrote:
 Hello all,

 Walter and I have had a long discussion following his trip to
 Australia. Following the current sprint for Win64 (which we all, I
 think, agree was long overdue and had to be done), the main area we
 need to work on (as I'm sure many would agree) is improving our
 process, in particular the approach to releasing the compiler and
 standard library.

*applause* It's about time we did this right.
 Walter has delegated me to lead this effort, and agreed to obey with
 whatever process comes forward as long as it's reasonably clear and
 simple.

Good to know.
 In turn, I'll be collecting thoughts and opinions in this thread,
 distilled from previous discussions. We should develop a few simple
 git scripts (such as git-start-new-feature or git-start-bugfix etc)
 supported by an equally simple piece of documentation describing how
 to start a new release, fix a bug, experiment with a new feature,
 and such.

Sounds OK to me, though it seems unnecessarily complicated. What's wrong with just doing things like: # New feature git checkout develop # latest dev branch git branch -b brand_new_feature # implement said feature git commit -a # Merge feature into dev branch git checkout develop git merge brand_new_feature # New release candidate git checkout develop git branch -b 2.062-rc1 # prepare code for release git commit -a # Release git checkout master git merge 2.062-rc1

master must be the development branch. This is a pitfall the Rust guys walked into: People accidentally send their pull requests to the master branch instead of the develop branch which overall ends up being a mess when a reviewer doesn't notice the error before merging. Otherwise sounds good. -- Alex Rnne Petersen alex lycus.org http://lycus.org
Dec 10 2012
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/10/12 7:19 PM, H. S. Teoh wrote:
 Sounds OK to me, though it seems unnecessarily complicated.

FWIW there's also this: http://drewfradette.ca/a-simpler-successful-git-branching-model/ Andrei
Dec 10 2012
parent =?ISO-8859-1?Q?S=F6nke_Ludwig?= <sludwig outerproduct.org> writes:
Am 11.12.2012 03:02, schrieb Andrei Alexandrescu:
 On 12/10/12 7:19 PM, H. S. Teoh wrote:
 Sounds OK to me, though it seems unnecessarily complicated.

FWIW there's also this: http://drewfradette.ca/a-simpler-successful-git-branching-model/ Andrei

Just throwing this in: http://vibed.org/temp/branch-model-small.png It avoids the pull request problem that was mentioned (pull target for a new feature may often erroneously be master vs. develop) and treats the release branches as the stable branches from which release tags are directly published. This seems pretty close to the actual mode of development and a single master for releases, as in the "successful git branching" models, seems to be insufficient anyway; continuous parallel releases of 1.x and 2.x (or 2.0.x and 2.1.x) seem not to really fit in there.
Dec 11 2012
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Monday, 10 December 2012 at 23:41:25 UTC, Andrei Alexandrescu 
wrote:
 (One piece that has been brought forward is 
 http://nvie.com/posts/a-successful-git-branching-model/ - 
 something to keep in mind.)

In this workflow, only one version of the software is produced. But we have an extra need : the one to combine language evolution (that may break) and long term support. This require some adjustment. Here is how I see things : New feature are develloped in their own branches. When a feature is ready, it is included in the dev branch. The feature must be rebased on the dev branch to be included. From time to time, dev branch is forked. The forked branch only receive bug fixes. The second digit of the version is incremented (minor). I'll call it stabilization branch. When the fork branch is stable enough, a tag is made and we have a realease. The revision number is incremented. In the future, more bug fixes can be done in that branch and new tag created for new release. No let's focus on bug fixes. As feature, they are made in their own branches. The branch will then be merged in any stabilization branch where it make sense and in dev. master is the highest numbered stabilization branch. By doing so, we ensure that user that get sources have a preview of the next thing that will be shipped. I would advice against maintaining a lot of stabilization branches as this is more work and too much of them don't bring any benefit. 2 to 3 of them is the right amount. The proposal works well with github, which is a plus considering we are using it.
Dec 10 2012
prev sibling next sibling parent Matt Soucy <msoucy csh.rit.edu> writes:
On 12/10/2012 06:41 PM, Andrei Alexandrescu wrote:
 In turn, I'll be collecting thoughts and opinions in this thread,
 distilled from previous discussions. We should develop a few simple git
 scripts (such as git-start-new-feature or git-start-bugfix etc)
 supported by an equally simple piece of documentation describing how to
 start a new release, fix a bug, experiment with a new feature, and such.

One thing that I don't think I've seen mentioned: The person who started that git-flow concept created a git plugin to do exactly this for that workflow: https://github.com/nvie/gitflow -Matt
Dec 10 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Monday, 10 December 2012 at 23:41:25 UTC, Andrei Alexandrescu 
wrote:
 Please chime in with ideas and thoughts.


 Thanks,

 Andrei

Andrei, as you know, a lot has already been discussed here: http://forum.dlang.org/thread/k99coi$1q5e$1 digitalmars.com In summary, here's where I understood things were left at ... Instead of all of our ideas getting mixed around in multiple threads and eventually vanishing into the black hole, we had decided that we needed a fixed location to discuss and work on the issue, and a place where documents that describe the process can be developed and stored. The poster named "1100110" started a git repo here https://github.com/D-Programming-Language-Stable/dmd/wiki/Specification I'm not sure it should be called "D-Programming-Language-Stable", it should be named something like "D-Programming-Language-Dev-Process", and it should only be concerned with developing the new development and release process for D. --rt
Dec 10 2012
prev sibling next sibling parent "foobar" <foo bar.com> writes:
On Tuesday, 11 December 2012 at 02:02:54 UTC, Andrei Alexandrescu
wrote:
 On 12/10/12 7:19 PM, H. S. Teoh wrote:
 Sounds OK to me, though it seems unnecessarily complicated.

FWIW there's also this: http://drewfradette.ca/a-simpler-successful-git-branching-model/ Andrei

First of all - Yay! There are still a few open questions that need to be decided before a suitable process can be defined. a. Should we have more than one stable release and provide support branches? That means we have version X which is latest stable with all the latest (stable) features included and we have an older Y version that only receives bug-fixes and is intended for support (think of Ubuntu's LTE versions). If we have such additional versions, how many do we have, for how long? My opinion - (based on Walter's) you can always download a previous version of the compiler. I'd say we should _at most_ support _one_ previous stable version with critical bug fixes only. we could always decide to support ad-hoc additional specific versions, for example before we introduce major paradigm shifting changes and this also depends on available manpower. B. should we have public pre-release versions? I think we should release additional "releases" - call them beta, pre-release, release candidates, whatever. These are for staging changes, allowing to field test new features and language changes before they are made final. Also allowing users to adjust their code-bases. Given the above, I think the "simpler flow" model above looks very promising and we should adopt it with minor changes: 1. Both "staging" and "release" should be "public" - the website lists both the stable versions and the staging versions. 2. the development branch should be the master branch for easier integration with pull requests. The staging releases (let's call them beta) will provide previews of upcoming and experimental features to try out without the promise of a stable API and would be released _monthly_. The stable releases will be every 3-4 months and will guaranty a stable API. A user could download DMD-201212 (i.e. this month) and play around with UDA and of course report bugs, but knowing that this feature could change syntax or even be completely removed thus allowing the developers to test stuff out in the open without committing to anything.
Dec 11 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Tuesday, 11 December 2012 at 13:19:56 UTC, foobar wrote:
 First of all - Yay!

 There are still a few open questions that need to be decided
 before a suitable process can be defined.

 I'd say we should _at most_
 support _one_ previous stable version with critical bug fixes
 only.

I agree with that as well, although I think that after a new major "stable" release, the previously stable should be supported (at a minimum kept available for download) for some time until the new stable is fully stabilized and most people have been able to adopt it. It may be good enough to just and pick and choose if the previous stable should get certain bug fixes or not until the support period runs out.
 B. should we have public pre-release versions?

A lot of people will want to use the latest available changes for actual development, so the "testing" or "pre-release" branch should be public and kept reasonably stable, although anything can happen, so it's not considered "stable", just "stable enough" given that it may be supporting new features and improvements that have been selected for the next major stable release.
 I think we should release additional "releases" - call them 
 beta,
 pre-release, release candidates, whatever. These are for staging
 changes, allowing to field test new features and language 
 changes
 before they are made final. Also allowing users to adjust their
 code-bases.

I think you'll need at a minimum experimental branches for testing out new ideas, the main development branch witch is considered unstable (the master branch is probably best for this one as was suggested), a pre-release or testing branch that is used for preparing for the next major stable release, and of course the current stable branch which only receives bug fixes up until the next pre-release branch is merged into stable. One more thing, is that we should adopt a version numbering system that is appropriate to indicate major, minor, and bug fix releases. The method of major.minor.revision can work well for us, but there may be alternatives that will work even better depending on what the process ends up being. What I'd hate to see continuing, is a major new release going out with no indication in the version number that it is a major new release as opposed to a minor revision change. For example, the current DMD stable is 2.060, and the next release will be 2.061, but it includes brand new poorly tested features, and one of them is still being debated on, therefore it may be subject to change. The next release will be anything but a minor update and it should not even be considered as a stable release, it's more like a pre-release version for testing and for adoption by those who absolutely need the latest "reasonably stable" version for their development work. --rt
Dec 11 2012
prev sibling next sibling parent "foobar" <foo bar.com> writes:
On Tuesday, 11 December 2012 at 19:57:55 UTC, Rob T wrote:
 On Tuesday, 11 December 2012 at 13:19:56 UTC, foobar wrote:
 First of all - Yay!

 There are still a few open questions that need to be decided
 before a suitable process can be defined.

 I'd say we should _at most_
 support _one_ previous stable version with critical bug fixes
 only.

I agree with that as well, although I think that after a new major "stable" release, the previously stable should be supported (at a minimum kept available for download) for some time until the new stable is fully stabilized and most people have been able to adopt it. It may be good enough to just and pick and choose if the previous stable should get certain bug fixes or not until the support period runs out.

By support I meant specifically _bug fixes_. You can already download all previous released versions from the website and in no way am I arguing to change that policy. Even if we ever get to a point where we don't want to keep older releases which I doubt very much (each zip is only a few MBs and therefore insignificant on today's storage) - we could still easily just checkout the specific release tagged in git and just build that.
 B. should we have public pre-release versions?

A lot of people will want to use the latest available changes for actual development, so the "testing" or "pre-release" branch should be public and kept reasonably stable, although anything can happen, so it's not considered "stable", just "stable enough" given that it may be supporting new features and improvements that have been selected for the next major stable release.

This is precisely what I addressed bellow. we have monthly build of our staging branch - call them monthly betas that include new upcoming features that are already stable enough to be released to the public for field testing and are tentative for the next actual release but until these feature actually get to a release they hold no guaranties and can be further modified based on the wider public testing - including changes in API. Once released, they do hold such guaranties of API stability. So these monthly betas will provide preview of language changes and allow people to prepare for future changes and also provide feedback.
 I think we should release additional "releases" - call them 
 beta,
 pre-release, release candidates, whatever. These are for 
 staging
 changes, allowing to field test new features and language 
 changes
 before they are made final. Also allowing users to adjust their
 code-bases.

I think you'll need at a minimum experimental branches for testing out new ideas, the main development branch witch is considered unstable (the master branch is probably best for this one as was suggested), a pre-release or testing branch that is used for preparing for the next major stable release, and of course the current stable branch which only receives bug fixes up until the next pre-release branch is merged into stable.

See comment above. the pre-release will contain new features already stable enough to be consumes by the general public _before_ we the developers are ready to a commit finally to their API. E.g. Walter's release of DMD with attributes that was already tested and working but after release people argued about changing its syntax from [attributes] to (attributes). developers can have their own experimental branches for their own tests and new ideas, but once a feature reaches pre-release it should already be tested and working and ready for public consumption without commiting to a final API.
 One more thing, is that we should adopt a version numbering 
 system that is appropriate to indicate major, minor, and bug 
 fix releases. The method of major.minor.revision can work well 
 for us, but there may be alternatives that will work even 
 better depending on what the process ends up being.

I really don't care about the numbering scheme and this is irrelevant to the topic of this discussion. We are discussing the PROCESS of development. How the releases are tagged is completely beside the point and could be named after sweet delights, cat names, Parks or even digits of PI. I really don't care and it really is _not important_. This is one of those fundamental things that are required to truly understand git - versions are the _content_ (code) they contain and are identified by a hash of that content. This is pure bikesheding but why not: Why not extend the astronomical theme to releases as well? What would you say about the latest Pluto release of DMD? ;) (Yeah, I know this is already used by the eclipse project..)
 What I'd hate to see continuing, is a major new release going 
 out with no indication in the version number that it is a major 
 new release as opposed to a minor revision change. For example, 
 the current DMD stable is 2.060, and the next release will be 
 2.061, but it includes brand new poorly tested features, and 
 one of them is still being debated on, therefore it may be 
 subject to change. The next release will be anything but a 
 minor update and it should not even be considered as a stable 
 release, it's more like a pre-release version for testing and 
 for adoption by those who absolutely need the latest 
 "reasonably stable" version for their development work.

 --rt

Dec 11 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Tuesday, 11 December 2012 at 23:15:27 UTC, foobar wrote:
 By support I meant specifically _bug fixes_. You can already 
 download all previous released versions from the website and in 
 no way am I arguing to change that policy.
 Even if we ever get to a point where we don't want to keep 
 older releases which I doubt very much (each zip is only a few 
 MBs and therefore insignificant on today's storage) - we could 
 still easily just checkout the specific release tagged in git 
 and just build that.

That's what I meant too, I agree with you on this point.
 This is precisely what I addressed bellow. we have monthly 
 build of our staging branch - call them monthly betas that 
 include new upcoming features that are already stable enough to 
 be released to the public for field testing and are tentative 
 for the next actual release but until these feature actually 
 get to a release they hold no guaranties and can be further 
 modified based on the wider public testing - including changes 
 in API. Once released, they do hold such guaranties of API 
 stability. So these monthly betas will provide preview of 
 language changes and allow people to prepare for future changes 
 and also provide feedback.

Having a pre-release build that can be downloaded and installed would be very nice to have if that's what you mean. Currently, to get DMD 2.061 "alpha" I have to jump through hoops to get it to a point where it is operational, and that is not so good as it limits severely how many users will be able to test it out.
 See comment above. the pre-release will contain new features 
 already stable enough to be consumes by the general public 
 _before_ we the developers are ready to a commit finally to 
 their API. E.g. Walter's release of DMD with attributes that 
 was already tested and working but after release people argued 
 about changing its syntax from [attributes] to  (attributes).

 developers can have their own experimental branches for their 
 own tests and new ideas, but once a feature reaches pre-release 
 it should already be tested and working and ready for public 
 consumption without commiting to a final API.

OK, but who has already tested it and how many people have been able to test it and comment on it? I was thinking more along the lines of how Debian does it, with a 4 staged release process: experimental => unstable => testing (pre-release) => stable. We could probably do away with a common "experimental" branch, leaving 3 common branches. A common "unstable branch" will allow more people to test out newly introduced ideas well before they become merged into pre-release, and that may help avoid problems as we're seeing with the sudden introduction of UDA's. I am hoping that we can have a pre-release version that will be stable enough to be used on real-world applications, knowing that some things can change but not in a major way. If I don't mind dealing with major changes, then I would instead make use of what's in the "unstable" branch, but I expect mostly compiler devs will be using unstable, and no one will be using it on anything important.
 I really don't care about the numbering scheme and this is 
 irrelevant to the topic of this discussion. We are discussing 
 the PROCESS of development. How the releases are tagged is 
 completely beside the point and could be named after sweet 
 delights, cat names, Parks or even digits of PI. I really don't 
 care and it really is _not important_.
 This is one of those fundamental things that are required to 
 truly understand git - versions are the _content_ (code) they 
 contain and are identified by a hash of that content.
 This is pure bikesheding but why not: Why not extend the 
 astronomical theme to releases as well? What would you say 
 about the latest Pluto release of DMD? ;)
 (Yeah, I know this is already used by the eclipse project..)

I'm very surprised by your comment on version numbers and I hope you are a significant minority holder of that view. How version numbers are assigned and what the number represents has to be made a defined part of the release process otherwise you'll end up with a meaningless random number or symbol (or nothing) for each new release, and the only information it will supply to you is "here's a new release". I don't see that as an improvement to the current situation, it just perpetuates the existing problem where users of the D compiler have no easy way to know if a new release is a major ground breaking one, or a minor one, or just a simple bug fix. In order to find out what a new release means in terms of the magnitude of changes, one has to read through a technical change log, and read through endless forum posts, and possibly even take a look at the commit log. Why not do something better than that when it's so easy to do? For example, if I see that the stable version has been updated, I want to quickly know if it's a bug fix update or something more significant. For mission critical applications, I'll jump on bug fix releases quickly, but I'll stay clear of any major releases up until it has had enough time to fully stabilize. I do this kind of filtering for all new releases of software that I use, and it's very easy to do when there's a version number that has meaning to it. --rt
Dec 11 2012
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Dec 12, 2012 at 02:03:56AM +0100, Rob T wrote:
[...]
 OK, but who has already tested it and how many people have been able
 to test it and comment on it? I was thinking more along the lines of
 how Debian does it, with a 4 staged release process: experimental =>
 unstable => testing (pre-release) => stable.

FYI, this is a misrepresentation of how Debian works. The actual process is unstable => testing => stable. I.e., most new packages are uploaded directly to unstable. The experimental branch is for packages that aren't fully working, may break your packaging system, contain untested code, etc.. Some stuff in experimental is never migrated to unstable, because they're for, um, experimenting with volatile or incomplete projects deemed unsuitable for general use. To apply the analogy to D development, this would be local git branches where, say, Walter is toying around with incomplete code for a new feature which may not even be compilable yet. The unstable branch would be more-or-less complete, compilable code that haven't been extensively tested yet (it more-or-less works, but may crash horribly, use at your own risk, etc.). After a period of limited use, it gets merged into testing, where a larger audience will begin using it. After it's been proven in the field, then only it gets put into stable. The experimental branch is really not needed as a formal part of the process, since the developers can just use temporary local git branches for that purpose without needing to push it to a public repository. So really, the Debian process isn't really that different from what you propose below:
 We could probably do away with a  common "experimental" branch,
 leaving 3 common branches.

T -- Microsoft is to operating systems & security ... what McDonalds is to gourmet cooking.
Dec 11 2012
prev sibling next sibling parent reply "Jesse Phillips" <Jesse.K.Phillips+D gmail.com> writes:
On Monday, 10 December 2012 at 23:41:25 UTC, Andrei Alexandrescu 
wrote:
 In turn, I'll be collecting thoughts and opinions in this 
 thread, distilled from previous discussions. We should develop 
 a few simple git scripts (such as git-start-new-feature or 
 git-start-bugfix etc) supported by an equally simple piece of 
 documentation describing how to start a new release, fix a bug, 
 experiment with a new feature, and such.

There may be some helpful scripts to facilitate a new workflow, but the examples given I feel just translate to learning a D specific git commands. Seriously you'll end up with a layer of abstraction that will make it harder for those familiar with git to understand and limit the ability of newer users to understand the state their working copy is in. git-start-new-feature: $ git checkout devbranch $ git checkout -b newfeaturename git-start-bugfix: $ git checkout stablebranch $ git checkout -b fixbugxxx How these should be folded back is the harder part. For example the bug fix should then be into stablebranch and devbranch. There is also the matter of pulling in changes of your stable or dev branch as you work, this is done with the add --rebase switch (and I believe opens the doors for more merge conflicts than a standard merge).
 (One piece that has been brought forward is 
 http://nvie.com/posts/a-successful-git-branching-model/ - 
 something to keep in mind.)

I would like to note that these simple flows do not mimic our world. We have three projects that need to be marked for release in tandem. It doesn't need address now, but it should be noted. I think it would be cool if the idea behind workbench project someone started was used officially to sync dmd/phobos/dlang.org and provide any of these needed deployment scripts. It would add to the complexity but much of it is already there this would just make it more apparent.
Dec 11 2012
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/12/12 1:37 AM, Jesse Phillips wrote:
 There may be some helpful scripts to facilitate a new workflow, but the
 examples given I feel just translate to learning a D specific git
 commands. Seriously you'll end up with a layer of abstraction that will
 make it harder for those familiar with git to understand and limit the
 ability of newer users to understand the state their working copy is in.

I disagree. I wrote a few git scripts for Facebook-specific workflows and a lot of people (including me) use them. Beyond that we need to have a wiki page with workflows: "To initiate a new feature" "To initiate a bug fix" "To initiate a library addition without formal review" "To initiate a library addition with formal review" ...
 git-start-new-feature:
 $ git checkout devbranch
 $ git checkout -b newfeaturename

 git-start-bugfix:
 $ git checkout stablebranch
 $ git checkout -b fixbugxxx

 How these should be folded back is the harder part. For example the bug
 fix should then be into stablebranch and devbranch.

All of these will be part of the instructions and the macros. The branch names will be standardized.
 There is also the matter of pulling in changes of your stable or dev
 branch as you work, this is done with the add --rebase switch (and I
 believe opens the doors for more merge conflicts than a standard merge).

All of these should be subsections in the documentation. "If your branch has conflicts" "If your branch became really messed up" ...
 (One piece that has been brought forward is
 http://nvie.com/posts/a-successful-git-branching-model/ - something to
 keep in mind.)

I would like to note that these simple flows do not mimic our world. We have three projects that need to be marked for release in tandem. It doesn't need address now, but it should be noted. I think it would be cool if the idea behind workbench project someone started was used officially to sync dmd/phobos/dlang.org and provide any of these needed deployment scripts. It would add to the complexity but much of it is already there this would just make it more apparent.

What is the workbench project? Andrei
Dec 12 2012
next sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
12/13/2012 12:13 AM, deadalnix пишет:
 Let me do some proposal. If people are happy with them, I can start some
 wikification.

 First here are the branch already existing :

 dev : development branch. It is used to merge new features.
 master : is the branch where the next version is stabilized (right now,
 2.061). It receive only bug fix. eventually new features if the feature
 is of strategic importance.
 2.60 : contains the current released version of D + bug fix.

 Release process :

 When branch 2.60 have enoug bug fixes to justify a new release, a tag is
 created (2.60.N+1). The amount and the importance of corrected bugs will
 dictate when the release is done?

 When master is stabilized enough, a new version of D can be released. A
 new branch s created from master, named 2.61 and a tag 2.61.0 is created :

 git checkout master
 git branch 2.61
 git checkout 2.61
 git tag 2.61.0

 Then, the dev version is merged into master and a new stabilization
 process begin :
 git checkout master
 git merge dev

 At some point, 2.60 will not receive any more bug fix, and will be
 considered very stable, or dead, depending on one's standard (both are
 kind of the same).

 I advice against maintaining a log of 2.XXX branches, as too much choice
 is as bad as no choice for the user, and it is no better for us. 2 is
 IMO the right amount.

 NB: I have chosen the 3 number system for branches names, this is up to
 debate, but as it is widely used in programming languages it is a good
 base. Anyway, let's not debate the name for too long, as it really don't
 matter, and concentrate on the workflow.

So far me likes.
 On Wednesday, 12 December 2012 at 16:14:44 UTC, Andrei Alexandrescu wrote:
 Beyond that we need to have a wiki page with workflows:

 "To initiate a new feature"

A new feature is developed in its own branch. The reference is the dev branch. In git it goes as follow : git checkout dev git branch featureName Development of the feature. You can work on many feature at the same time; each one in its own branch. You initiate them the same way and switch from one another with git checkout. Before submitting a feature, it has to be reabsed as follow : git checkout featureName git rebase dev

Just make sure your local dev is always in sync with upstream! Or rather add an 'upstream' remote as github.com/D-Programming-Language/phobos.git (dmd, druntime etc.) and always do: git pull --rebase upstream dev git push --force origin featureName before submitting pull via github.
 When the feature is ready to inclusion into dev, the dev can make a pull
 request.

 "To initiate a bug fix"

The same as for new feature, except that you start from the branch the bug is in. If the bug is in several branches, then the most stable one is choosen, as i is usually easier to bubble up the bugfix from a stable branch to dev than the other way around.
 "To initiate a library addition without formal review"

 "To initiate a library addition with formal review"

 ...

I don't really know what to do here. I guess bug fixes on the library can be done the same way, but something specific probably have to be done with new modules.
 All of these should be subsections in the documentation.

 "If your branch has conflicts"

If your branch has conflict, it is up to you to resolve them. The conflict will most likely occurs when rebasing. In such case, git status will inform you on which files are impacted. Edit the file to resolve the conflict and do git add file, then git rebase --continue . If you don't want to resolve the conflict now, you can abort the rebase by doing git rebase --abort
 "If your branch became really messed up"

Well this one is difficult, because it is unclear what is wrong, so unclear ho to solve it.

I find it to be 'git reset' to some early point, stash changes, fast forward from the upstream version to the head of current branch and stash pop your local changes. But it really depends on symptoms and what the end result should be discard changes?, save changes elsewhere but restore to something stable? which branch is messed up? It's an infinite list of questions.
 Is that good ?

I think it's good. But personally I'd expect: * master to be what you define as dev, because e.g. GitHub puts master as default target branch when making pull requests. Yeah, I know it's their quirk that it's easy to miss. Still leaving less burden to check the branch label for both reviewers and pull authors is a net gain. * And what you describe as master (it's a snapshot or a pre-release to me) to be named e.g. staging. And we can as well drop the dead branch 'dev' then. -- Dmitry Olshansky
Dec 13 2012
next sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
12/14/2012 3:34 AM, deadalnix пишет:
 On Thursday, 13 December 2012 at 20:48:30 UTC, deadalnix wrote:
 On Thursday, 13 December 2012 at 20:04:50 UTC, Dmitry Olshansky wrote:
 I think it's good.

 But personally I'd expect:

 * master to be what you define as dev, because e.g. GitHub puts
 master as default target branch when making pull requests. Yeah, I
 know it's their quirk that it's easy to miss. Still leaving less
 burden to check the branch label for both reviewers and pull authors
 is a net gain.

 * And what you describe as master (it's a snapshot or a pre-release
 to me) to be named e.g. staging.

 And we can as well drop the dead branch 'dev' then.

That sound also like a good plan to me.

Updated to follow the idea, plus added bunch of process description. Feel free to comment in order to refine this. http://wiki.dlang.org/Release_Process

I wasn't comfortable doing speculative edits to the wiki directly so here a few more comments: I think one of major goals is to be able to continue ongoing development while at the _same time_ preparing a release. To me number one problem is condensed in the statement "we are going to release do not merge anything but regressions" the process should sidestep this "lock-based" work-flow. Could be useful to add something along these line to goals section. (i.e. the speed and smoothness is a goal) Second point is about merging master into staging - why not just rewrite it with master branch altogether after each release? master is the branch with correct history (all new stuff is rebased on it) thus new staging will have that too. -- Dmitry Olshansky
Dec 15 2012
next sibling parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
12/15/2012 11:03 PM, Brad Roberts пишет:
 On 12/15/2012 2:29 AM, Dmitry Olshansky wrote:

 I think one of major goals is to be able to continue ongoing development while
at the _same time_ preparing a release.
 To me number one problem is condensed in the statement "we are going to
release do not merge anything but regressions"
 the process should sidestep this "lock-based" work-flow. Could be useful to
add something along these line to goals
 section. (i.e. the speed and smoothness is a goal)

I've been staying out of this thread for the most part, but I feel the need to comment on this part specifically. It's quite common for most major projects to have a clear "we're wrapping up a release" phase where work _is_ restricted to bug fixing and stabilizing. They don't stop people from working off in their development branches (no one could effectively impose such restrictions even if they wanted to), but they _do_ tighten down on what's allowed to be merged. This is a forcing function that's just required. There's a lot of issues that otherwise won't get sufficient attention. If all it took was altruism then regressions would be fixed immediately, bugs would always be fixed in highest priority to lowest priority (assuming that could even be effectively designed), etc.

I understand the desire of focusing people attention on a priority but there is no denying the fact that only a small subset of folks that is able to (say) fix a regression in the compiler. What I'm trying to avoid here is a situation where the compiler team fights with last regressions but the phobos development is frozen. Or the other way around. There could be a couple of these ping-pongs during the process. And that's what I think we had and is suboptimal. It's part of process to have a frozen view t in a form of staging branch but keep the master branch going. Otherwise it's just more branches (and pain) for no particular purpose.
 Without the 'ok, guys, focus in this smaller more critical subset of bugs'
step, release branches would be created and
 never polished (or focused down to the release manager to do all the work if
he's that skilled and/or generous of his time).

In the grander scale it boils down to managing who does what. Basically there should be a release check-list that everybody knows about: a list of issues and takers that currently work on them (Bugzilla provides that and should be more heavily used). If it's more or less covered (or nobody else could do it) the others may go on with the development on master. The main problem I see with "everybody focus on these" is that I expect the number of contributors to grow but their area of expertise to get be more and more specialized (in general). Thus IMHO it won't scale.
 There's a phrase I'm trying to remember, but it's something to the effect that
'hope isn't a recipe for success.'
 Hoping that people fix regressions on release critical bugs isn't sufficient. 
Incentive and steering is required.  The
 desire to ungate master branch merges is one approach that's been shown to be
successful.

contributors by providing shorter feedback cycle. -- Dmitry Olshansky
Dec 15 2012
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/16/12 6:15 AM, Joseph Rushton Wakeling wrote:
 On 12/15/2012 09:39 PM, deadalnix wrote:
 Can we drop the LTS name ? It reminds me of ubuntu, and I clearly hope
 that
 people promoting that idea don't plan to reproduce ubuntu's scheme :
 - it is not suitable for a programming language (as stated 3 time now,
 so just
 read before why I won't repeat it).
 - ubuntu is notoriously unstable.

Call them "stable release cycles" if you like, which is what they are intended to be.

Just one tidbit of information: I talked to Walter and we want to build into the process the ability to modify any particular release. (One possibility is to do so as part of paid support for large corporate users.) That means there needs to be one branch per release. Andrei
Dec 16 2012
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/16/12 10:15 AM, Joseph Rushton Wakeling wrote:
 On 12/16/2012 04:05 PM, Andrei Alexandrescu wrote:
 Just one tidbit of information: I talked to Walter and we want to
 build into the
 process the ability to modify any particular release. (One possibility
 is to do
 so as part of paid support for large corporate users.) That means
 there needs to
 be one branch per release.

Can you expand on the requirements/goals there a bit? Would be useful to have a good sense of what kind of modifications you have in mind.

Right now we're using a tagging system for releases, implying releases are just snapshots of a continuum. But what we need is e.g. to be able to patch 2.065 to fix a bug for a client who can't upgrade right now. That means one branch per release.
 I don't think anything in what I've proposed (elsewhere in this thread,
 I'm afraid I haven't written it up on the wiki yet...) is incompatible
 with the notion of one branch per release.

I agree. Just wanted to add this to the general discussion. Andrei
Dec 16 2012
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/16/12 11:56 AM, foobar wrote:
 I don't see why that is a requirement (having a branch per release).
 We can still have a single stable branch with tags for releases and when
 Walter needs to provide special customizations he can always just branch
 off of the tagged release.

To the extent possible, a good process should facilitate the desirable scenarios. Clearly this is possible, but that doesn't mean we shouldn't make it easy within reason.
 This should be Walter's business and not part
 of the "official" community process.

I think this is rather shortsighted. Corporate support by Walter is only one possible scenario, but there are many others.
 in git terms, assuming we have a tagged release of 2.61
 $ git checkout -b 2.61-partner 2.61
 This branches off a new branch "2.61-partner" for the specific partner
 modification based off the contents of the "2.61" release.

 Also, this kinda messes with the notion of integration. A single stable
 branch helps prevent "forgotten" bug-fixes. I.e a critical bug was fixed
 on release N, what will ensure it will be included in release N+1? If
 Release N+1 is on the same branch than the bug-fix is included by
 default and prevented the need to perform a manual operation (a merge)
 that could be forgotten.

The prevalent use of the feature would be that a bug in a future release is back-patched onto an older release. Andrei
Dec 16 2012
parent Jacob Carlborg <doob me.com> writes:
On 2012-12-16 20:04, Jesse Phillips wrote:

 I see it unreasonable to maintain every release. If we need to make a
 special "maintenance" release then we directly handle the issue being
 requested instead of providing every fix applicable.

We could say that we support X number of older releases. -- /Jacob Carlborg
Dec 16 2012
prev sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sun, Dec 16, 2012 at 11:43:12PM +0100, David Nadlinger wrote:
 On Sunday, 16 December 2012 at 22:18:14 UTC, SomeDude wrote:
This sounds to me like a bad idea. And indeed, I haven't heard of
any other project doing this.

Having release branches is a common practice for many open source projects. For example, KDE creates a branch per minor release, with patch releases being made off that branch. LLVM also has a branch for each release.

Not just open source projects; at my day job, we do this as well. Development continues on the main branch, and releases are branched off this main branch at various point. We don't release immediately after branching, but run development in parallel for a period of time until the code to be released is well-tested and stabilized. Then a release is made. After a release is made, critical bugfixes and major issues are still being made to the release branch, and patch releases are issued periodically from the release branch. Sometimes we go up to two different release branches receiving bugfixes before the older release is phased out. Bugfixes made to the release branches are propagated to the development branch where applicable. Usually, if the corresponding piece of code hasn't changed much, the same bug exists in both places and generally the same fix applies to both of them. Usually, if the code has diverged significantly, the bug doesn't exist in the new code and the fix can just remain in the release branch. (Sometimes two different fixes are needed for the same bug 'cos the code has diverged a lot, but then it can be treated as two separate bug fixes.) Furthermore, sometimes a critical bug is found in the release branch but the fix is already in the development branch. In that case, we backport the fix from the dev branch, which may sometimes require significant changes in order to account for the code divergence between the two branches. Or if even that is insufficient, we have to make a fix specific to the release branch. Anyway, I'm saying all this to point out that we should not get too worked up about the exact details of how all these things work; in practice, things will always be far more complicated than the idealized model of the release process that we put on paper. Let's just concentrate on the most salient points of it -- what branches there are, when we use them, what is necessary to port bugfixes across them, etc., and leave the details to the actual people doing the work. Trying to dictate everything down to the fine points is a waste of time, because the core devs probably aren't going to follow it anyway. T -- Claiming that your operating system is the best in the world because more people use it is like saying McDonalds makes the best food in the world. -- Carl B. Constantine
Dec 16 2012
prev sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sun, Dec 16, 2012 at 10:05:58AM -0500, Andrei Alexandrescu wrote:
[...]
 Just one tidbit of information: I talked to Walter and we want to
 build into the process the ability to modify any particular release.
 (One possibility is to do so as part of paid support for large
 corporate users.) That means there needs to be one branch per
 release.

This can be done easily if we maintain each release as a separate git branch. It does not even require modifying the current process, since as I understand it we're already proposing to branch when making a release. So in theory, if there's an urgent need, we can sidestep the "maintain at most two stable releases at a time" part of the process, checkout an older release, apply a bugfix, release another patch, etc.. If we name the git branches properly, there shouldn't be a problem with the proliferation of branches, as it will be obvious what each branch is for. T -- Try to keep an open mind, but not so open your brain falls out. -- theboz
Dec 16 2012
prev sibling next sibling parent "foobar" <foo bar.com> writes:
On Wednesday, 12 December 2012 at 01:03:59 UTC, Rob T wrote:
 On Tuesday, 11 December 2012 at 23:15:27 UTC, foobar wrote:
 By support I meant specifically _bug fixes_. You can already 
 download all previous released versions from the website and 
 in no way am I arguing to change that policy.
 Even if we ever get to a point where we don't want to keep 
 older releases which I doubt very much (each zip is only a few 
 MBs and therefore insignificant on today's storage) - we could 
 still easily just checkout the specific release tagged in git 
 and just build that.

That's what I meant too, I agree with you on this point.
 This is precisely what I addressed bellow. we have monthly 
 build of our staging branch - call them monthly betas that 
 include new upcoming features that are already stable enough 
 to be released to the public for field testing and are 
 tentative for the next actual release but until these feature 
 actually get to a release they hold no guaranties and can be 
 further modified based on the wider public testing - including 
 changes in API. Once released, they do hold such guaranties of 
 API stability. So these monthly betas will provide preview of 
 language changes and allow people to prepare for future 
 changes and also provide feedback.

Having a pre-release build that can be downloaded and installed would be very nice to have if that's what you mean. Currently, to get DMD 2.061 "alpha" I have to jump through hoops to get it to a point where it is operational, and that is not so good as it limits severely how many users will be able to test it out.
 See comment above. the pre-release will contain new features 
 already stable enough to be consumes by the general public 
 _before_ we the developers are ready to a commit finally to 
 their API. E.g. Walter's release of DMD with attributes that 
 was already tested and working but after release people argued 
 about changing its syntax from [attributes] to  (attributes).

 developers can have their own experimental branches for their 
 own tests and new ideas, but once a feature reaches 
 pre-release it should already be tested and working and ready 
 for public consumption without commiting to a final API.

OK, but who has already tested it and how many people have been able to test it and comment on it? I was thinking more along the lines of how Debian does it, with a 4 staged release process: experimental => unstable => testing (pre-release) => stable. We could probably do away with a common "experimental" branch, leaving 3 common branches. A common "unstable branch" will allow more people to test out newly introduced ideas well before they become merged into pre-release, and that may help avoid problems as we're seeing with the sudden introduction of UDA's. I am hoping that we can have a pre-release version that will be stable enough to be used on real-world applications, knowing that some things can change but not in a major way. If I don't mind dealing with major changes, then I would instead make use of what's in the "unstable" branch, but I expect mostly compiler devs will be using unstable, and no one will be using it on anything important.
 I really don't care about the numbering scheme and this is 
 irrelevant to the topic of this discussion. We are discussing 
 the PROCESS of development. How the releases are tagged is 
 completely beside the point and could be named after sweet 
 delights, cat names, Parks or even digits of PI. I really 
 don't care and it really is _not important_.
 This is one of those fundamental things that are required to 
 truly understand git - versions are the _content_ (code) they 
 contain and are identified by a hash of that content.
 This is pure bikesheding but why not: Why not extend the 
 astronomical theme to releases as well? What would you say 
 about the latest Pluto release of DMD? ;)
 (Yeah, I know this is already used by the eclipse project..)

I'm very surprised by your comment on version numbers and I hope you are a significant minority holder of that view. How version numbers are assigned and what the number represents has to be made a defined part of the release process otherwise you'll end up with a meaningless random number or symbol (or nothing) for each new release, and the only information it will supply to you is "here's a new release". I don't see that as an improvement to the current situation, it just perpetuates the existing problem where users of the D compiler have no easy way to know if a new release is a major ground breaking one, or a minor one, or just a simple bug fix. In order to find out what a new release means in terms of the magnitude of changes, one has to read through a technical change log, and read through endless forum posts, and possibly even take a look at the commit log. Why not do something better than that when it's so easy to do? For example, if I see that the stable version has been updated, I want to quickly know if it's a bug fix update or something more significant. For mission critical applications, I'll jump on bug fix releases quickly, but I'll stay clear of any major releases up until it has had enough time to fully stabilize. I do this kind of filtering for all new releases of software that I use, and it's very easy to do when there's a version number that has meaning to it. --rt

To summarize: 1. Yes, I do suggest that "beta" versions will be available to download and prominently displayed on the download web-page. 2. The version scheme is meaningless. Please check out http://www.mozilla.org/en-US/firefox/channel/#firefox as an example. It's perfectly clear, you can choose what Mozilla calls a channel - i.e "release" or "beta". We can put both in a two column table with high-level description of changes and links to more-detailed change logs. 3. Debian is a HUGE project with lots of manpower. D is not. I think it is enough to have two _publicly visible download channels_ - release and beta (testing). Those are the only two that should be formally defined. There is no need to define prior stages of development as people can just informally create their own local branches for their own experiments and they could also push them upstream to share with other developers without any requirement to list an executable on the official download page. The source code is available on github - if you wanted to you could pull whatever branch and build it yourself at your own risk. 4. The only official executables on the download page are for two channels - release and beta. You can grab the latest stable "release" that only gets critical bug-fixes or the monthly beta that also contains preview features such as the upcoming user-defined attributes. You can also grab previous versions of both release channels by clicking the "previous versions" link. Anything else has zero guaranties and you are required to build yourself from source.
Dec 12 2012
prev sibling next sibling parent "Jesse Phillips" <Jessekphillips+D gmail.com> writes:
On Wednesday, 12 December 2012 at 16:14:44 UTC, Andrei 
Alexandrescu wrote:
 I disagree. I wrote a few git scripts for Facebook-specific 
 workflows and a lot of people (including me) use them.

May have been a knee jerk reaction. Good to hear it has been used.
 git-start-new-feature:
 $ git checkout devbranch
 $ git checkout -b newfeaturename

 git-start-bugfix:
 $ git checkout stablebranch
 $ git checkout -b fixbugxxx


 All of these will be part of the instructions and the macros. 
 The branch names will be standardized.

I'm not sure I follow on the name standardizing. Doesn't that make it so a person can only work on one bugfix and one feature at a time? The name used for the dev and stable branches should be standardized (I don't care much what master is, but I generally like it as the dev branch)
 All of these should be subsections in the documentation.

 "If your branch has conflicts"

 "If your branch became really messed up"

Yep, just pointing out where I see the difficulty and scripts may be of more use.
 What is the workbench project?


 Andrei

Found it, by Nathan Swan http://forum.dlang.org/post/ygwzndxbwzrxzlhuooty forum.dlang.org
Dec 12 2012
prev sibling next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 12/11/2012 12:41 AM, Andrei Alexandrescu wrote:
 In turn, I'll be collecting thoughts and opinions in this thread, distilled
from
 previous discussions. We should develop a few simple git scripts (such as
 git-start-new-feature or git-start-bugfix etc) supported by an equally simple
 piece of documentation describing how to start a new release, fix a bug,
 experiment with a new feature, and such.

 (One piece that has been brought forward is
 http://nvie.com/posts/a-successful-git-branching-model/ - something to keep in
 mind.)

 Please chime in with ideas and thoughts.

It seems to me that while process is clearly very important, what's maybe necessary first is to identify what the actual results of the process are intended to be. What should a 'stable version' of D actually mean, in practice? What should its lifetime be? Should there be interim releases in-between stable versions which can introduce e.g. breaking features, and should those be treated only as beta versions or as releases in their own right? How is that coupled (or decoupled) from releases of the library? etc. I'd personally rather get those issues addressed out before talking about git methods, which to be honest I suspect will be similar no matter what the release model. So, with that in mind, here are some thoughts/suggestions. People have talked about Debian's unstable => testing => stable model, but instead, why not consider something more akin to Ubuntu's LTS and interim releases? * STABLE/Long-term support releases would be guaranteed to receive bugfixes and have no breaking changes for 3 years. * A new LTS release would be made every 2 years, so that there would be a 1-year overlap between the current LTS release and the previous one. * Optionally, non-breaking new features could be included in LTS release updates during this timeframe, subject to strict testing. Such new feature updates should only go to the latest LTS release (so, during the 1-year overlap period, the older LTS would continue to receive bugfixes but not have new features added). The process for including new features should probably be more liberal for Phobos than for other parts of the D toolchain. * In between LTS releases, there should be regular (3- or 6-monthly?) interim releases that have gone through testing and are considered 'stable' in a working sense. These releases _can_ make breaking changes, subject to strict conditions, and should be more liberal about new feature inclusion in general. The idea should be that the interim releases converge towards the next LTS release, so the conditions for make breaking changes and the testing of new features both become more and more stringent the closer to the next LTS release you are. * For preference, deprecated features to be 'removed' by simply removing documentation and support for a feature rather than removing the feature itself (thinking here particularly of Phobos and Walter's observations). The time periods involved could of course be tweaked, e.g. an LTS release might be supported for 5 years and be released every 3 or 4. There could also be an even larger-scale cadence (10 years?) for really major redesigns. The above reflects my personal preference that I'd rather have a language which is willing to continue to iron out annoying idiosyncrasies or bad design decisions -- but which can still provide a good stable base for development, making sure that there is a good overlap period between LTS releases so that developers have a substantial amount of time to update their codebases.
Dec 12 2012
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Dec 12, 2012 at 07:04:17PM +0100, Joseph Rushton Wakeling wrote:
[...]
 It seems to me that while process is clearly very important, what's
 maybe necessary first is to identify what the actual results of the
 process are intended to be.  What should a 'stable version' of D
 actually mean, in practice?  What should its lifetime be?  Should
 there be interim releases in-between stable versions which can
 introduce e.g. breaking features, and should those be treated only
 as beta versions or as releases in their own right?  How is that
 coupled (or decoupled) from releases of the library? etc.
 
 I'd personally rather get those issues addressed out before talking
 about git methods, which to be honest I suspect will be similar no
 matter what the release model.

+1. Thanks for the reminder to not lose sight of the forest for the trees.
 So, with that in mind, here are some thoughts/suggestions.
 
 People have talked about Debian's unstable => testing => stable
 model, but instead, why not consider something more akin to Ubuntu's
 LTS and interim releases?
 
    * STABLE/Long-term support releases would be guaranteed to receive
      bugfixes and have no breaking changes for 3 years.
 
    * A new LTS release would be made every 2 years, so that there
      would be a 1-year overlap between the current LTS release and the
      previous one.

 The above reflects my personal preference that I'd rather have a
 language which is willing to continue to iron out annoying
 idiosyncrasies or bad design decisions -- but which can still
 provide a good stable base for development, making sure that there
 is a good overlap period between LTS releases so that developers
 have a substantial amount of time to update their codebases.

I like this idea!! I think this is much better than the current choice of either breaking existing code and annoying existing users or living with bad design decisions forever. One year is a good amount of time to upgrade people's codebases, so we won't be hampered by bad design decisions in the past, but can move forward with breaking changes. Freezing the language along with bad design decisions is what made C++ the monster that it is today. (Of course, the actual overlap times can be tweaked, but I definitely support the idea of being able to introduce breaking changes *and* not immediately alienate a good proportion of the userbase.) T -- The peace of mind---from knowing that viruses which exploit Microsoft system vulnerabilities cannot touch Linux---is priceless. -- Frustrated system administrator.
Dec 12 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Wednesday, 12 December 2012 at 10:22:32 UTC, foobar wrote:
 To summarize:
 2. The version scheme is meaningless. Please check out 
 http://www.mozilla.org/en-US/firefox/channel/#firefox as an 
 example. It's perfectly clear, you can choose what Mozilla 
 calls a channel - i.e "release" or "beta".

I agree Firefox has a meaningless version numbering system because Firefox adopted a meaningless version numbering system. I don't understand the rational behind the decision to switch from the previous one that was meaningful. What advantage do you gain by dropping a major, minor, and revision from your version numbering system?
 We can put both in a two column table with high-level 
 description of changes and links to more-detailed change logs.

There's no harm in doing that, but what would be most useful is knowing if the change is a major one or a minor one or just for bug fixes.
 3. Debian is a HUGE project with lots of manpower. D is not.

We all know this, but that does not mean we cannot do a lot better than we're doing now, and if we can get rid of the waste, like those endless debates that cannot be resolved with one branch, then there'll effectively be more manpower available.
 I think it is enough to have two _publicly visible download 
 channels_ - release and beta (testing). Those are the only two 
 that should be formally defined. There is no need to define 
 prior stages of development as people can just informally 
 create their own local branches for their own experiments and 
 they could also push them upstream to share with other 
 developers without any requirement to list an executable on the 
 official download page. The source code is available on github 
 - if you wanted to you could pull whatever branch and build it 
 yourself at your own risk.

But that's almost what we have now, and it does not work and it never will work. The problem with only two official branches, is that one has to be for a stable version, and the other has to be for the "new stuff", but there's two kinds of "new stuff". One kind is for breaking changes, or for introducing new concepts that have not been well tested. and the other kind is for field testing new concepts that are reasonably well understood and cause no breaking changes. What real-world programmers want to have, is a reasonably stable "beta" version that they can use for real work - this is NOT for playing around with, but for making things work for real. They want the beta because it has something that the stable does not yet have. What these guys don't want, and cannot put up with for real world programming, is an unstable alpha, but that's what you are proposing we continue on with, which is exactly what we have now and it does not work. In addition, once the beta/alpha is close to ready to be released into stable, it has to be frozen for a few months, where no new features are added and only bug fixes continue on, but then you've just lost your unstable branch! Have a look at the thread on the introduction of UDA's to see what's going on, and ask yourself if a two branch system will solve that kind of never ending problem.
 4. The only official executables on the download page are for 
 two channels - release and beta. You can grab the latest stable 
 "release" that only gets critical bug-fixes or the monthly beta 
 that also contains preview features such as the upcoming 
 user-defined attributes. You can also grab previous versions of 
 both release channels by clicking the "previous versions" link. 
 Anything else has zero guaranties and you are required to build 
 yourself from source.

That all works fine. What I think is glaringly missing is the unstable branch, so we need at a minimum 3 branches. --rt
Dec 12 2012
prev sibling next sibling parent "Jesse Phillips" <Jessekphillips+D gmail.com> writes:
On Wednesday, 12 December 2012 at 10:22:32 UTC, foobar wrote:
 To summarize:

 2. The version scheme is meaningless. Please check out 
 http://www.mozilla.org/en-US/firefox/channel/#firefox as an 
 example. It's perfectly clear, you can choose what Mozilla 
 calls a channel - i.e "release" or "beta".

This is a poor example as it doesn't show how the development team develops these versions. If we are going to have a branch supported separate from the beta/dev builds then we need a way to say that this stable version is newer but is not as new as the beta/dev. If we release 2.61 as a stable, we would then develop new features in which version? 2.62 beta 1? If so, when we release 2.61 with a bug fix which version do we release? 2.62? 2.61 stable 2? You are right that version numbers have absolutely no meaning, I don't remember if it was you, but they are also not as important as the process improvements. However if we assign meaning to numbers there can be benefit. Mozilla basically just did away with the Major version. (I say we do a Linux and leave the 2 then realize it has no meaning and up it to 3 when we reach the age of the 2.0 kernel) We should combine your beta with an added version number. 2.61.0 => bugfixes => 2.61.1 2.61.0 => newfeatures => 2.62.0 beta 1 2.62 => preparing for stabilizing => 2.62 rc1 just some thoughts.
Dec 12 2012
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 12 December 2012 at 18:04:43 UTC, Joseph Rushton 
Wakeling wrote:
 People have talked about Debian's unstable => testing => stable 
 model, but instead, why not consider something more akin to 
 Ubuntu's LTS and interim releases?

No, no, please. As already stated, distro release systems intend to solve the exact opposite problem of ours. If you want to look at what is done in the field, look at other programming languages.
Dec 12 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Wednesday, 12 December 2012 at 18:33:17 UTC, Jesse Phillips 
wrote:
 If we release 2.61 as a stable, we would then develop new 
 features in which version? 2.62 beta 1? If so, when we release 
 2.61 with a bug fix which version do we release? 2.62? 2.61 
 stable 2?

 You are right that version numbers have absolutely no meaning, 
 I don't remember if it was you, but they are also not as 
 important as the process improvements. However if we assign 
 meaning to numbers there can be benefit. Mozilla basically just 
 did away with the Major version. (I say we do a Linux and leave 
 the 2 then realize it has no meaning and up it to 3 when we 
 reach the age of the 2.0 kernel)

 We should combine your beta with an added version number.

The beta foobar speaks of is actually a combined alpha+beta, and that'll cause lots of problems and also delay releases because the combined alpha+beta will have to be frozen at some point prior to release as the next stable (i.e., frozen from enhancements that are not bug fixes only). Freezing the alpha+beta combo is not a problem for the beta portion, but it is a problem for the alpha portion that has no requirement to be frozen. You could just branch off the beta from the alpha into a special frozen branch, but then what are you doing? It makes no sense. Also important is that fact that there will be many programmers who could not wait for the next stable release and want to use the latest beta, but there is none, there's only and alpha+beta combo, and that'll mean the current problem we have right now will not be resolved where newly introduced features mess up the beta portion that people want to be able to rely on for real-world programming. The separation of alpha and beta allows for a beta that is at least far more stable than an alpha-beta combo, and that will satisfy the people who wish to move past the stable release into a beta, but wish to avoid the alpha entirely.
 2.61.0 => bugfixes => 2.61.1
 2.61.0 => newfeatures => 2.62.0 beta 1
 2.62 => preparing for stabilizing => 2.62 rc1

 just some thoughts.

A 3 point numbering system used to differentiate between the stable bug fix releases and the beta version works just fine, and there's no need to introduce descriptive symbols, although that can be useful to indicate what is stable, beta or alpha. In any event, I fully agree that the process is the most important item to work out first and foremost. When I mentioned version numbers, I simply wanted to remind people that it should also be on the table for change AND should be included into the process at some point to make it a well understood concept as opposed to what we have now. The response I got from Foobar was very surprising to me which is why I explored it further. As was suggested in another post, I'll do a little digging to see what is going on with the development cycle of other programming languages. Best not to try and reinvent the wheel if we can. One thing about D is that I suspect there's a sizable faction that does not wish to do what other programming languages are doing, and that's to avoid introducing breaking changes completely. Personally I'd rather see flawed concepts get fixed at some point rather than have to live with them forever, the point being that we may be somewhat unique with our development process requirements. --rt
Dec 12 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Wednesday, 12 December 2012 at 18:47:58 UTC, deadalnix wrote:
 On Wednesday, 12 December 2012 at 18:04:43 UTC, Joseph Rushton 
 Wakeling wrote:
 People have talked about Debian's unstable => testing => 
 stable model, but instead, why not consider something more 
 akin to Ubuntu's LTS and interim releases?

No, no, please. As already stated, distro release systems intend to solve the exact opposite problem of ours. If you want to look at what is done in the field, look at other programming languages.

Point taken. Do you have any links or further info to offer on this line of thinking? I'd like to investigate, so a starting point would be nice to have if you can offer one up. Thanks! --rt
Dec 12 2012
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 12 December 2012 at 19:33:00 UTC, Rob T wrote:
 On Wednesday, 12 December 2012 at 18:47:58 UTC, deadalnix wrote:
 On Wednesday, 12 December 2012 at 18:04:43 UTC, Joseph Rushton 
 Wakeling wrote:
 People have talked about Debian's unstable => testing => 
 stable model, but instead, why not consider something more 
 akin to Ubuntu's LTS and interim releases?

No, no, please. As already stated, distro release systems intend to solve the exact opposite problem of ours. If you want to look at what is done in the field, look at other programming languages.

Point taken. Do you have any links or further info to offer on this line of thinking? I'd like to investigate, so a starting point would be nice to have if you can offer one up. Thanks!

I did earlier, but let me repeat myself. Distro rely on many program, that they intend to integrate. Programming language rely on almost nothing but many programs rely on them. The problems you have to solve are the exact opposite of one another. I suggest considering how things are done in python, java, PHP or other successful languages.
Dec 12 2012
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
Let me do some proposal. If people are happy with them, I can 
start some wikification.

First here are the branch already existing :

dev : development branch. It is used to merge new features.
master : is the branch where the next version is stabilized 
(right now, 2.061). It receive only bug fix. eventually new 
features if the feature is of strategic importance.
2.60 : contains the current released version of D + bug fix.

Release process :

When branch 2.60 have enoug bug fixes to justify a new release, a 
tag is created (2.60.N+1). The amount and the importance of 
corrected bugs will dictate when the release is done?

When master is stabilized enough, a new version of D can be 
released. A new branch s created from master, named 2.61 and a 
tag 2.61.0 is created :

git checkout master
git branch 2.61
git checkout 2.61
git tag 2.61.0

Then, the dev version is merged into master and a new 
stabilization process begin :
git checkout master
git merge dev

At some point, 2.60 will not receive any more bug fix, and will 
be considered very stable, or dead, depending on one's standard 
(both are kind of the same).

I advice against maintaining a log of 2.XXX branches, as too much 
choice is as bad as no choice for the user, and it is no better 
for us. 2 is IMO the right amount.

NB: I have chosen the 3 number system for branches names, this is 
up to debate, but as it is widely used in programming languages 
it is a good base. Anyway, let's not debate the name for too 
long, as it really don't matter, and concentrate on the workflow.

On Wednesday, 12 December 2012 at 16:14:44 UTC, Andrei 
Alexandrescu wrote:
 Beyond that we need to have a wiki page with workflows:

 "To initiate a new feature"

A new feature is developed in its own branch. The reference is the dev branch. In git it goes as follow : git checkout dev git branch featureName Development of the feature. You can work on many feature at the same time; each one in its own branch. You initiate them the same way and switch from one another with git checkout. Before submitting a feature, it has to be reabsed as follow : git checkout featureName git rebase dev When the feature is ready to inclusion into dev, the dev can make a pull request.
 "To initiate a bug fix"

The same as for new feature, except that you start from the branch the bug is in. If the bug is in several branches, then the most stable one is choosen, as i is usually easier to bubble up the bugfix from a stable branch to dev than the other way around.
 "To initiate a library addition without formal review"

 "To initiate a library addition with formal review"

 ...

I don't really know what to do here. I guess bug fixes on the library can be done the same way, but something specific probably have to be done with new modules.
 All of these should be subsections in the documentation.

 "If your branch has conflicts"

If your branch has conflict, it is up to you to resolve them. The conflict will most likely occurs when rebasing. In such case, git status will inform you on which files are impacted. Edit the file to resolve the conflict and do git add file, then git rebase --continue . If you don't want to resolve the conflict now, you can abort the rebase by doing git rebase --abort
 "If your branch became really messed up"

Well this one is difficult, because it is unclear what is wrong, so unclear ho to solve it. Is that good ?
Dec 12 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Wednesday, 12 December 2012 at 18:04:43 UTC, Joseph Rushton 
Wakeling wrote:
 It seems to me that while process is clearly very important, 
 what's maybe necessary first is to identify what the actual 
 results of the process are intended to be.  What should a 
 'stable version' of D actually mean, in practice?
  What should its lifetime be?  Should there be interim releases 
 in-between stable versions which can introduce e.g. breaking 
 features, and should those be treated only as beta versions or 
 as releases in their own right?  How is that coupled (or 
 decoupled) from releases of the library? etc.

 I'd personally rather get those issues addressed out before 
 talking about git methods, which to be honest I suspect will be 
 similar no matter what the release model.

Agreed 100%. We're going about this completely ass backwards. Here's my stab at defining the main points to achieve: 1) We want improved stability and predictability with compiler releases to prevent a new release from unexpectedly breaking existing D code without significant prior warnings and time to prepare for it. 2) Stable releases should remain stable while being maintained, which means "bug fixes only". 3) We don't want to cast the language in stone, so new features must be allowed to be introduced in a more predictable and planned way that allows D programmers to confidently use the new features in their production code. 4) We want to allow D to continue to be enhanced in a way that allows the D compiler developers to do what they do best, while not interfering with the growing D user base who wish to rely on relative stability and predictability. Points 3 and 4 may seem the same, but the important difference is that there are two types of D users, those who use the language and those who develop the language and compiler (a person may do both of course), so the goals and requirements for each group of user are different and may conflict. If I write stuff like this in here, it'll just end up disappearing into the black hole. Can we agree to create a wiki page for defining what the main goals are, and what we wish to achieve? Once we have consensus on what is to be achieved, we can define a process and test it to see if it will achieve what we want. This is a much better approach than trying to build a process to achieve something that is not yet defined and agreed on. --rt
Dec 12 2012
prev sibling next sibling parent "Jesse Phillips" <Jessekphillips+D gmail.com> writes:
On Wednesday, 12 December 2012 at 21:17:31 UTC, Rob T wrote:

 Can we agree to create a wiki page for defining what the main 
 goals are, and what we wish to achieve?

I agree with the listed points.
Dec 12 2012
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Dec 12, 2012 at 10:17:30PM +0100, Rob T wrote:
[...]
 If I write stuff like this in here, it'll just end up disappearing
 into the black hole.
 
 Can we agree to create a wiki page for defining what the main goals
 are, and what we wish to achieve?
 
 Once we have consensus on what is to be achieved, we can define a
 process and test it to see if it will achieve what we want.
 
 This is a much better approach than trying to build a process to
 achieve something that is not yet defined and agreed on.

Please let's use the new wiki at wiki.dlang.org to coordinate this effort. We don't want this discussion to vanish into the ether after 2 months, and we end up back at square one. T -- There are two ways to write error-free programs; only the third one works.
Dec 12 2012
prev sibling next sibling parent "Jesse Phillips" <Jessekphillips+D gmail.com> writes:
On Wednesday, 12 December 2012 at 19:28:54 UTC, Rob T wrote:

 2.61.0 => bugfixes => 2.61.1
 2.61.0 => newfeatures => 2.62.0 beta 1
 2.62 => preparing for stabilizing => 2.62 rc1

 just some thoughts.

A 3 point numbering system used to differentiate between the stable bug fix releases and the beta version works just fine, and there's no need to introduce descriptive symbols, although that can be useful to indicate what is stable, beta or alpha.

So I say let us stay away from beta/alpha because those mean something different for different people and we need to define what they mean in D. 1. Bug fix only (how important is backwards compatibility here [when derived from a bug]) 2. Backward compatible additions 3. Vetting This seems to be the main releases that are being talk about. For which you are probably talking about the ability to use a 3 point system. The mapping I think people envision from this 1 & 2 => stable release 3 => dev release [side rant] When doing version numbering there are two main approaches I've seen used, maybe this has a formal name, but I'll call them front loaded and back loaded. Front loaded version generally comes from having goals/milestones for a version. In this case all the work for that version comes before the release. This is generally seen pre-1.0 as that will be the stable release. Back loaded is what generally comes from deciding on a new direction and where D currently is. In this case all the work is done within the version. This is generally seen in 2.0, software is released with the 2.xx stamp but is not fulfilling all the requirements needed to meet being the second version. [/side rant] I'd like to see what you envision for mapping these versions. vetting.additions.bug? Do we release a stable then increment vetting and develop under that (back loaded)? Or stable.vetting.additions/bug Do we increment stable upon release and continue increasing vetting as we release those for the volatile changes. (front loaded)
Dec 12 2012
prev sibling next sibling parent "Jesse Phillips" <Jesse.K.Phillips+D gmail.com> writes:
This is how I see the two approaches without wrecking havoc on 
current version numbers. I'll use [notes] to reference to the "A 
successful Git branching model" article.

---------------front loaded (2.0 still back loaded)
stable
2.61   // always release whole number as stable
2.61.1 [Seen in the master branch coming from hotfix branch]
2.61.2
dev
2.62  // develop on whole number
2.63 [not shown they only do a single stable release]
2.64 [these releases are inclusive of hotfix, feature, and 
release branch]
2.65 [first green circle]
2.66
2.67
rc
2.65rc1 // beat release into submission
2.65rc2 [this is the release branch]
----------------back loaded
stable
2.61.7 // always release in development
2.61.8 // continue work after release
2.61.9
dev
2.62 // always start development on whole
2.62.1
2.62.2
2.62.3
2.63
2.63.1
rc
2.62.3rc1 // beat release into submission
2.62.4rc2 // maybe change dev number
Dec 12 2012
prev sibling next sibling parent "foobar" <foo bar.com> writes:
On Wednesday, 12 December 2012 at 18:25:10 UTC, Rob T wrote:
 On Wednesday, 12 December 2012 at 10:22:32 UTC, foobar wrote:
 To summarize:
 2. The version scheme is meaningless. Please check out 
 http://www.mozilla.org/en-US/firefox/channel/#firefox as an 
 example. It's perfectly clear, you can choose what Mozilla 
 calls a channel - i.e "release" or "beta".

I agree Firefox has a meaningless version numbering system because Firefox adopted a meaningless version numbering system. I don't understand the rational behind the decision to switch from the previous one that was meaningful. What advantage do you gain by dropping a major, minor, and revision from your version numbering system?
 We can put both in a two column table with high-level 
 description of changes and links to more-detailed change logs.

There's no harm in doing that, but what would be most useful is knowing if the change is a major one or a minor one or just for bug fixes.
 3. Debian is a HUGE project with lots of manpower. D is not.

We all know this, but that does not mean we cannot do a lot better than we're doing now, and if we can get rid of the waste, like those endless debates that cannot be resolved with one branch, then there'll effectively be more manpower available.
 I think it is enough to have two _publicly visible download 
 channels_ - release and beta (testing). Those are the only two 
 that should be formally defined. There is no need to define 
 prior stages of development as people can just informally 
 create their own local branches for their own experiments and 
 they could also push them upstream to share with other 
 developers without any requirement to list an executable on 
 the official download page. The source code is available on 
 github - if you wanted to you could pull whatever branch and 
 build it yourself at your own risk.

But that's almost what we have now, and it does not work and it never will work. The problem with only two official branches, is that one has to be for a stable version, and the other has to be for the "new stuff", but there's two kinds of "new stuff". One kind is for breaking changes, or for introducing new concepts that have not been well tested. and the other kind is for field testing new concepts that are reasonably well understood and cause no breaking changes. What real-world programmers want to have, is a reasonably stable "beta" version that they can use for real work - this is NOT for playing around with, but for making things work for real. They want the beta because it has something that the stable does not yet have. What these guys don't want, and cannot put up with for real world programming, is an unstable alpha, but that's what you are proposing we continue on with, which is exactly what we have now and it does not work. In addition, once the beta/alpha is close to ready to be released into stable, it has to be frozen for a few months, where no new features are added and only bug fixes continue on, but then you've just lost your unstable branch! Have a look at the thread on the introduction of UDA's to see what's going on, and ask yourself if a two branch system will solve that kind of never ending problem.
 4. The only official executables on the download page are for 
 two channels - release and beta. You can grab the latest 
 stable "release" that only gets critical bug-fixes or the 
 monthly beta that also contains preview features such as the 
 upcoming user-defined attributes. You can also grab previous 
 versions of both release channels by clicking the "previous 
 versions" link. Anything else has zero guaranties and you are 
 required to build yourself from source.

That all works fine. What I think is glaringly missing is the unstable branch, so we need at a minimum 3 branches. --rt

I really didn't want to get dragged into the versioning scheme debate as it is just bikesheding in disguise but oh well.. I'll ask in return, what's the added benefit of your scheme? I honestly see no added benefit whatsoever. There's no intrinsic benefit compared to a changelog that lists "Jelly bean" after "Ice cream sandwich". How do you know what was added to a version? Simple, you have a "what's new" page that lists new major features denoted by "New feature:" tag and bug-fixes denoted by "Bug-Fix:" tag. Here's an example - http://www.mozilla.org/en-US/firefox/17.0.1/releasenotes/ Regarding the important (IMO) part of the discussion - 2 vs. 3 levels - I feel I haven't explained myself properly so let me clarify: I was taking into account what I thought was a reasonable assumption that D2 is in the process of stabilization and no major redesigns are planed. This means the "Major" version is 2 as in D2. My stable releases are meant to be the "Minor" releases on the D2 branch. These releases can introduce new backwards-compatible features and restricted breaking changes when absolutely required _over several releases_ with _a well defined migration(*) process_ - in other words support evolution. Major redesigns belong under a "revolution" such as the python3 effort. There is no point on setting time-lines for that. If and when we accumulate enough reasons/momentum/etc for such a major break we'll just create a new "major" version of D called D3 which right now is completely off the table. This is _not_ exactly what we have now because now new features are introduced right away, without public field testing (the beta channel). We have no defined criteria as to when a feature is mature enough to have it's syntax/API frozen and should be graduated from beta phase and integrated into stable. That is the point of defining the process. (*) A well defined migration path should span several stable versions and define these stages: 1. Warn the user that a feature is going away/ is going to be changed 2. Deprecate the feature and don't accept it at compilation (unless -d is used) 3. remove the feature completely.
Dec 12 2012
prev sibling next sibling parent "foobar" <foo bar.com> writes:
On Wednesday, 12 December 2012 at 18:33:17 UTC, Jesse Phillips 
wrote:
 On Wednesday, 12 December 2012 at 10:22:32 UTC, foobar wrote:
 To summarize:

 2. The version scheme is meaningless. Please check out 
 http://www.mozilla.org/en-US/firefox/channel/#firefox as an 
 example. It's perfectly clear, you can choose what Mozilla 
 calls a channel - i.e "release" or "beta".

This is a poor example as it doesn't show how the development team develops these versions. If we are going to have a branch supported separate from the beta/dev builds then we need a way to say that this stable version is newer but is not as new as the beta/dev. If we release 2.61 as a stable, we would then develop new features in which version? 2.62 beta 1? If so, when we release 2.61 with a bug fix which version do we release? 2.62? 2.61 stable 2? You are right that version numbers have absolutely no meaning, I don't remember if it was you, but they are also not as important as the process improvements. However if we assign meaning to numbers there can be benefit. Mozilla basically just did away with the Major version. (I say we do a Linux and leave the 2 then realize it has no meaning and up it to 3 when we reach the age of the 2.0 kernel) We should combine your beta with an added version number. 2.61.0 => bugfixes => 2.61.1 2.61.0 => newfeatures => 2.62.0 beta 1 2.62 => preparing for stabilizing => 2.62 rc1 just some thoughts.

Per my answer to Rob: D2 *is* the major version. releases should be minor versions and largely backwards compatible - some evolution is allowed given some reasonable restrictions like a proper migration path over several releases. Critical bug-fixes can go directly on stable releases but nothing else. Other enhancements, new features, etc, go first on monthly beta versions and need to pass acceptance tests & community approval before graduating to stable. So a big feature such as "User defined attributes" can spend several cycles of beta testing and can go redesigns until everyone agrees that the final design is worthy of inclusion in the next stable release.
Dec 12 2012
prev sibling next sibling parent "foobar" <foo bar.com> writes:
On Wednesday, 12 December 2012 at 19:28:54 UTC, Rob T wrote:

 The beta foobar speaks of is actually a combined alpha+beta, 
 --rt

No. I have already addressed this point before. Alpha is not part of the _official_ release process. As a developer I can create my own local branch "DMD-new_feature" to experiment with some new idea I had. I can later on push this upstream and share my new cool feature with other developers to play with. besides other developers, users can also fetch these new branches and build themselves. This is done _before_ it matures enough to be included in beta testing.
Dec 12 2012
prev sibling next sibling parent "Marco Nembrini" <marco.nembrini.co gmail.com> writes:
 I really don't care about the numbering scheme and this is 
 irrelevant to the topic of this discussion. We are discussing 
 the PROCESS of development. How the releases are tagged is 
 completely beside the point and could be named after sweet 
 delights, cat names, Parks or even digits of PI. I really don't 
 care and it really is _not important_.
 

I agree that the numbering scheme should be chosen after the process is defined and not before, and numbering schemes are meaningless (like firefox's) _unless_ they represent some guarantee that the developer is willing to make. In that case the can be very effective imho (see http://semver.org/ for an example). So if the process chosen makes guarantees about the project, those should be expressed by the version number. For example ubuntu's Server LTS guarantees 5 years of support, and the numbering scheme is year.month, so you know right away that ubuntu 8.04 is still supported right now. They could have named it Ubuntu Pluto and you could have looked up the end date for the support, but since you have to differentiate your releases anyway, you might as well use meaningful numbers.
Dec 12 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday, December 12, 2012 19:04:17 Joseph Rushton Wakeling wrote:
 It seems to me that while process is clearly very important, what's maybe
 necessary first is to identify what the actual results of the process are
 intended to be.  What should a 'stable version' of D actually mean, in
 practice? What should its lifetime be?  Should there be interim releases
 in-between stable versions which can introduce e.g. breaking features, and
 should those be treated only as beta versions or as releases in their own
 right?  How is that coupled (or decoupled) from releases of the library?
 etc.
 
 I'd personally rather get those issues addressed out before talking about
 git methods, which to be honest I suspect will be similar no matter what
 the release model.

Agreed.
 So, with that in mind, here are some thoughts/suggestions.
 
 People have talked about Debian's unstable => testing => stable model, but
 instead, why not consider something more akin to Ubuntu's LTS and interim
 releases?

That makes a _lot_ more sense than the unstable => testing => stable model. I like the idea of having an LTS release at some interval (probably 1 year) where that branch has bug fix releases more or less monthly. We then have a dev release cycle where we put out a version of the master branch every 2 or 3 months or so, where it includes whatever new features we want to include but haven't been merged into the LTS branch yet. Then with the next major LTS release, we merge in all of the new stuff from the master branch so that it's essentially up to date with master (possibly modulo some newer features or other major changes if they weren't deemed ready yet). So, we then end up with the LTS/stable branch having a versioning scheme something like 2.061.1, 2.061.2, 2.061.3, 2.061.4 ... 2.062.1, 2.062.2, ... and the master/dev branch has a versioning scheme like 2.062.dev1, 2.062.dev2, 2.062.dev3, 2.062, 2.063.dev1, ... The releases on the master/dev branch which aren't marked with dev are then the new major LTS release, which the LTS/stable branch bases its next years worth of minor releases off of. The exact way to name the versions probably needs some work, but the basic model sounds fairly straightforward and clean to me. We then have longer term releases for those who want stability where they still get bug fixes fairly frequently (possibly more frequently than now, since there won't be new features being added, causing more risk for releasing), and we continue to have dev releases more or less as we have been for those that want new features faster. As for what that means for github and branching, I'd expect us to continue more or less as we have been for master except that newer features and other major changes would start out on feature branches so that they can be properly sorted out before being merged into master, and we'd actually branch when doing betas for releases and do the releases from those branches rather than master. Then the folks mananging the stable branch would merge in bug fixes as we go along, doing their minor releases whenever they feel it's appropriate. As for the standard library and its releases, I wouldn't really expect them to be decoupled from the compiler's releases but rather that they would continue more or less as they have been. I think that it's too often the case that changes in them need to be made at the same time for it to really work to decouple them at this point. - Jonathan M Davis
Dec 13 2012
prev sibling next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 12/12/2012 07:47 PM, deadalnix wrote:
 No, no, please. As already stated, distro release systems intend to solve the
 exact opposite problem of ours.

Yes, I did already read your earlier email objecting to a "Debian-like" model. The thing is, it's not clear to me what actual details you're objecting to, apart from citing a distro as inspiration. I'm not wedded to any detail of what I suggested, although certain parts of it seem common sense, but it's hard to deal with an objection that isn't backed up with details and counter-examples.
 If you want to look at what is done in the field, look at other programming
languages.

Could you give concrete examples to contrast with what I've proposed?
Dec 13 2012
prev sibling next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 12/13/2012 10:07 AM, Jonathan M Davis wrote:
 That makes a _lot_ more sense than the unstable => testing => stable model.

 I like the idea of having an LTS release at some interval (probably 1 year)
 where that branch has bug fix releases more or less monthly. We then have a dev
 release cycle where we put out a version of the master branch every 2 or 3
 months or so, where it includes whatever new features we want to include but
 haven't been merged into the LTS branch yet. Then with the next major LTS
 release, we merge in all of the new stuff from the master branch so that it's
 essentially up to date with master (possibly modulo some newer features or
 other major changes if they weren't deemed ready yet).

I wonder if I maybe created some confusion by conflating "LTS" and "stable" -- I don't think I should really have done that. The problem that I see with what you describe is that it gives no clear mechanism for how breaking changes should be handled -- I don't mean the decision-making process of "Should we make this change?" but the means in which users are supported in making the shift from one LTS to another. So, perhaps it would help if I proposed instead: -- A _stable_ release is what we all understand, a release that has been properly tested and has no outstanding bug reports at the time of first release, and to which only bugfixes are subsequently made -- LTS should refer to a _cycle_ of stable releases within which no breaking changes are made -- Successive LTS cycles should overlap so that the previous cycle is still receiving bugfixes for some time after the new LTS cycle begins. In this way, users have a safe period in which to update their codebases to the new LTS. So -- forgetting D's versioning scheme for a moment -- suppose you label a given LTS cycle with N. Then N.0, N.1, N.2, ... would be the stable releases containing new but non-breaking features, which might be made annually, or every 6 months. N.x.0, N.x.1, N.x.2, ... would be the bugfix-only updates to those releases, made as often as necessary. Once LTS cycle N+1 has begun, then you stop making N.x releases and only apply further bug fixes. Likewise, bugfixes to N.x stop getting released once N.x+1 is released. So, if you assume a stable release annually, and that an LTS cycle lasts 3 years, you might have: 1st year N.0.0 - N.0.1 [bugfixes] - N.0.2 ... 2nd year N.1.0 [non-breaking new features] - N.1.1 - N.1.2 ... [new LTS cycle begins, maybe with breaking changes] 3rd year N.2.0 N+1.0.0 - N.2.1 - N+1.0.1 - N.2.2 - N+1.0.2 ... ... 4th year N+1.1.0 [entirely moved over to new LTS] - N+1.1.1 - N+1.1.2 ... 5th year N+1.2.0 N+2.0.0 [new LTS begins] 6th year N+2.1.0 7th year N+2.2.0 N+3.0.0 ... etc. Of course, the timeline could be altered but the point would be that for every LTS cycle you'd have up to a year in which to switch, and _within_ every LTS cycle you would receive stable updates with bugfixes (regularly) and well-tested new features (annually, 6-monthly?). That's surely more work for developers but is probably worth it in terms of security for users of D. In the event that a new LTS is accompanied by no breaking changes, you might do away with the year's overlap to save the extra work. If you want to translate the above into actual D version numbers, you could do it in the form V.N.x.y where V is the D language version (2 for the foreseeable future:-), N the LTS cycle, etc. That leaves out "dev" releases, but you might want to number them differently anyway -- e.g. dev-YYYYmmdd or whatever.
 As for the standard library and its releases, I wouldn't really expect them to
 be decoupled from the compiler's releases but rather that they would continue
 more or less as they have been. I think that it's too often the case that
 changes in them need to be made at the same time for it to really work to
 decouple them at this point.

My feeling here was that as it stands Phobos is still somewhat more in flux than D itself, not so much in terms of breaking changes as in terms of new features being added. So, I was wondering if it might be worthwhile to make new-Phobos-feature releases on a more regular basis at least in the short term while the library is still being expanded. But I won't press for it too much.
Dec 13 2012
prev sibling next sibling parent "RenatoUtsch" <renatoutsch gmail.com> writes:
On Thursday, 13 December 2012 at 15:44:25 UTC, Joseph Rushton 
Wakeling wrote:
 On 12/13/2012 10:07 AM, Jonathan M Davis wrote:
 That makes a _lot_ more sense than the unstable => testing => 
 stable model.

 I like the idea of having an LTS release at some interval 
 (probably 1 year)
 where that branch has bug fix releases more or less monthly. 
 We then have a dev
 release cycle where we put out a version of the master branch 
 every 2 or 3
 months or so, where it includes whatever new features we want 
 to include but
 haven't been merged into the LTS branch yet. Then with the 
 next major LTS
 release, we merge in all of the new stuff from the master 
 branch so that it's
 essentially up to date with master (possibly modulo some newer 
 features or
 other major changes if they weren't deemed ready yet).

I wonder if I maybe created some confusion by conflating "LTS" and "stable" -- I don't think I should really have done that. The problem that I see with what you describe is that it gives no clear mechanism for how breaking changes should be handled -- I don't mean the decision-making process of "Should we make this change?" but the means in which users are supported in making the shift from one LTS to another. So, perhaps it would help if I proposed instead: -- A _stable_ release is what we all understand, a release that has been properly tested and has no outstanding bug reports at the time of first release, and to which only bugfixes are subsequently made -- LTS should refer to a _cycle_ of stable releases within which no breaking changes are made -- Successive LTS cycles should overlap so that the previous cycle is still receiving bugfixes for some time after the new LTS cycle begins. In this way, users have a safe period in which to update their codebases to the new LTS. So -- forgetting D's versioning scheme for a moment -- suppose you label a given LTS cycle with N. Then N.0, N.1, N.2, ... would be the stable releases containing new but non-breaking features, which might be made annually, or every 6 months. N.x.0, N.x.1, N.x.2, ... would be the bugfix-only updates to those releases, made as often as necessary. Once LTS cycle N+1 has begun, then you stop making N.x releases and only apply further bug fixes. Likewise, bugfixes to N.x stop getting released once N.x+1 is released. So, if you assume a stable release annually, and that an LTS cycle lasts 3 years, you might have: 1st year N.0.0 - N.0.1 [bugfixes] - N.0.2 ... 2nd year N.1.0 [non-breaking new features] - N.1.1 - N.1.2 ... [new LTS cycle begins, maybe with breaking changes] 3rd year N.2.0 N+1.0.0 - N.2.1 - N+1.0.1 - N.2.2 - N+1.0.2 ... ... 4th year N+1.1.0 [entirely moved over to new LTS] - N+1.1.1 - N+1.1.2 ... 5th year N+1.2.0 N+2.0.0 [new LTS begins] 6th year N+2.1.0 7th year N+2.2.0 N+3.0.0 ... etc. Of course, the timeline could be altered but the point would be that for every LTS cycle you'd have up to a year in which to switch, and _within_ every LTS cycle you would receive stable updates with bugfixes (regularly) and well-tested new features (annually, 6-monthly?). That's surely more work for developers but is probably worth it in terms of security for users of D. In the event that a new LTS is accompanied by no breaking changes, you might do away with the year's overlap to save the extra work. If you want to translate the above into actual D version numbers, you could do it in the form V.N.x.y where V is the D language version (2 for the foreseeable future:-), N the LTS cycle, etc. That leaves out "dev" releases, but you might want to number them differently anyway -- e.g. dev-YYYYmmdd or whatever.
 As for the standard library and its releases, I wouldn't 
 really expect them to
 be decoupled from the compiler's releases but rather that they 
 would continue
 more or less as they have been. I think that it's too often 
 the case that
 changes in them need to be made at the same time for it to 
 really work to
 decouple them at this point.

My feeling here was that as it stands Phobos is still somewhat more in flux than D itself, not so much in terms of breaking changes as in terms of new features being added. So, I was wondering if it might be worthwhile to make new-Phobos-feature releases on a more regular basis at least in the short term while the library is still being expanded. But I won't press for it too much.

This approach looks the best to D. As we have the "problem" that some users want to see D more stable and other users want to see it evolve, to not turn into what C++ has become, this approach looks the best solution of all that were proposed here. It will satisfy the users that like the language to be more stable for serious development, while the language can still evolve and the language devs will be able to do the things right. if each LTS is supported for about 5 years, and with 1 year supporting the last LTS after a new one is released, D will be stable enough for serious development, what will attract much people, as the language is simply incredible, the only problem (at least with all people I know) is that it is not really stable. While the LTS versions are supported, breaking changes can be introduced in non-LTS versions that are supported for bugfixes for... say 6 monthes, and non-breaking changes are merged on both the LTS and the non-LTS. This will satisfy all D users... it will only require more work from the D team.
Dec 13 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Thursday, 13 December 2012 at 15:44:25 UTC, Joseph Rushton 
Wakeling wrote:
 My feeling here was that as it stands Phobos is still somewhat 
 more in flux than D itself, not so much in terms of breaking 
 changes as in terms of new features being added.  So, I was 
 wondering if it might be worthwhile to make new-Phobos-feature 
 releases on a more regular basis at least in the short term 
 while the library is still being expanded.  But I won't press 
 for it too much.

I'll be the first to agree that Phobos should be separated from the compiler releases, but not entirely. I wish the std lib could be separated into an "essential component" and a "non-essential component" so that a partial decoupling could be achieved, that way the essential component could be directly locked into successive compiler releases, while leaving out the non-essential as a separate add-on package following it's own course of development. But IMO we're straying off-topic because what we really need to do first, is fire up a wiki page to write down and agree on (as best as we can) what we want to achieve (a list of goals), and only after that can we continue on deciding what process will meet all of the agreed on goals. Anyone proficient in wiki pages want to create a new page for defining the goals? --rt
Dec 13 2012
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Dec 13, 2012 at 07:26:18PM +0100, Rob T wrote:
[...]
 But IMO we're straying off-topic because what we really need to do
 first, is fire up a wiki page to write down and agree on (as best as
 we can) what we want to achieve (a list of goals), and only after
 that can we continue on deciding what process will meet all of the
 agreed on goals.
 
 Anyone proficient in wiki pages want to create a new page for
 defining the goals?

There's been too much talk and little action, so I've gone ahead with a stub page for this discussion: http://wiki.dlang.org/Proposed_new_D_development_process Please fill in the details as you understand them, so that we have something to start with. Also, it will probably be best if we continue this discussion on the talk page, so that everything is in one place. Since each of us may have conflicting ideas about what the final process should be, let's adopt the convention that if something on the page is not how you understand things should be, you should discuss on the talk page before making the change, so that we don't devolve into an edit war. T -- Computerese Irregular Verb Conjugation: I have preferences. You have biases. He/She has prejudices. -- Gene Wirchenko
Dec 13 2012
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Thursday, 13 December 2012 at 18:41:47 UTC, H. S. Teoh wrote:
 On Thu, Dec 13, 2012 at 07:26:18PM +0100, Rob T wrote:
 [...]
 But IMO we're straying off-topic because what we really need 
 to do
 first, is fire up a wiki page to write down and agree on (as 
 best as
 we can) what we want to achieve (a list of goals), and only 
 after
 that can we continue on deciding what process will meet all of 
 the
 agreed on goals.
 
 Anyone proficient in wiki pages want to create a new page for
 defining the goals?

There's been too much talk and little action, so I've gone ahead with a stub page for this discussion: http://wiki.dlang.org/Proposed_new_D_development_process Please fill in the details as you understand them, so that we have something to start with. Also, it will probably be best if we continue this discussion on the talk page, so that everything is in one place. Since each of us may have conflicting ideas about what the final process should be, let's adopt the convention that if something on the page is not how you understand things should be, you should discuss on the talk page before making the change, so that we don't devolve into an edit war.

I just started this http://wiki.dlang.org/Release_Process Feel free to merge or whatever.
Dec 13 2012
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Thursday, 13 December 2012 at 20:04:50 UTC, Dmitry Olshansky 
wrote:
 I think it's good.

 But personally I'd expect:

 * master to be what you define as dev, because e.g. GitHub puts 
 master as default target branch when making pull requests. 
 Yeah, I know it's their quirk that it's easy to miss. Still 
 leaving less burden to check the branch label for both 
 reviewers and pull authors is a net gain.

 * And what you describe as master (it's a snapshot or a 
 pre-release to me) to be named e.g. staging.

 And we can as well drop the dead branch 'dev' then.

That sound also like a good plan to me.
Dec 13 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Thursday, 13 December 2012 at 18:41:47 UTC, H. S. Teoh wrote:
 Since each of us may have conflicting ideas about what the 
 final process
 should be, let's adopt the convention that if something on the 
 page is
 not how you understand things should be, you should discuss on 
 the talk
 page before making the change, so that we don't devolve into an 
 edit
 war.


 T

I was wondering how we'll resolve conflicts, because not everyone will agree on everything thats for certain. We don't really have a voting system in place, and there's no "benevolent dictator" heading the project unless Andrei has appointed himself for resolving conflicts, but he's not really participating much at this point. We're kinda leaderless and left with no mechanism for making the inevitable hard decisions that we'll have to make if this effort is to move ahead. Any ideas? --rt
Dec 13 2012
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Dec 13, 2012 at 10:36:17PM +0100, Rob T wrote:
 On Thursday, 13 December 2012 at 18:41:47 UTC, H. S. Teoh wrote:
Since each of us may have conflicting ideas about what the final
process should be, let's adopt the convention that if something on
the page is not how you understand things should be, you should
discuss on the talk page before making the change, so that we don't
devolve into an edit war.


 I was wondering how we'll resolve conflicts, because not everyone will
 agree on everything thats for certain. We don't really have a voting
 system in place, and there's no "benevolent dictator" heading the
 project unless Andrei has appointed himself for resolving conflicts,
 but he's not really participating much at this point.  We're kinda
 leaderless and left with no mechanism for making the inevitable hard
 decisions that we'll have to make if this effort is to move ahead.
 
 Any ideas?

I don't have a good answer to this, but maybe we could use Wikipedia's method of garnering consensus before a major change is made? I don't know how well it will work, given our relatively small numbers, though. It would be nice if Andrei could step in and head up this process. But then again, he's a busy man and only does D on his limited free time, so it may be just a matter of waiting for a bit for him to pipe up. T -- Mediocrity has been pushed to extremes.
Dec 13 2012
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Thursday, 13 December 2012 at 20:48:30 UTC, deadalnix wrote:
 On Thursday, 13 December 2012 at 20:04:50 UTC, Dmitry Olshansky 
 wrote:
 I think it's good.

 But personally I'd expect:

 * master to be what you define as dev, because e.g. GitHub 
 puts master as default target branch when making pull 
 requests. Yeah, I know it's their quirk that it's easy to 
 miss. Still leaving less burden to check the branch label for 
 both reviewers and pull authors is a net gain.

 * And what you describe as master (it's a snapshot or a 
 pre-release to me) to be named e.g. staging.

 And we can as well drop the dead branch 'dev' then.

That sound also like a good plan to me.

Updated to follow the idea, plus added bunch of process description. Feel free to comment in order to refine this. http://wiki.dlang.org/Release_Process
Dec 13 2012
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Dec 14, 2012 at 12:34:13AM +0100, deadalnix wrote:
 On Thursday, 13 December 2012 at 20:48:30 UTC, deadalnix wrote:

 Updated to follow the idea, plus added bunch of process description.
 Feel free to comment in order to refine this.
 
 http://wiki.dlang.org/Release_Process

I've rearranged the page a bit and added more higher-level discussion. Please feel free to edit/correct any mistakes I may have had. T -- MSDOS = MicroSoft's Denial Of Service
Dec 13 2012
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-12-11 00:41, Andrei Alexandrescu wrote:
 Hello all,


 Walter and I have had a long discussion following his trip to Australia.
 Following the current sprint for Win64 (which we all, I think, agree was
 long overdue and had to be done), the main area we need to work on (as
 I'm sure many would agree) is improving our process, in particular the
 approach to releasing the compiler and standard library.

 Walter has delegated me to lead this effort, and agreed to obey with
 whatever process comes forward as long as it's reasonably clear and simple.

 In turn, I'll be collecting thoughts and opinions in this thread,
 distilled from previous discussions. We should develop a few simple git
 scripts (such as git-start-new-feature or git-start-bugfix etc)
 supported by an equally simple piece of documentation describing how to
 start a new release, fix a bug, experiment with a new feature, and such.

 (One piece that has been brought forward is
 http://nvie.com/posts/a-successful-git-branching-model/ - something to
 keep in mind.)

Don't know if this has already been said but there already exists a git command for this workflow: https://github.com/nvie/gitflow It's available in MacPorts so I should assume it's available in most Linux package mangers as well. -- /Jacob Carlborg
Dec 14 2012
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
--047d7bea3518382edc04d0cef10d
Content-Type: text/plain; charset=UTF-8

One thing that I think would be really awesome when this is all rolling, is
attach a system that does nightly builds of the dev line every evening.
I've often had to pester Walter to produce a new build for us when a
critical bug/feature was fixed.

--047d7bea3518382edc04d0cef10d
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

One thing that I think would be really awesome when this is all rolling, is=
 attach a system that does nightly builds of the dev line every evening.<di=
v>I&#39;ve often had to pester Walter to produce a new build for us when a =
critical bug/feature was fixed.</div>

--047d7bea3518382edc04d0cef10d--
Dec 14 2012
parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
12/14/2012 4:12 PM, Manu пишет:
 One thing that I think would be really awesome when this is all rolling,
 is attach a system that does nightly builds of the dev line every evening.
 I've often had to pester Walter to produce a new build for us when a
 critical bug/feature was fixed.

Then this could be of interest: http://forum.dlang.org/post/mailman.2633.1355345752.5162.digitalmars-d puremagic.com From that I take it's basically coming soon. -- Dmitry Olshansky
Dec 14 2012
prev sibling next sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 12/14/12, Manu <turkeyman gmail.com> wrote:
 I've often had to pester Walter to produce a new build for us when a
 critical bug/feature was fixed.

Why waste Walter's time when building is as simple as calling make?
Dec 14 2012
parent reply Jacob Carlborg <doob me.com> writes:
On 2012-12-14 13:15, Andrej Mitrovic wrote:

 Why waste Walter's time when building is as simple as calling make?

Agree. And with DVM it's just one command, after all repositories are updated, instead of calling "make" three times. https://github.com/jacob-carlborg/dvm -- /Jacob Carlborg
Dec 14 2012
parent Jacob Carlborg <doob me.com> writes:
On 2012-12-14 14:10, Dan wrote:

 Just saw after posting. The description looks great. So, this is what D
 developers are using? Good stuff. Thanks.

At least some are using it. Not the whole community. The latest release has 20 downloads across Linux, Windows and Mac OS X. https://github.com/jacob-carlborg/dvm/downloads -- /Jacob Carlborg
Dec 14 2012
prev sibling next sibling parent "Dan" <dbdavidson yahoo.com> writes:
On Friday, 14 December 2012 at 12:15:28 UTC, Andrej Mitrovic 
wrote:
 On 12/14/12, Manu <turkeyman gmail.com> wrote:
 I've often had to pester Walter to produce a new build for us 
 when a
 critical bug/feature was fixed.

Why waste Walter's time when building is as simple as calling make?

His point was he clearly wants *not* to waste Walter's time. In the discussions of build system one thread was about using Ruby. A comment was why not just use D to script it, you can do everything in D that you can from ruby (or shell)? I wrote a small script to see how cumbersome it would be to download and build D with that approach: one piece grabs latest source (phobos, runtime, dmd, tools), one builds it. http://pastebin.com/CC7DvZuJ It ain't pretty and tailored to linux. I don't know how D developers set up their environment - a technote may exist somewhere? The focus in this thread is on release process which is critical, but an addition to wiki page on "how to get started as a developer on D project" might be useful. I don't think it is that complex to build D, but for me it was not as simple as just calling make because you have to get the prereqs, know things like phobos inludes the runtime, so build runtime before phobos, and if you want release and debug builds you should have separate folders since they build in place. I also made a small change local to posix.mak, to be able to build debug. ------ -GFLAGS = $(WARNINGS) -D__pascal= -fno-exceptions -O2 +ifneq (x,x$(DEBUG)) + GFLAGS = $(WARNINGS) -D__pascal= -fno-exceptions -g -DDEBUG=1 -DUNITTEST $(COV) +else + GFLAGS = $(WARNINGS) -D__pascal= -fno-exceptions -O2 +endif ------ This may all be bad advice, since I only did it to see if I could step into associative array code to track down a bug and I don't know the good D dev setup for linux. Any ideas that push toward continuous build/integration/test are the right direction.
Dec 14 2012
prev sibling next sibling parent "Dan" <dbdavidson yahoo.com> writes:
On Friday, 14 December 2012 at 12:56:42 UTC, Jacob Carlborg wrote:
 On 2012-12-14 13:15, Andrej Mitrovic wrote:

 Why waste Walter's time when building is as simple as calling 
 make?

Agree. And with DVM it's just one command, after all repositories are updated, instead of calling "make" three times. https://github.com/jacob-carlborg/dvm

Just saw after posting. The description looks great. So, this is what D developers are using? Good stuff. Thanks.
Dec 14 2012
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--20cf306f73da4c70ec04d0cff0ac
Content-Type: text/plain; charset=UTF-8

On 14 December 2012 13:07, Dan <dbdavidson yahoo.com> wrote:

 On Friday, 14 December 2012 at 12:15:28 UTC, Andrej Mitrovic wrote:

 On 12/14/12, Manu <turkeyman gmail.com> wrote:

 I've often had to pester Walter to produce a new build for us when a
 critical bug/feature was fixed.


His point was he clearly wants *not* to waste Walter's time. In the discussions of build system one thread was about using Ruby. A comment was why not just use D to script it, you can do everything in D that you can from ruby (or shell)? I wrote a small script to see how cumbersome it would be to download and build D with that approach: one piece grabs latest source (phobos, runtime, dmd, tools), one builds it. http://pastebin.com/CC7DvZuJ It ain't pretty and tailored to linux. I don't know how D developers set up their environment - a technote may exist somewhere? The focus in this thread is on release process which is critical, but an addition to wiki page on "how to get started as a developer on D project" might be useful. I don't think it is that complex to build D, but for me it was not as simple as just calling make because you have to get the prereqs, know things like phobos inludes the runtime, so build runtime before phobos, and if you want release and debug builds you should have separate folders since they build in place. I also made a small change local to posix.mak, to be able to build debug. ------ -GFLAGS = $(WARNINGS) -D__pascal= -fno-exceptions -O2 +ifneq (x,x$(DEBUG)) + GFLAGS = $(WARNINGS) -D__pascal= -fno-exceptions -g -DDEBUG=1 -DUNITTEST $(COV) +else + GFLAGS = $(WARNINGS) -D__pascal= -fno-exceptions -O2 +endif ------ This may all be bad advice, since I only did it to see if I could step into associative array code to track down a bug and I don't know the good D dev setup for linux. Any ideas that push toward continuous build/integration/test are the right direction.

Thank you. --20cf306f73da4c70ec04d0cff0ac Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 14 December 2012 13:07, Dan <span dir=3D"ltr">&lt;<a href=3D"mailto:dbda= vidson yahoo.com" target=3D"_blank">dbdavidson yahoo.com</a>&gt;</span> wro= te:<br><div class=3D"gmail_extra"><div class=3D"gmail_quote"><blockquote cl= ass=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;p= adding-left:1ex"> <div class=3D"im">On Friday, 14 December 2012 at 12:15:28 UTC, Andrej Mitro= vic wrote:<br> </div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-l= eft:1px #ccc solid;padding-left:1ex"><div class=3D"im"> On 12/14/12, Manu &lt;<a href=3D"mailto:turkeyman gmail.com" target=3D"_bla= nk">turkeyman gmail.com</a>&gt; wrote:<br> </div><div class=3D"im"><blockquote class=3D"gmail_quote" style=3D"margin:0= 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> I&#39;ve often had to pester Walter to produce a new build for us when a<br=

<br> </blockquote> <br></div><div class=3D"im"> Why waste Walter&#39;s time when building is as simple as calling make?<br> </div></blockquote> <br> His point was he clearly wants *not* to waste Walter&#39;s time.<br> <br> In the discussions of build system one thread was about using Ruby. A comme= nt was why not just use D to script it, you can do everything in D that you= can from ruby (or shell)? I wrote a small script to see how cumbersome it = would be to download and build D with that approach: one piece grabs latest= source (phobos, runtime, dmd, tools), one builds it. <a href=3D"http://pas= tebin.com/CC7DvZuJ" target=3D"_blank">http://pastebin.com/CC7DvZuJ</a><br> <br> It ain&#39;t pretty and tailored to linux. I don&#39;t know how D developer= s set up their environment - a technote may exist somewhere? The focus in t= his thread is on release process which is critical, but an addition to wiki= page on &quot;how to get started as a developer on D project&quot; might b= e useful. I don&#39;t think it is that complex to build D, but for me it wa= s not as simple as just calling make because you have to get the prereqs, k= now things like phobos inludes the runtime, so build runtime before phobos,= and if you want release and debug builds you should have separate folders = since they build in place.<br> I also made a small change local to posix.mak, to be able to build debug.<b= r> <br> ------<br> -GFLAGS =3D $(WARNINGS) -D__pascal=3D -fno-exceptions -O2<br> +ifneq (x,x$(DEBUG))<br> + =C2=A0 =C2=A0 =C2=A0 GFLAGS =3D $(WARNINGS) -D__pascal=3D -fno-exceptions= -g -DDEBUG=3D1 -DUNITTEST $(COV)<br> +else<br> + =C2=A0 =C2=A0 =C2=A0 GFLAGS =3D $(WARNINGS) -D__pascal=3D -fno-exceptions= -O2<br> +endif<br> ------<br> <br> This may all be bad advice, since I only did it to see if I could step into= associative array code to track down a bug and I don&#39;t know the good D= dev setup for linux.<br> <br> Any ideas that push toward continuous build/integration/test are the right = direction.<br> </blockquote></div><br></div><div class=3D"gmail_extra">Thank you.</div> --20cf306f73da4c70ec04d0cff0ac--
Dec 14 2012
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Dec 14, 2012 at 02:07:40PM +0100, Dan wrote:
[...]
 I don't know how D developers set up their environment - a technote
 may exist somewhere? The focus in this thread is on release process
 which is critical, but an addition to wiki page on "how to get started
 as a developer on D project" might be useful. I don't think it is that
 complex to build D, but for me it was not as simple as just calling
 make because you have to get the prereqs, know things like phobos
 inludes the runtime, so build runtime before phobos, and if you want
 release and debug builds you should have separate folders since they
 build in place.

http://wiki.dlang.org/Building_DMD T -- War doesn't prove who's right, just who's left. -- BSD Games' Fortune
Dec 14 2012
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Dec 14, 2012 at 06:29:50AM -0800, H. S. Teoh wrote:
 On Fri, Dec 14, 2012 at 02:07:40PM +0100, Dan wrote:
 [...]
 I don't know how D developers set up their environment - a technote
 may exist somewhere? The focus in this thread is on release process
 which is critical, but an addition to wiki page on "how to get started
 as a developer on D project" might be useful. I don't think it is that
 complex to build D, but for me it was not as simple as just calling
 make because you have to get the prereqs, know things like phobos
 inludes the runtime, so build runtime before phobos, and if you want
 release and debug builds you should have separate folders since they
 build in place.

http://wiki.dlang.org/Building_DMD

Also, a more general guide to contributing to D is here: http://wiki.dlang.org/Get_involved (As this is a wiki, if you find any errors/omissions please edit and improve the page.) T -- Study gravitation, it's a field with a lot of potential.
Dec 14 2012
prev sibling next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Dec 10, 2012 at 06:41:25PM -0500, Andrei Alexandrescu wrote:
 Hello all,
 
 Walter and I have had a long discussion following his trip to
 Australia. Following the current sprint for Win64 (which we all, I
 think, agree was long overdue and had to be done), the main area we
 need to work on (as I'm sure many would agree) is improving our
 process, in particular the approach to releasing the compiler and
 standard library.
 
 Walter has delegated me to lead this effort, and agreed to obey with
 whatever process comes forward as long as it's reasonably clear and
 simple.
 
 In turn, I'll be collecting thoughts and opinions in this thread,
 distilled from previous discussions. We should develop a few simple
 git scripts (such as git-start-new-feature or git-start-bugfix etc)
 supported by an equally simple piece of documentation describing how
 to start a new release, fix a bug, experiment with a new feature,
 and such.
 
 (One piece that has been brought forward is
 http://nvie.com/posts/a-successful-git-branching-model/ - something
 to keep in mind.)
 
 Please chime in with ideas and thoughts.

Hi Andrei, A number of us have put up a draft of a proposed release process on the wiki, based on some of the things discussed in this thread. http://wiki.dlang.org/Release_Process It would be *very* helpful if you could chime in with some comments from your POV as one of the core developers (in fact, it would be very good if we can have some input from _any_ of the core devs): http://wiki.dlang.org/Talk:Release_Process Right now we think we have some good ideas going, but we don't want to go too far afield without any input from the core devs, since they are the ones (mainly) who will be implementing this process. T -- Three out of two people have difficulties with fractions. -- Dirk Eddelbuettel
Dec 14 2012
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/14/12 10:02 AM, H. S. Teoh wrote:
 A number of us have put up a draft of a proposed release process on the
 wiki, based on some of the things discussed in this thread.

 	http://wiki.dlang.org/Release_Process

Seen it, will give a thorough read today. Thanks! Andrei
Dec 14 2012
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/19/12 1:45 PM, H. S. Teoh wrote:
 On Fri, Dec 14, 2012 at 10:16:45AM -0500, Andrei Alexandrescu wrote:
 On 12/14/12 10:02 AM, H. S. Teoh wrote:
 A number of us have put up a draft of a proposed release process on
 the wiki, based on some of the things discussed in this thread.

 	http://wiki.dlang.org/Release_Process

Seen it, will give a thorough read today. Thanks!

Just to follow up, any comments on what's there currently? I fear that some of us on this thread may be over-engineering the whole thing without input from the core devs who will actually be implementing this process.

I was hoping for more consensus to build in this thread. Right now it seems there's still quite a bit of controversy about what the best way to go is. Andrei
Dec 19 2012
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/19/12 4:23 PM, foobar wrote:
 On Wednesday, 19 December 2012 at 20:51:57 UTC, deadalnix wrote:
 On Wednesday, 19 December 2012 at 19:56:47 UTC, Rob T wrote:

 Do we all agree that we need a "stable" branch?

No. Stable isn't a boolean criteria. You'll find different degree of stability going from not so stable (dev version) to very stable (dead project). The wiki already mention a process with a branch per version of the software.

Let's generalize this point for the sake of reaching consensus - we need _at least one_ "stable" branch which is separate from "staging". We are still conflicted as to what should be the maximum amount. For the record, I'm with the camp advocating at most a fixed amount countable on one hand. That's an O(1) with a very small constant as opposed to the O(n) suggestion by Andrei. I hope Andrei appreciates the order of efficiency here.

I agree with one "stable" branch. Andrei
Dec 19 2012
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/19/12 4:40 PM, deadalnix wrote:
 On Wednesday, 19 December 2012 at 21:30:44 UTC, Andrei Alexandrescu wrote:
 On 12/19/12 4:23 PM, foobar wrote:
 On Wednesday, 19 December 2012 at 20:51:57 UTC, deadalnix wrote:
 On Wednesday, 19 December 2012 at 19:56:47 UTC, Rob T wrote:

 Do we all agree that we need a "stable" branch?

No. Stable isn't a boolean criteria. You'll find different degree of stability going from not so stable (dev version) to very stable (dead project). The wiki already mention a process with a branch per version of the software.

Let's generalize this point for the sake of reaching consensus - we need _at least one_ "stable" branch which is separate from "staging". We are still conflicted as to what should be the maximum amount. For the record, I'm with the camp advocating at most a fixed amount countable on one hand. That's an O(1) with a very small constant as opposed to the O(n) suggestion by Andrei. I hope Andrei appreciates the order of efficiency here.

I agree with one "stable" branch.

This does conflict with the requirement you gave before about being able to support anything, as previous stable version cannot be revised. Or does stable here mean supported ? (which means we still have branch per version, but only one version is supported)

Walter needs to chime in about that. One possibility is to continue using tags for marking releases, and then branch for the few important releases that we want to patch. Andrei
Dec 19 2012
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/19/12 5:07 PM, deadalnix wrote:
 On Wednesday, 19 December 2012 at 21:48:22 UTC, Andrei Alexandrescu wrote:
 Walter needs to chime in about that. One possibility is to continue
 using tags for marking releases, and then branch for the few important
 releases that we want to patch.

Note that what is described on the wiki distinguish new version (that actually bring new stuff) and revision (that contains only bugfixes). Does having branches that are not used anymore is a problem ? They'll not make the repository much bigger because the data is in history anyway.

I kinda like the idea that we have tags for releases not planned for long term support and distinguished branches for those that we do. Clarifies intent a lot. Andrei
Dec 19 2012
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/19/12 5:43 PM, deadalnix wrote:
 On Wednesday, 19 December 2012 at 22:30:29 UTC, Andrei Alexandrescu wrote:
 On 12/19/12 5:07 PM, deadalnix wrote:
 On Wednesday, 19 December 2012 at 21:48:22 UTC, Andrei Alexandrescu
 wrote:
 Walter needs to chime in about that. One possibility is to continue
 using tags for marking releases, and then branch for the few important
 releases that we want to patch.

Note that what is described on the wiki distinguish new version (that actually bring new stuff) and revision (that contains only bugfixes). Does having branches that are not used anymore is a problem ? They'll not make the repository much bigger because the data is in history anyway.

I kinda like the idea that we have tags for releases not planned for long term support and distinguished branches for those that we do. Clarifies intent a lot.

Should something be released if nobody will support it ?

My understanding is that's what many projects do. Supporting each minor release would make for a ton of work.
 If it is in order to provide overview of what is coming in the next
 version, I guess the best option is to provide a snapshot of the staging
 branch on regular basis (and call it beta or whatever).

No, I'm not talking about previews.
 Reading this answer, I start to think that we have something pretty
 close in mind but fail to explain it to each other.

That's quite possible. Again, I'm not an expert in the matter and turned to the community for that reason. What we're looking for is a process that makes things better for everyone involved without having too much overhead. Andrei
Dec 19 2012
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Dec 19, 2012 at 04:48:22PM -0500, Andrei Alexandrescu wrote:
 On 12/19/12 4:40 PM, deadalnix wrote:
On Wednesday, 19 December 2012 at 21:30:44 UTC, Andrei Alexandrescu wrote:
On 12/19/12 4:23 PM, foobar wrote:



Let's generalize this point for the sake of reaching consensus - we
need _at least one_ "stable" branch which is separate from
"staging". We are still conflicted as to what should be the maximum
amount. For the record, I'm with the camp advocating at most a
fixed amount countable on one hand. That's an O(1) with a very
small constant as opposed to the O(n) suggestion by Andrei. I hope
Andrei appreciates the order of efficiency here.

I agree with one "stable" branch.

This does conflict with the requirement you gave before about being able to support anything, as previous stable version cannot be revised. Or does stable here mean supported ? (which means we still have branch per version, but only one version is supported)

Walter needs to chime in about that. One possibility is to continue using tags for marking releases, and then branch for the few important releases that we want to patch.

This is a good idea, to avoid cluttering the git repo with branches. (But then again, branches in git are cheap so I don't think this is really that big of a deal.) T -- Just because you survived after you did it, doesn't mean it wasn't stupid!
Dec 19 2012
prev sibling parent "foobar" <foo bar.com> writes:
On Wednesday, 19 December 2012 at 21:53:04 UTC, H. S. Teoh wrote:
 On Wed, Dec 19, 2012 at 04:48:22PM -0500, Andrei Alexandrescu 
 wrote:
 On 12/19/12 4:40 PM, deadalnix wrote:
On Wednesday, 19 December 2012 at 21:30:44 UTC, Andrei 
Alexandrescu wrote:
On 12/19/12 4:23 PM, foobar wrote:



Let's generalize this point for the sake of reaching 
consensus - we
need _at least one_ "stable" branch which is separate from
"staging". We are still conflicted as to what should be the 
maximum
amount. For the record, I'm with the camp advocating at 
most a
fixed amount countable on one hand. That's an O(1) with a 
very
small constant as opposed to the O(n) suggestion by Andrei. 
I hope
Andrei appreciates the order of efficiency here.

I agree with one "stable" branch.

This does conflict with the requirement you gave before about being able to support anything, as previous stable version cannot be revised. Or does stable here mean supported ? (which means we still have branch per version, but only one version is supported)

Walter needs to chime in about that. One possibility is to continue using tags for marking releases, and then branch for the few important releases that we want to patch.

This is a good idea, to avoid cluttering the git repo with branches. (But then again, branches in git are cheap so I don't think this is really that big of a deal.) T

This has nothing to do with repository size, git optimizes that anyway. This is a very good decision which I endorse (well, I suggested it in the first place..). Having a single stable branch helps integration and minimizes human error in the process.
Dec 19 2012
prev sibling next sibling parent "Jesse Phillips" <Jesse.K.Phillips+D gmail.com> writes:
 From the discussion there appears to be more that believe that 
master should remain the development branch.

 From this, 'staging' is seen as the branch to prepare a release 
from (bug fix only).

We are currently in the 2.61 freeze, let us move this freeze to 
the staging branch now!

Any disagreement with this? Does anyone see this changing as we 
continue to work out the flow? All I can see is bikeshedding the 
name.
Dec 14 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Friday, 14 December 2012 at 15:36:05 UTC, Jesse Phillips wrote:
 From the discussion there appears to be more that believe that 
 master should remain the development branch.

 From this, 'staging' is seen as the branch to prepare a release 
 from (bug fix only).

 We are currently in the 2.61 freeze, let us move this freeze to 
 the staging branch now!
 Any disagreement with this? Does anyone see this changing as we 
 continue to work out the flow? All I can see is bikeshedding 
 the name.

Yes, we could use 2.61 as the first trial to get things moving along, even if it's not 100% ready and agreed on, it seems obvious that we'll want to use a "staging branch" to work out the bugs while in freeze - I don't think anyone is seriously objecting to that, right? However here's a cautionary note: I read that Walter has specific motives right now concerning a timely release for a strategic D user that can help move D into a more prominent position, so if we're to support a release as smoothly as possible, I wonder if now is or is not the time to try this out on 2.61? My gut says, "of course go for it", because I can't see how anything could get worse instead of better by doing so, however just mentioning that this release is an important one to carefully consider. BTW, the wiki page is looking very good guys. Well done!!! --rt
Dec 14 2012
prev sibling next sibling parent "Jesse Phillips" <Jessekphillips+D gmail.com> writes:
On Friday, 14 December 2012 at 17:17:16 UTC, Rob T wrote:
 However here's a cautionary note: I read that Walter has 
 specific motives right now concerning a timely release for a 
 strategic D user that can help move D into a more prominent 
 position, so if we're to support a release as smoothly as 
 possible, I wonder if now is or is not the time to try this out 
 on 2.61?

I thought on this and believe it is a good time to make 2.61 stable. Especially with Walter wanting to keep the feature (syntax) in just for them. They can trust it will remain a warning for x years. But let us do this for a shorter than planned stable. This will allow us to try out the new process, and be no worse off than before. But let us first get this branch. Ignoring how long this release will last we should all be agreeable on continuing development in master and fixing release bugs in 'staging'
Dec 14 2012
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Dec 14, 2012 at 06:17:15PM +0100, Rob T wrote:
[...]
 BTW, the wiki page is looking very good guys. Well done!!!

Thanks, but we do still have issues to work out, details to fill in, etc.. We'd love to have your input too! T -- We've all heard that a million monkeys banging on a million typewriters will eventually reproduce the entire works of Shakespeare. Now, thanks to the Internet, we know this is not true. -- Robert Wilensk
Dec 14 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Friday, 14 December 2012 at 19:01:23 UTC, H. S. Teoh wrote:
 On Fri, Dec 14, 2012 at 06:17:15PM +0100, Rob T wrote:
 [...]
 BTW, the wiki page is looking very good guys. Well done!!!

Thanks, but we do still have issues to work out, details to fill in, etc.. We'd love to have your input too! T

Yes very true, long ways to go yet. I will try and continue contributing! To me, all things considered, this is THE most important issue limiting the growth and adoption of D because absolutely everything else depends on it running smoothly. --rt
Dec 14 2012
prev sibling next sibling parent "RenatoUtsch" <renatoutsch gmail.com> writes:
On Saturday, 15 December 2012 at 10:29:55 UTC, Dmitry Olshansky
wrote:
 12/14/2012 3:34 AM, deadalnix пишет:
 On Thursday, 13 December 2012 at 20:48:30 UTC, deadalnix wrote:
 On Thursday, 13 December 2012 at 20:04:50 UTC, Dmitry 
 Olshansky wrote:
 I think it's good.

 But personally I'd expect:

 * master to be what you define as dev, because e.g. GitHub 
 puts
 master as default target branch when making pull requests. 
 Yeah, I
 know it's their quirk that it's easy to miss. Still leaving 
 less
 burden to check the branch label for both reviewers and pull 
 authors
 is a net gain.

 * And what you describe as master (it's a snapshot or a 
 pre-release
 to me) to be named e.g. staging.

 And we can as well drop the dead branch 'dev' then.

That sound also like a good plan to me.

Updated to follow the idea, plus added bunch of process description. Feel free to comment in order to refine this. http://wiki.dlang.org/Release_Process

I wasn't comfortable doing speculative edits to the wiki directly so here a few more comments: I think one of major goals is to be able to continue ongoing development while at the _same time_ preparing a release. To me number one problem is condensed in the statement "we are going to release do not merge anything but regressions" the process should sidestep this "lock-based" work-flow. Could be useful to add something along these line to goals section. (i.e. the speed and smoothness is a goal) Second point is about merging master into staging - why not just rewrite it with master branch altogether after each release? master is the branch with correct history (all new stuff is rebased on it) thus new staging will have that too.

Here what I proposed on the discussion page, what do you think? ------------------ I have come with a good idea for the release schedule. Say that we are 3 or 4 monthes before the next LTS release. We need to branch the staging branch in a 2.N (put the contents of the staging branch in the 2.N branch) branch, where N is the new LTS version. Then, no other features can be included in this 2.N branch, only bugfixes are allowed. This period will have one RC every month (?) with the latest fixes in the 2.N branch. After the 3 or 4 monthes period we'll tag the 2.N.0 release. After every 4~6 monthes, we'll release a new 2.N.x version with the latest bugfixes in the 2.N branch, but with no additional features (including non-breaking features here will just be more work to the devs, I don't think it is a good idea). While these bugfix releases are being made, every feature that stabilizes enough in the master branch is merged into the staging branch, where the feature shouldn't have much breaking changes (although changes are still allowed, the branch is not frozen). Every 3 monthes, dev releases are made with these features that made their way into the staging branch. This is done while the fixes in the 2.N branch are made. Then, 4 monthes prior to the next LTS release, a new 2.N+1 branch will be created with the staging branch contents. The cycle will repeat, these 4 monthes will have no additional features on the 2.N+1 branch, and neither on the next 3 years. This organization, in dev and LTS releases, will allow releases for the ones that want a stable environment to develop (the LTS releases) and the ones that want the latest great features from D will have a somewhat stable environment (the dev releases, somewhat like the ones we have now, maybe a little more stable) to use. On top of that, the staging branch will never be frozen, so development will never stop, as someone was saying on the forums that was a bad idea. And, when the new LTS release is made (2 years after the older LTS), the older LTS will be mantained for more one year, what means that each LTS will be mantained for 3 years. What do you think? RenatoUtsch (talk) 14:14, 15 December 2012 (CET) ------------------ http://wiki.dlang.org/Talk:Release_Process#Expanding_LTS
Dec 15 2012
prev sibling next sibling parent Brad Roberts <braddr puremagic.com> writes:
On 12/15/2012 2:29 AM, Dmitry Olshansky wrote:

 I think one of major goals is to be able to continue ongoing development while
at the _same time_ preparing a release.
 To me number one problem is condensed in the statement "we are going to
release do not merge anything but regressions"
 the process should sidestep this "lock-based" work-flow. Could be useful to
add something along these line to goals
 section. (i.e. the speed and smoothness is a goal)

I've been staying out of this thread for the most part, but I feel the need to comment on this part specifically. It's quite common for most major projects to have a clear "we're wrapping up a release" phase where work _is_ restricted to bug fixing and stabilizing. They don't stop people from working off in their development branches (no one could effectively impose such restrictions even if they wanted to), but they _do_ tighten down on what's allowed to be merged. This is a forcing function that's just required. There's a lot of issues that otherwise won't get sufficient attention. If all it took was altruism then regressions would be fixed immediately, bugs would always be fixed in highest priority to lowest priority (assuming that could even be effectively designed), etc. Without the 'ok, guys, focus in this smaller more critical subset of bugs' step, release branches would be created and never polished (or focused down to the release manager to do all the work if he's that skilled and/or generous of his time). There's a phrase I'm trying to remember, but it's something to the effect that 'hope isn't a recipe for success.' Hoping that people fix regressions on release critical bugs isn't sufficient. Incentive and steering is required. The desire to ungate master branch merges is one approach that's been shown to be successful.
Dec 15 2012
prev sibling next sibling parent "Jesse Phillips" <Jessekphillips+d gmail.com> writes:
On Saturday, 15 December 2012 at 19:03:49 UTC, Brad Roberts wrote:

 This is a forcing function that's just required.

There is a focusing that needs to happen, but as you say, you can't really dictate where someone puts their time (for open source). So it is best to let the person who decided to review a pull to also spend the time to merge it. But strongly encourage everyone on the team to get the release out. Remember we are preparing for growth and will want to be ready to handle all types of contributors.
Dec 15 2012
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Saturday, 15 December 2012 at 19:03:49 UTC, Brad Roberts wrote:
 On 12/15/2012 2:29 AM, Dmitry Olshansky wrote:

 I think one of major goals is to be able to continue ongoing 
 development while at the _same time_ preparing a release.
 To me number one problem is condensed in the statement "we are 
 going to release do not merge anything but regressions"
 the process should sidestep this "lock-based" work-flow. Could 
 be useful to add something along these line to goals
 section. (i.e. the speed and smoothness is a goal)

I've been staying out of this thread for the most part, but I feel the need to comment on this part specifically. It's quite common for most major projects to have a clear "we're wrapping up a release" phase where work _is_ restricted to bug fixing and stabilizing. They don't stop people from working off in their development branches (no one could effectively impose such restrictions even if they wanted to), but they _do_ tighten down on what's allowed to be merged. This is a forcing function that's just required. There's a lot of issues that otherwise won't get sufficient attention. If all it took was altruism then regressions would be fixed immediately, bugs would always be fixed in highest priority to lowest priority (assuming that could even be effectively designed), etc. Without the 'ok, guys, focus in this smaller more critical subset of bugs' step, release branches would be created and never polished (or focused down to the release manager to do all the work if he's that skilled and/or generous of his time). There's a phrase I'm trying to remember, but it's something to the effect that 'hope isn't a recipe for success.' Hoping that people fix regressions on release critical bugs isn't sufficient. Incentive and steering is required. The desire to ungate master branch merges is one approach that's been shown to be successful.

Very good point. The final sprint is a good practice to adopt.
Dec 15 2012
prev sibling next sibling parent "Jesse Phillips" <Jessekphillips+D gmail.com> writes:
On Saturday, 15 December 2012 at 10:29:55 UTC, Dmitry Olshansky 
wrote:
 Second point is about merging master into staging - why not 
 just rewrite it with master branch altogether after each 
 release?
 master is the branch with correct history (all new stuff is 
 rebased on it) thus new staging will have that too.

Why you don't rewrite is because it is a public branch. Unlike feature branches which will basically be thrown out everyone on the development team will need to have staging updated. If we rewrite history then instead of $ git pull staging At random times it will be (I don't know the commands and won't even look it up) It just won't be pretty. I've made modifications to the graphic hoping to illustrate some thoughts. http://i.imgur.com/rJVSg.png This does not depict what is currently described (in terms of branching). But is what I've written under http://wiki.dlang.org/Release_Process#Release_Schedule I see patches going into the LTS-1 (if applicable), the LTS-1 is then merged into the latest LTS, which is merged into any active staging, that is then merged into master. The monthly release don't get bug fixes (just wait for the next month). I've removed some version numbering since I don't know if we should have a distinct numbering for LTS and Monthly. I've already give some thoughts on this: http://forum.dlang.org/post/ydmgqmbqngwderfkljde forum.dlang.org
Dec 15 2012
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Saturday, 15 December 2012 at 20:32:42 UTC, Jesse Phillips 
wrote:
 On Saturday, 15 December 2012 at 10:29:55 UTC, Dmitry Olshansky 
 wrote:
 Second point is about merging master into staging - why not 
 just rewrite it with master branch altogether after each 
 release?
 master is the branch with correct history (all new stuff is 
 rebased on it) thus new staging will have that too.

Why you don't rewrite is because it is a public branch. Unlike feature branches which will basically be thrown out everyone on the development team will need to have staging updated. If we rewrite history then instead of $ git pull staging At random times it will be (I don't know the commands and won't even look it up) It just won't be pretty. I've made modifications to the graphic hoping to illustrate some thoughts. http://i.imgur.com/rJVSg.png This does not depict what is currently described (in terms of branching). But is what I've written under http://wiki.dlang.org/Release_Process#Release_Schedule I see patches going into the LTS-1 (if applicable), the LTS-1 is then merged into the latest LTS, which is merged into any active staging, that is then merged into master. The monthly release don't get bug fixes (just wait for the next month). I've removed some version numbering since I don't know if we should have a distinct numbering for LTS and Monthly. I've already give some thoughts on this: http://forum.dlang.org/post/ydmgqmbqngwderfkljde forum.dlang.org

Can we drop the LTS name ? It reminds me of ubuntu, and I clearly hope that people promoting that idea don't plan to reproduce ubuntu's scheme : - it is not suitable for a programming language (as stated 3 time now, so just read before why I won't repeat it). - ubuntu is notoriously unstable.
Dec 15 2012
prev sibling next sibling parent "RenatoUtsch" <renatoutsch gmail.com> writes:
On Saturday, 15 December 2012 at 20:39:22 UTC, deadalnix wrote:
 On Saturday, 15 December 2012 at 20:32:42 UTC, Jesse Phillips 
 wrote:
 On Saturday, 15 December 2012 at 10:29:55 UTC, Dmitry 
 Olshansky wrote:
 Second point is about merging master into staging - why not 
 just rewrite it with master branch altogether after each 
 release?
 master is the branch with correct history (all new stuff is 
 rebased on it) thus new staging will have that too.

Why you don't rewrite is because it is a public branch. Unlike feature branches which will basically be thrown out everyone on the development team will need to have staging updated. If we rewrite history then instead of $ git pull staging At random times it will be (I don't know the commands and won't even look it up) It just won't be pretty. I've made modifications to the graphic hoping to illustrate some thoughts. http://i.imgur.com/rJVSg.png This does not depict what is currently described (in terms of branching). But is what I've written under http://wiki.dlang.org/Release_Process#Release_Schedule I see patches going into the LTS-1 (if applicable), the LTS-1 is then merged into the latest LTS, which is merged into any active staging, that is then merged into master. The monthly release don't get bug fixes (just wait for the next month). I've removed some version numbering since I don't know if we should have a distinct numbering for LTS and Monthly. I've already give some thoughts on this: http://forum.dlang.org/post/ydmgqmbqngwderfkljde forum.dlang.org

Can we drop the LTS name ? It reminds me of ubuntu, and I clearly hope that people promoting that idea don't plan to reproduce ubuntu's scheme : - it is not suitable for a programming language (as stated 3 time now, so just read before why I won't repeat it). - ubuntu is notoriously unstable.

Of course, lets just call it "stable", then. Or you have a better name? Anyways, I do think that "stable" releases every 3 or more years and monthly or every 3 months releases are the best solution to the current D users. -- Renato
Dec 15 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Saturday, 15 December 2012 at 19:03:49 UTC, Brad Roberts wrote:
 On 12/15/2012 2:29 AM, Dmitry Olshansky wrote:

 I think one of major goals is to be able to continue ongoing 
 development while at the _same time_ preparing a release.
 To me number one problem is condensed in the statement "we are 
 going to release do not merge anything but regressions"
 the process should sidestep this "lock-based" work-flow. Could 
 be useful to add something along these line to goals
 section. (i.e. the speed and smoothness is a goal)

I've been staying out of this thread for the most part, but I feel the need to comment on this part specifically. It's quite common for most major projects to have a clear "we're wrapping up a release" phase where work _is_ restricted to bug fixing and stabilizing. They don't stop people from working off in their development branches (no one could effectively impose such restrictions even if they wanted to), but they _do_ tighten down on what's allowed to be merged. This is a forcing function that's just required. There's a lot of issues that otherwise won't get sufficient attention. If all it took was altruism then regressions would be fixed immediately, bugs would always be fixed in highest priority to lowest priority (assuming that could even be effectively designed), etc. Without the 'ok, guys, focus in this smaller more critical subset of bugs' step, release branches would be created and never polished (or focused down to the release manager to do all the work if he's that skilled and/or generous of his time). There's a phrase I'm trying to remember, but it's something to the effect that 'hope isn't a recipe for success.' Hoping that people fix regressions on release critical bugs isn't sufficient. Incentive and steering is required. The desire to ungate master branch merges is one approach that's been shown to be successful.

I feel you've made a very important point here, and I've put up a section in the wiki process talk page devoted to the subject. http://wiki.dlang.org/Talk:Release_Process#The_Path_of_Least_Resistance:_Incentives_and_Barriers Although debating ideas in here is welcome, please be sure to post your ideas in the talk page, esp after there's a conclusion made, otherwise your excellent idea may vanish into the Ether and never become implemented as it should have been. --rt
Dec 15 2012
prev sibling next sibling parent "Jesse Phillips" <Jessekphillips+D gmail.com> writes:
On Saturday, 15 December 2012 at 20:39:22 UTC, deadalnix wrote:
 Can we drop the LTS name ? It reminds me of ubuntu, and I 
 clearly hope that people promoting that idea don't plan to 
 reproduce ubuntu's scheme :
  - it is not suitable for a programming language (as stated 3 
 time now, so just read before why I won't repeat it).

You don't need to repeat your self, you need to expand on your points. Joseph has already requested that you give specifics of your objection, you have explained why the situation is different but not what needs to be different. Your points were specific to Debian's model, which is not Ubuntu's.
  - ubuntu is notoriously unstable.

I don't know anyone who uses the LTS releases. That isn't to say no one is, but Ubuntu is doing a lot of experimenting in their 6 month releases.
Dec 15 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Sunday, 16 December 2012 at 02:03:34 UTC, Jesse Phillips wrote:
 On Saturday, 15 December 2012 at 20:39:22 UTC, deadalnix wrote:
 Can we drop the LTS name ? It reminds me of ubuntu, and I 
 clearly hope that people promoting that idea don't plan to 
 reproduce ubuntu's scheme :
 - it is not suitable for a programming language (as stated 3 
 time now, so just read before why I won't repeat it).

You don't need to repeat your self, you need to expand on your points. Joseph has already requested that you give specifics of your objection, you have explained why the situation is different but not what needs to be different. Your points were specific to Debian's model, which is not Ubuntu's.
 - ubuntu is notoriously unstable.

I don't know anyone who uses the LTS releases. That isn't to say no one is, but Ubuntu is doing a lot of experimenting in their 6 month releases.

I think if we focus on the end results that the Ubuntu process is designed to accomplish and what the Debian is designed to accomplish we can start to think about which model is more likely to produce the desired results that we wish to achieve with the D process. I'll sum it up as follows: What both systems attempt to accomplish is in conflict, Ubuntu attempts to create a reasonably stable distribution with more recent updates so that users have access to current software, but that means there will be more bugs and less stability. Debian attempts to distribute a bug free stable distribution, however the delays in getting there mean that the software is often much less current than what is currently available. The end results may be summarized as follows: I would *never* use Ubuntu for mission critical tasks, for example I would not use it to power a 24/7 remote server. I may use it for a workstation at home, but not at work, although I may use it at work if I needed access to the most current software. Why not use it for mission critical tasks? Because it's unstable. Why is it unstable? Because it's based on Debian's unstable branch. Debian stable on the other hand, is rock solid, and works well for mission critical tasks, and is suitable for use in a server environment, however, it does not come with the most current software, a some packages may be a couple of versions behind (which can be very frustrating at times). So to achieve stability vs instability, you have to trade away less mature versions of software for older more mature versions of software. Debian's process is design specifically for stability, Ubuntu's process is designed specifically to provide the most current software in a reasonably stable way. --rt
Dec 15 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Thursday, 13 December 2012 at 07:18:16 UTC, foobar wrote:
 Per my answer to Rob:
 D2 *is* the major version.
 releases should be minor versions and largely backwards 
 compatible - some evolution is allowed given some reasonable 
 restrictions like a proper migration path over several releases.
 Critical bug-fixes can go directly on stable releases but 
 nothing else.

I just realized that we are mixing together the version numbers for two entirely different things. As you have correctly pointed out, one of the version numbers is for the D language specification, which is version 2.something, the other version is for the compiler releases, which has the major language specification version assigned to identify it as supporting version 2.something of the D language. The remaining numbers indicate incremental releases which may also roughly correspond to the minor evolutionary changes to the language specification, i.e., does DMD 2.061 also mean D specification 2.061? I think it makes good sense that the "2" is used to indicate that the compiler supports D major version 2 of the specification, but we should all be clear that is all that it represents. Therefore DMD 3.x will never appear unless there's a D language ver 3 specification for it to support. A completely separate issue that should be dealt with, is that the language specification's version number is not indicated anywhere that I could see. We just assume it's version 2.something and we have no idea where the "something" part is currently at or what has changed since ver 2.0. This is no good because it means that we cannot indicate in the compiler release change log what minor version of the 2.x specification the compiler is actually supporting. --rt
Dec 15 2012
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 16 December 2012 at 02:03:34 UTC, Jesse Phillips wrote:
 On Saturday, 15 December 2012 at 20:39:22 UTC, deadalnix wrote:
 Can we drop the LTS name ? It reminds me of ubuntu, and I 
 clearly hope that people promoting that idea don't plan to 
 reproduce ubuntu's scheme :
 - it is not suitable for a programming language (as stated 3 
 time now, so just read before why I won't repeat it).

You don't need to repeat your self, you need to expand on your points. Joseph has already requested that you give specifics of your objection, you have explained why the situation is different but not what needs to be different.

This is completely backward, but I'll do it anyway. But first, let me explain why it is backward. You are using distro's versionning system as a base of reflexion. But such system is made to achieve different goal than a programming language. I shouldn't be here explaining why this is wrong, you should be here explaining me why it can be applied anyway. Otherwise, anyone can come with any point, whatever how stupid it is, and each time we have to prove that person wrong. When you come with something, you have to explain why it make sens, not the other way around. Back to the point, and it will be the last time. A distro is a set of programs. The goal of the distro is to provide a set of programs, as up to date as possible, that integrate nicely with each other, and with as few bugs as possible. Some of these goals are in conflict, so we see different pattern emerge, with different tradeoff, as ubuntu and debians's processes. From one version to another, distro don't need backward compatibility. Easy migration is important, but not compatibility. This is why debian can switch to multiarch in its next version (which break a hell lot of things). You don't need revision of the distro because software are updated on a per software basis when problems are detected (package in fact, but it don't really matter). This is very different from a programming language where : - no package update is possible. The concept of package don't even exist in our world. - backward compatibility is a very important point, when it isn't for distros. The only goal that is coming is trying to reach some level of stability. Everything else is completely different.
 Your points were specific to Debian's model, which is not 
 Ubuntu's.

 - ubuntu is notoriously unstable.

I don't know anyone who uses the LTS releases. That isn't to say no one is, but Ubuntu is doing a lot of experimenting in their 6 month releases.

I used to work with ubuntu. I've done a ton of things with that distro. It IS unstable. In fact, it is based on debian unstable, so it isn't really a surprise.
Dec 15 2012
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 16 December 2012 at 03:59:33 UTC, Rob T wrote:
 On Thursday, 13 December 2012 at 07:18:16 UTC, foobar wrote:
 Per my answer to Rob:
 D2 *is* the major version.
 releases should be minor versions and largely backwards 
 compatible - some evolution is allowed given some reasonable 
 restrictions like a proper migration path over several 
 releases.
 Critical bug-fixes can go directly on stable releases but 
 nothing else.

I just realized that we are mixing together the version numbers for two entirely different things. As you have correctly pointed out, one of the version numbers is for the D language specification, which is version 2.something, the other version is for the compiler releases, which has the major language specification version assigned to identify it as supporting version 2.something of the D language. The remaining numbers indicate incremental releases which may also roughly correspond to the minor evolutionary changes to the language specification, i.e., does DMD 2.061 also mean D specification 2.061?

Yes it does. If new features are introduced, then this is by definition a change in the specs.
 I think it makes good sense that the "2" is used to indicate 
 that the compiler supports D major version 2 of the 
 specification, but we should all be clear that is all that it 
 represents. Therefore DMD 3.x will never appear unless there's 
 a D language ver 3 specification for it to support.

A version 3 would means introducing change in the specification that may break a lot of code. This is type of stuff you shouldn't do often.
 A completely separate issue that should be dealt with, is that 
 the language specification's version number is not indicated 
 anywhere that I could see. We just assume it's version 
 2.something and we have no idea where the "something" part is 
 currently at or what has changed since ver 2.0. This is no good 
 because it means that we cannot indicate in the compiler 
 release change log what minor version of the 2.x specification 
 the compiler is actually supporting.

The compiler source code is probably what we have that look like the most as a spec right now.
Dec 15 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Sunday, 16 December 2012 at 07:35:27 UTC, deadalnix wrote:
 The only goal that is coming is trying to reach some level of 
 stability. Everything else is completely different.

There are still some clear similarities between what Debian is doing and what I presume most people do in software development. For the software I develop we have what is called a "live" branch, which is the code that is in active use by our customers. This branch corresponds to the stable branch we're trying to achieve, and it only gets critical bug fixes until the next major update which includes new features and/or major adjustments as well as non-critical bug fixes. We also have a testing branch, which includes the latest pre-release code for the next major update. This code is running on a VM and is tested by our customers (and the developers), under conditions similar to "live". There's also a common "dev" branch for the code which is in development but not yet ready for testing. This branch corresponds to the Dev (Master) Branch, The master branch gets updated from individual forks owned by each developer. Coordination among developers is essential to prevent duplications and major conflicts. So in essence we're following a similar model to the Debian model of (Master Dev) which is "unstable " => Pre-release Testing => Stable. It works great, and I see no way to remove any of the branches without seriously compromising the end result. For example, we can't go directly from Dev to Stable, that would be like committing suicide. We can't use testing for dev updates, it's too destabilizing and we'd never get a properly tested stable release out unless all development updates were halted for a long period of time, but then the individual forks would pile up with major changes that would be very difficult to sort out when merged all back together. We need 3 common branches, and I just don't see a way out of that. BTW, I'm using Mercurial which has many similarities to Git. Unfortunately I don't know my way around Git very well at this time. --rt
Dec 16 2012
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 16 December 2012 at 08:30:04 UTC, Rob T wrote:
 On Sunday, 16 December 2012 at 07:35:27 UTC, deadalnix wrote:
 The only goal that is coming is trying to reach some level of 
 stability. Everything else is completely different.

There are still some clear similarities between what Debian is doing and what I presume most people do in software development. For the software I develop we have what is called a "live" branch, which is the code that is in active use by our customers. This branch corresponds to the stable branch we're trying to achieve, and it only gets critical bug fixes until the next major update which includes new features and/or major adjustments as well as non-critical bug fixes. We also have a testing branch, which includes the latest pre-release code for the next major update. This code is running on a VM and is tested by our customers (and the developers), under conditions similar to "live". There's also a common "dev" branch for the code which is in development but not yet ready for testing. This branch corresponds to the Dev (Master) Branch, The master branch gets updated from individual forks owned by each developer. Coordination among developers is essential to prevent duplications and major conflicts. So in essence we're following a similar model to the Debian model of (Master Dev) which is "unstable " => Pre-release Testing => Stable. It works great, and I see no way to remove any of the branches without seriously compromising the end result. For example, we can't go directly from Dev to Stable, that would be like committing suicide. We can't use testing for dev updates, it's too destabilizing and we'd never get a properly tested stable release out unless all development updates were halted for a long period of time, but then the individual forks would pile up with major changes that would be very difficult to sort out when merged all back together. We need 3 common branches, and I just don't see a way out of that. BTW, I'm using Mercurial which has many similarities to Git. Unfortunately I don't know my way around Git very well at this time.

So distro's versioning system is good for a programming language because you use it successfully in your software which isn't a programming language (and we also don't know according to which goal it is successful) ? By the way, debian testing is not what you think it is : http://www.debian.org/devel/testing.en.html
Dec 16 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Sunday, 16 December 2012 at 07:39:09 UTC, deadalnix wrote:
 The compiler source code is probably what we have that look 
 like the most as a spec right now.

Unfortunately that is likely the case. I hope we can all agree that the specification should be managed better, but it's a much lesser problem at the moment than what we're trying to solve in here. I think once we can establish a process for development testing and release, the other problem areas will stand out a lot more clearly for elimination. --rt
Dec 16 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Sunday, 16 December 2012 at 08:52:24 UTC, deadalnix wrote:
 So distro's versioning system is good for a programming 
 language because you use it successfully in your software which 
 isn't a programming language (and we also don't know according 
 to which goal it is successful) ?

Let's not get different things mixed together. There is a compiler, which is software. There is a language, but it is not software, it's the specification that the compiler implements. It does not matter how branches are versioned, what matters is how they move from a highly unstable state towards an increasingly stable state until it is released in stable form for the end-user to use. I will agree with you that there are unique challenges for a compiler, however we're not really talking about a compiler specifically at this stage, we're talking about establishing a way of moving source code from an unstable form into a stable form. What the compiler needs to achieve is a totally different concept, which is unfortunately not well documented. There is the current language specification, which is improperly managed as I pointed out in a previous post, and there needs to be certain guidelines set for the compiler to follow that are not a part of the language specification, but we're straying way off course at this point - those problem areas will have to be dealt with later, otherwise absolutely nothing at all will get done. There's just too many things to fix up around here, so we have to pick and chose very carefully what to deal with first before moving on to the next problem.
 By the way, debian testing is not what you think it is : 
 http://www.debian.org/devel/testing.en.html

I think that I do know what Debian testing is. I do understand that the Debian distro is made up out of individual packages, but these packages can be generalized to what goes on when software is developed. For example, new features = new packages, code modifications = updated packages. When you distill things down to a process level, the payload managed by the process no longer matters so much. --rt
Dec 16 2012
prev sibling next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 12/16/2012 08:35 AM, deadalnix wrote:
 On Sunday, 16 December 2012 at 02:03:34 UTC, Jesse Phillips wrote:
 You don't need to repeat your self, you need to expand on your points. Joseph
 has already requested that you give specifics of your objection, you have
 explained why the situation is different but not what needs to be different.

This is completely backward, but I'll do it anyway. But first, let me explain why it is backward. You are using distro's versionning system as a base of reflexion. But such system is made to achieve different goal than a programming language. I shouldn't be here explaining why this is wrong, you should be here explaining me why it can be applied anyway.

And once again, you're failing to address the specifics of my proposal, simply saying "A distro and a programming language have different requirements." Because while some of my proposals were inspired by some of the ways in which Ubuntu manages its releases, and share some of the same terminology, _they are not the same_. In fact the only thing that it really shares with Ubuntu is the idea that the support and maintenance periods for long-term "stable" versions should overlap so that users who require that stable base have adequate time to upgrade. That's useful for _any_ software no matter what the purpose. So once again I'd ask you to engage with the specifics of my proposal -- because from what I've read so far, it seems like the _only_ source of your objections is the use of the words "Ubuntu" and "Long-Term Support", and that if I'd written the same proposal without those words, you'd have given a different response.
Dec 16 2012
prev sibling next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 12/15/2012 09:39 PM, deadalnix wrote:
 Can we drop the LTS name ? It reminds me of ubuntu, and I clearly hope that
 people promoting that idea don't plan to reproduce ubuntu's scheme :
   - it is not suitable for a programming language (as stated 3 time now, so
just
 read before why I won't repeat it).
   - ubuntu is notoriously unstable.

Call them "stable release cycles" if you like, which is what they are intended to be. It's just that "long-term support" actually conveys quite well what _users_ can expect using these versions. By the way, no one has proposed "using Ubuntu's scheme". I made a proposal that drew on a number of ideas inspired by Ubuntu's release model, including terminology, but the actual detail was quite different and was tailored to my perceived requirements of D, not an operating system. So, once again, I'd appreciate your response to those details, rather than your perception that I'm proposing "using Ubuntu's scheme".
Dec 16 2012
prev sibling next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 12/16/2012 04:05 PM, Andrei Alexandrescu wrote:
 Just one tidbit of information: I talked to Walter and we want to build into
the
 process the ability to modify any particular release. (One possibility is to do
 so as part of paid support for large corporate users.) That means there needs
to
 be one branch per release.

Can you expand on the requirements/goals there a bit? Would be useful to have a good sense of what kind of modifications you have in mind. I don't think anything in what I've proposed (elsewhere in this thread, I'm afraid I haven't written it up on the wiki yet...) is incompatible with the notion of one branch per release.
Dec 16 2012
prev sibling next sibling parent "foobar" <foo bar.com> writes:
On Sunday, 16 December 2012 at 15:05:58 UTC, Andrei Alexandrescu 
wrote:
 On 12/16/12 6:15 AM, Joseph Rushton Wakeling wrote:
 On 12/15/2012 09:39 PM, deadalnix wrote:
 Can we drop the LTS name ? It reminds me of ubuntu, and I 
 clearly hope
 that
 people promoting that idea don't plan to reproduce ubuntu's 
 scheme :
 - it is not suitable for a programming language (as stated 3 
 time now,
 so just
 read before why I won't repeat it).
 - ubuntu is notoriously unstable.

Call them "stable release cycles" if you like, which is what they are intended to be.

Just one tidbit of information: I talked to Walter and we want to build into the process the ability to modify any particular release. (One possibility is to do so as part of paid support for large corporate users.) That means there needs to be one branch per release. Andrei

I don't see why that is a requirement (having a branch per release). We can still have a single stable branch with tags for releases and when Walter needs to provide special customizations he can always just branch off of the tagged release. This should be Walter's business and not part of the "official" community process. in git terms, assuming we have a tagged release of 2.61 $ git checkout -b 2.61-partner 2.61 This branches off a new branch "2.61-partner" for the specific partner modification based off the contents of the "2.61" release. Also, this kinda messes with the notion of integration. A single stable branch helps prevent "forgotten" bug-fixes. I.e a critical bug was fixed on release N, what will ensure it will be included in release N+1? If Release N+1 is on the same branch than the bug-fix is included by default and prevented the need to perform a manual operation (a merge) that could be forgotten.
Dec 16 2012
prev sibling next sibling parent "Jesse Phillips" <Jesse.K.Phillips+D gmail.com> writes:
On Sunday, 16 December 2012 at 18:37:48 UTC, Andrei Alexandrescu 
wrote:

 The prevalent use of the feature would be that a bug in a 
 future release is back-patched onto an older release.


 Andrei

The issue I see is we will have X releases. A bug is fixed, do we now apply it to all X or only Y releases? I see it unreasonable to maintain every release. If we need to make a special "maintenance" release then we directly handle the issue being requested instead of providing every fix applicable. Branches work when needing to handle continuous support, but tags are the perfect lite weight solution when you'll have to handle one off support. It can be done right now, but I don't know what version you'd release 2.57 + 1 as.
Dec 16 2012
prev sibling next sibling parent "Jesse Phillips" <Jesse.K.Phillips+D gmail.com> writes:
On Sunday, 16 December 2012 at 07:35:27 UTC, deadalnix wrote:
 I shouldn't be here explaining why this is wrong, you should be 
 here explaining me why it can be applied anyway.

We are developing a process, specifying what it will look like. If I'm building an airplane and say I think it should be kind of like a car, I'd expect your response would be. You can't build an airplane with wheels, windshield, and a cabin for passengers. Cars have those and meant for driving on the ground with very sticked adherence to the laws of gravity. What I expect you to say, we shouldn't be building an airplane, helicopters are much better at traveling and landing in tight places and that is what we need. Airplanes are good for when we need long distance. Instead I hear, plans are good for long distance we shouldn't build them.
 Otherwise, anyone can come with any point, whatever how stupid 
 it is, and each time we have to prove that person wrong. When 
 you come with something, you have to explain why it make sens, 
 not the other way around.

We have explained why it makes sense, the goals are already up on the wiki. So you should review those goals and explain why the plan does not meet those, or correct the goals. You are doing neither.
 Back to the point, and it will be the last time. A distro is a 
 set of programs. The goal of the distro is to provide a set of 
 programs, as up to date as possible, that integrate nicely with 
 each other, and with as few bugs as possible. Some of these 
 goals are in conflict, so we see different pattern emerge, with 
 different tradeoff, as ubuntu and debians's processes.

I said you don't have to repeat yourself.
Dec 16 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Sunday, 16 December 2012 at 16:23:57 UTC, Andrei Alexandrescu 
wrote:
 Right now we're using a tagging system for releases, implying 
 releases are just snapshots of a continuum. But what we need is 
 e.g. to be able to patch 2.065 to fix a bug for a client who 
 can't upgrade right now. That means one branch per release.

I don't see a problem with retaining release branches for some time, and I think we were planning to do this as a means to support previous stable versions for some time period. However, patching a downstream release while leaving upstream unpatched is not going to work well within the public branches, the flow must be from up stream to down stream, otherwise it'll be a confusing mess where older versions may have specific bug fixes but higher versions don't have the same bug fixes. In addition stable branches are forbidden from receiving new features because that will destabilize them and in addition upstream will be out of sync, causing a lot of confusion. From what I can see, part of what you are proposing will have to be performed off-site as a private affair between the corporate user and the service provider. What we can certainly provide is a stable branch as-is when it was released, and that branch can certainly receive any patches that made their way that far down stream, however the patches will be bug fixes only. --rt
Dec 16 2012
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 16 December 2012 at 18:37:48 UTC, Andrei Alexandrescu 
wrote:
 This should be Walter's business and not part
 of the "official" community process.

I think this is rather shortsighted. Corporate support by Walter is only one possible scenario, but there are many others.

Indeed. To avoid the proliferation of branch that need support, it is better to slow down the number of new feature that each release provide and release them by packs. So less branch need to be maintained.
 The prevalent use of the feature would be that a bug in a 
 future release is back-patched onto an older release.

It is usually easier to go from the oldest to newest than the other way around.
Dec 16 2012
prev sibling next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 12/16/2012 05:23 PM, Andrei Alexandrescu wrote:
 Right now we're using a tagging system for releases, implying releases are just
 snapshots of a continuum. But what we need is e.g. to be able to patch 2.065 to
 fix a bug for a client who can't upgrade right now. That means one branch per
 release.

Ahh, OK. That makes complete sense, in fact I'd assumed it was a given that we'd wind up with one branch per release and release-specific bugfixes, which is why your remarks made me wonder if there was something more that needed explaining.
Dec 16 2012
prev sibling next sibling parent "SomeDude" <lovelydear mailmetrash.com> writes:
On Sunday, 16 December 2012 at 15:05:58 UTC, Andrei Alexandrescu 
wrote:
 On 12/16/12 6:15 AM, Joseph Rushton Wakeling wrote:
 On 12/15/2012 09:39 PM, deadalnix wrote:
 Can we drop the LTS name ? It reminds me of ubuntu, and I 
 clearly hope
 that
 people promoting that idea don't plan to reproduce ubuntu's 
 scheme :
 - it is not suitable for a programming language (as stated 3 
 time now,
 so just
 read before why I won't repeat it).
 - ubuntu is notoriously unstable.

Call them "stable release cycles" if you like, which is what they are intended to be.

Just one tidbit of information: I talked to Walter and we want to build into the process the ability to modify any particular release. (One possibility is to do so as part of paid support for large corporate users.) That means there needs to be one branch per release. Andrei

This sounds to me like a bad idea. And indeed, I haven't heard of any other project doing this. If you do so, you'll quickly stop maintaining the older branches (especially when the corporate users pay for their specific developments), forcing users to hop to newer branches, with the possibility of breaking changes, so it's no different than today's situation. If they really want specific developments, let them have their own branch and not interfere with a community driven process. In fact, I may sound harsh and a bit extreme, but I think paying users should have a priority to bugfixes and that's it, not on the development of the language itself, as they will attempt to rush half baked features. I do think a bleeding edge branch and one or two stable branches (let's say one per year for the last two years) is good enough. And one branch for the paying user, which will merge to the bleeding edge if its specific developments prove worthwhile. But as soon as there are more than one paying user, it will instantaneously become impractical, so that really, they should just be limited to bugfixes. BTW, will they share with the community their own developments ?
Dec 16 2012
prev sibling next sibling parent "SomeDude" <lovelydear mailmetrash.com> writes:
On Sunday, 16 December 2012 at 22:18:14 UTC, SomeDude wrote:
 On Sunday, 16 December 2012 at 15:05:58 UTC, Andrei 
 Alexandrescu wrote:
 On 12/16/12 6:15 AM, Joseph Rushton Wakeling wrote:
 On 12/15/2012 09:39 PM, deadalnix wrote:
 Can we drop the LTS name ? It reminds me of ubuntu, and I 
 clearly hope
 that
 people promoting that idea don't plan to reproduce ubuntu's 
 scheme :
 - it is not suitable for a programming language (as stated 3 
 time now,
 so just
 read before why I won't repeat it).
 - ubuntu is notoriously unstable.

Call them "stable release cycles" if you like, which is what they are intended to be.

Just one tidbit of information: I talked to Walter and we want to build into the process the ability to modify any particular release. (One possibility is to do so as part of paid support for large corporate users.) That means there needs to be one branch per release. Andrei

This sounds to me like a bad idea. And indeed, I haven't heard of any other project doing this. If you do so, you'll quickly stop maintaining the older branches (especially when the corporate users pay for their specific developments), forcing users to hop to newer branches, with the possibility of breaking changes, so it's no different than today's situation. If they really want specific developments, let them have their own branch and not interfere with a community driven process. In fact, I may sound harsh and a bit extreme, but I think paying users should have a priority to bugfixes and that's it, not on the development of the language itself, as they will attempt to rush half baked features. I do think a bleeding edge branch and one or two stable branches (let's say one per year for the last two years) is good enough. And one branch for the paying user, which will merge to the bleeding edge if its specific developments prove worthwhile. But as soon as there are more than one paying user, it will instantaneously become impractical, so that really, they should just be limited to bugfixes. BTW, will they share with the community their own developments ?

Actually, it's pretty much what's written in the wiki. http://wiki.dlang.org/Release_Process#Release_Schedule
Dec 16 2012
prev sibling next sibling parent "David Nadlinger" <see klickverbot.at> writes:
On Sunday, 16 December 2012 at 22:18:14 UTC, SomeDude wrote:
 This sounds to me like a bad idea. And indeed, I haven't heard 
 of any other project doing this.

Having release branches is a common practice for many open source projects. For example, KDE creates a branch per minor release, with patch releases being made off that branch. LLVM also has a branch for each release. David
Dec 16 2012
prev sibling next sibling parent "Jesse Phillips" <Jesse.K.Phillips+D gmail.com> writes:
On Sunday, 16 December 2012 at 20:08:11 UTC, Jacob Carlborg wrote:
 On 2012-12-16 20:04, Jesse Phillips wrote:

 I see it unreasonable to maintain every release. If we need to 
 make a
 special "maintenance" release then we directly handle the 
 issue being
 requested instead of providing every fix applicable.

We could say that we support X number of older releases.

That was my Y. A patch should be created off the oldest supported version. If Y is 5 then that means checking 5 releases to decided where to create the patch from. The current suggestion makes Y at most 3 and generally 2. What is being requested is to back port a patch for customer using version Z, even if Z is 2.34. It is a special case and the driving force could be any number of reasons. The age is not really important, but the point is the community officially dropped support but someone made a compelling argument to bring that support back.
Dec 16 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, December 16, 2012 11:23:57 Andrei Alexandrescu wrote:
 Right now we're using a tagging system for releases, implying releases
 are just snapshots of a continuum. But what we need is e.g. to be able
 to patch 2.065 to fix a bug for a client who can't upgrade right now.
 That means one branch per release.

Well, you can still do that. You just create a branch from the tag when you need a branch. You don't have to branch from the latest. You can branch from any point on the tree, and tags provide a good marker for points to branch if you need to. Then the official release is still clearly tagged, but you can have branches based off it when you need it. It also avoids some minor overhead when patching releases is rare. My main concern though would be that the actual release needs to be clearly marked separately from any patches, so it pretty much requires a tag regardless of what you do with branches. What I would have expected that we do (though I haven't read through most of this thread or the wiki yet, so I don't know what's being proposed) would be to branch when we do beta, and that branch would be what would ultimately end up where we release from, with a tag on the commit which was the actual release. If further commits were made for specific clients after the release, then either you'd make a branch from that which was specific to that client, or you'd put them on the same branch, where'd they'd be after the tag for the release and wouldn't affect it. - Jonathan M Davis
Dec 16 2012
prev sibling next sibling parent "SomeDude" <lovelydear mailmetrash.com> writes:
On Sunday, 16 December 2012 at 22:43:13 UTC, David Nadlinger 
wrote:
 On Sunday, 16 December 2012 at 22:18:14 UTC, SomeDude wrote:
 This sounds to me like a bad idea. And indeed, I haven't heard 
 of any other project doing this.

Having release branches is a common practice for many open source projects. For example, KDE creates a branch per minor release, with patch releases being made off that branch. LLVM also has a branch for each release. David

This can only work if there are very few releases, else one has to patch all the branches since a bug has been discovered. If you release once per month, you potentially have to patch 10 or 20 branches.
Dec 16 2012
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Monday, 17 December 2012 at 05:29:23 UTC, SomeDude wrote:
 On Sunday, 16 December 2012 at 22:43:13 UTC, David Nadlinger 
 wrote:
 On Sunday, 16 December 2012 at 22:18:14 UTC, SomeDude wrote:
 This sounds to me like a bad idea. And indeed, I haven't 
 heard of any other project doing this.

Having release branches is a common practice for many open source projects. For example, KDE creates a branch per minor release, with patch releases being made off that branch. LLVM also has a branch for each release. David

This can only work if there are very few releases, else one has to patch all the branches since a bug has been discovered. If you release once per month, you potentially have to patch 10 or 20 branches.

This is exactly to handle this kind of grunt work that computer and software were invented.
Dec 16 2012
prev sibling next sibling parent "Jesse Phillips" <Jessekphillips+D gmail.com> writes:
On Monday, 17 December 2012 at 07:04:29 UTC, deadalnix wrote:

 This is exactly to handle this kind of grunt work that computer 
 and software were invented.

Yep, and why Git handles merging so well. But it doesn't know which branches you are merging into until you tell it. So you can go ahead and write the program to do that for you.
Dec 17 2012
prev sibling next sibling parent "foobar" <foo bar.com> writes:
On Sunday, 16 December 2012 at 22:43:13 UTC, David Nadlinger 
wrote:
 On Sunday, 16 December 2012 at 22:18:14 UTC, SomeDude wrote:
 This sounds to me like a bad idea. And indeed, I haven't heard 
 of any other project doing this.

Having release branches is a common practice for many open source projects. For example, KDE creates a branch per minor release, with patch releases being made off that branch. LLVM also has a branch for each release. David

Huh? Both LLVM and KDE are developed on *subversion* and as such their work-flows are not applicable. Not to mention that KDE is vastly different in concept and goals than a programming language. Subversion is conceptually very different from git and its model imposes practical restrictions that are not relevant for git, mostly with regards to branches, merging, etc. Actions which are first class and trivial to accomplish in git. This is analogous to designing highways based on the speed properties of bicycles.
Dec 17 2012
prev sibling next sibling parent "foobar" <foo bar.com> writes:
On Sunday, 16 December 2012 at 23:18:20 UTC, Jonathan M Davis 
wrote:
 On Sunday, December 16, 2012 11:23:57 Andrei Alexandrescu wrote:
 Right now we're using a tagging system for releases, implying 
 releases
 are just snapshots of a continuum. But what we need is e.g. to 
 be able
 to patch 2.065 to fix a bug for a client who can't upgrade 
 right now.
 That means one branch per release.

Well, you can still do that. You just create a branch from the tag when you need a branch. You don't have to branch from the latest. You can branch from any point on the tree, and tags provide a good marker for points to branch if you need to. Then the official release is still clearly tagged, but you can have branches based off it when you need it. It also avoids some minor overhead when patching releases is rare. My main concern though would be that the actual release needs to be clearly marked separately from any patches, so it pretty much requires a tag regardless of what you do with branches. What I would have expected that we do (though I haven't read through most of this thread or the wiki yet, so I don't know what's being proposed) would be to branch when we do beta, and that branch would be what would ultimately end up where we release from, with a tag on the commit which was the actual release. If further commits were made for specific clients after the release, then either you'd make a branch from that which was specific to that client, or you'd put them on the same branch, where'd they'd be after the tag for the release and wouldn't affect it. - Jonathan M Davis

Precisely. Thank you.
Dec 17 2012
prev sibling next sibling parent "David Nadlinger" <see klickverbot.at> writes:
On Monday, 17 December 2012 at 17:31:45 UTC, foobar wrote:
 Huh?
 Both LLVM and KDE are developed on *subversion* and as such 
 their work-flows are not applicable. Not to mention that KDE is 
 vastly different in concept and goals than a programming 
 language.

 Subversion is conceptually very different from git and its 
 model imposes practical restrictions that are not relevant for 
 git, mostly with regards to branches, merging, etc. Actions 
 which are first class and trivial to accomplish in git. This is 
 analogous to designing highways based on the speed properties 
 of bicycles.

Guess what, I know that. The post by SomeDude just claimed that release branches in general are impractical and not used by open source projects, which is wrong. David
Dec 17 2012
prev sibling next sibling parent "foobar" <foo bar.com> writes:
On Monday, 17 December 2012 at 17:45:12 UTC, David Nadlinger 
wrote:
 On Monday, 17 December 2012 at 17:31:45 UTC, foobar wrote:
 Huh?
 Both LLVM and KDE are developed on *subversion* and as such 
 their work-flows are not applicable. Not to mention that KDE 
 is vastly different in concept and goals than a programming 
 language.

 Subversion is conceptually very different from git and its 
 model imposes practical restrictions that are not relevant for 
 git, mostly with regards to branches, merging, etc. Actions 
 which are first class and trivial to accomplish in git. This 
 is analogous to designing highways based on the speed 
 properties of bicycles.

Guess what, I know that. The post by SomeDude just claimed that release branches in general are impractical and not used by open source projects, which is wrong. David

At least the first part in that sentence is correct - there are more practical work flows that are just more difficult to achieve in svn. The branch per release in those projects is just a consequence of SVN limitations. hence open source projects _that use git_ don't need to follow this route. Either way, this is completely irrelevant for our purpose. The process should be designed with DVCS in mind since we already settled on this [very successful] model for D. We should avoid designing a work-flow based on other models and limited experience with git. Suggestions such as implementing specific [shell?] scripts and having branch-per-release brings no improvement over what we already have. I personally transitioned my [previous] team from ancient systems (rcs, cvs, proprietary in-house crap) to git. It requires not only memorizing a few new command line commands but also grokking a different model. Those that do, use git to great effect and greatly increase their efficiency, others simply have a "ci" script that calls "git commit" and gain nothing. At the moment we may use git commands but really we are still developing on mostly a subversion model. Walter used to accept patches and those were simply replaced by pull requests. There isn't any change in the mental model required to really benefit from a decentralized system such as git. This is what the process discussion is ultimately meant to fix.
Dec 17 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Monday, 17 December 2012 at 18:14:54 UTC, foobar wrote:
 At the moment we may use git commands but really we are still 
 developing on mostly a subversion model. Walter used to accept 
 patches and those were simply replaced by pull requests. There 
 isn't any change in the mental model required to really benefit 
 from a decentralized system such as git. This is what the 
 process discussion is ultimately meant to fix.

I think you've made a very good point. In order to make the most out of the way decentralized systems work vs a centralized one, will require an adjustment in the way people think. If we choose to follow the centralized model nothing will be gained when using a decentralized service, however centralization has its place and there's an obvious need to merge decentralized branches into a common centralize branch so that there's a common branch to use for testing and performing releases (etc). I find it's great to debate ideas in here first, not the talk page, but any conclusions or interesting points of view should be posted into the wiki talk page so that it is not lost. IMO this one should go in the wiki if only to remind people that we have the flexibility of decentralized model to take advantage of. --rt
Dec 17 2012
prev sibling next sibling parent "foobar" <foo bar.com> writes:
On Monday, 17 December 2012 at 21:03:04 UTC, Rob T wrote:
 On Monday, 17 December 2012 at 18:14:54 UTC, foobar wrote:
 At the moment we may use git commands but really we are still 
 developing on mostly a subversion model. Walter used to accept 
 patches and those were simply replaced by pull requests. There 
 isn't any change in the mental model required to really 
 benefit from a decentralized system such as git. This is what 
 the process discussion is ultimately meant to fix.

I think you've made a very good point. In order to make the most out of the way decentralized systems work vs a centralized one, will require an adjustment in the way people think. If we choose to follow the centralized model nothing will be gained when using a decentralized service, however centralization has its place and there's an obvious need to merge decentralized branches into a common centralize branch so that there's a common branch to use for testing and performing releases (etc). I find it's great to debate ideas in here first, not the talk page, but any conclusions or interesting points of view should be posted into the wiki talk page so that it is not lost. IMO this one should go in the wiki if only to remind people that we have the flexibility of decentralized model to take advantage of. --rt

DVCS is not about centralized vs. non centralized. This is a common misunderstanding which I too had when I started using git. The actual difference is a client-server topology (CVS, SVN, etc) vs. P2P or perhaps "free-form" topology. By making all users equal with their own full copies of the repository, git and similar systems made the topology an aspect of the human work-flow instead of part of the technical design & implementation. This gives you the freedom to have whatever topology you want - a star, a circle, whatever. For instance, Linux is developed with a "web of trust". the topology represents trust relationships. Thus, all the people Linus is pulling from directly are "core" developers he personally trust. They in turn trust other developers, and so on and so forth. Linus's version is the semi-official release of the Linux kernel but it is not the only release. For instance, Linux distributions can have their own repositories, and Google maintains their own repository for the android fork. So in fact, there are *multiple* repositories that represent graph roots in this "web of trust" topology. What about D? The current git-hub repository owned by Walter and the core devs (the github organization) is the official repository. *But*, we could also treat other compilers as roots. more over, there's no requirement for developers to go through the "main" github repository to share and sync. E.g Walter can pull directly from Don's repository *without* going through a formal branch on github. This in fact should be *the default workflow* for internal collaboration to reduce clutter and facilitate better organization. This is why I'm arguing fiercely against having any sort of official alpha stage. There is no need standardizing this and it only mixes "private" developer only code and builds with builds aimed at end-users (those would be us, people writing D code and compiling with DMD). If you look on SVN servers, you often find an endless list of developer folders, "test" branches, "experimental" branches, etc, etc. As a user, this is highly annoying to figure out what branches are meant for user consumption (releases, betas, preview for a new feature) and what isn't, dev X's private place to conduct experiments. This is only a result of the client-server topology imposed by svn architecture. The official process should only standardize and focus on the points where integration is required. Everything else is much better served as being left alone. On the other hand, I think that we need to add more focus on pre/post stages of the actual coding. This means, the planning, setting priorities (aka roadmap, mile-stones, etc) as well as reducing regressions. I saw a post suggesting an automated build-bot that will run the test suit and build nightlies. What about DIPs, how do they integrate in the work-flow? I think Andrei talked about the DIPs before but I don't thik it was discussed as part of this thread.
Dec 17 2012
prev sibling next sibling parent "foobar" <foo bar.com> writes:
On Monday, 17 December 2012 at 22:02:45 UTC, foobar wrote:
 On Monday, 17 December 2012 at 21:03:04 UTC, Rob T wrote:
 On Monday, 17 December 2012 at 18:14:54 UTC, foobar wrote:
 At the moment we may use git commands but really we are still 
 developing on mostly a subversion model. Walter used to 
 accept patches and those were simply replaced by pull 
 requests. There isn't any change in the mental model required 
 to really benefit from a decentralized system such as git. 
 This is what the process discussion is ultimately meant to 
 fix.

I think you've made a very good point. In order to make the most out of the way decentralized systems work vs a centralized one, will require an adjustment in the way people think. If we choose to follow the centralized model nothing will be gained when using a decentralized service, however centralization has its place and there's an obvious need to merge decentralized branches into a common centralize branch so that there's a common branch to use for testing and performing releases (etc). I find it's great to debate ideas in here first, not the talk page, but any conclusions or interesting points of view should be posted into the wiki talk page so that it is not lost. IMO this one should go in the wiki if only to remind people that we have the flexibility of decentralized model to take advantage of. --rt

DVCS is not about centralized vs. non centralized. This is a common misunderstanding which I too had when I started using git. The actual difference is a client-server topology (CVS, SVN, etc) vs. P2P or perhaps "free-form" topology. By making all users equal with their own full copies of the repository, git and similar systems made the topology an aspect of the human work-flow instead of part of the technical design & implementation. This gives you the freedom to have whatever topology you want - a star, a circle, whatever. For instance, Linux is developed with a "web of trust". the topology represents trust relationships. Thus, all the people Linus is pulling from directly are "core" developers he personally trust. They in turn trust other developers, and so on and so forth. Linus's version is the semi-official release of the Linux kernel but it is not the only release. For instance, Linux distributions can have their own repositories, and Google maintains their own repository for the android fork. So in fact, there are *multiple* repositories that represent graph roots in this "web of trust" topology. What about D? The current git-hub repository owned by Walter and the core devs (the github organization) is the official repository. *But*, we could also treat other compilers as roots. more over, there's no requirement for developers to go through the "main" github repository to share and sync. E.g Walter can pull directly from Don's repository *without* going through a formal branch on github. This in fact should be *the default workflow* for internal collaboration to reduce clutter and facilitate better organization. This is why I'm arguing fiercely against having any sort of official alpha stage. There is no need standardizing this and it only mixes "private" developer only code and builds with builds aimed at end-users (those would be us, people writing D code and compiling with DMD). If you look on SVN servers, you often find an endless list of developer folders, "test" branches, "experimental" branches, etc, etc. As a user, this is highly annoying to figure out what branches are meant for user consumption (releases, betas, preview for a new feature) and what isn't, dev X's private place to conduct experiments. This is only a result of the client-server topology imposed by svn architecture. The official process should only standardize and focus on the points where integration is required. Everything else is much better served as being left alone. On the other hand, I think that we need to add more focus on pre/post stages of the actual coding. This means, the planning, setting priorities (aka roadmap, mile-stones, etc) as well as reducing regressions. I saw a post suggesting an automated build-bot that will run the test suit and build nightlies. What about DIPs, how do they integrate in the work-flow? I think Andrei talked about the DIPs before but I don't thik it was discussed as part of this thread.

I forgot to explain that the multiple roots is applicable to D as well in that we have there fully working compilers - LDC, GDC and DMD. They currently share the same front-end - GDC and LDC merge DMD's code. This situation is also sub optimal and requires manual work and has redundancies in the process. E.g. what if the LDC guys fix a bug in the "shared" front-end? Should this be integrated back to DMD? What if Walter comes up with a different fix?
Dec 17 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Monday, 17 December 2012 at 22:10:12 UTC, foobar wrote:
 I forgot to explain that the multiple roots is applicable to D 
 as well in that we have there fully working compilers - LDC, 
 GDC and DMD. They currently share the same front-end - GDC and 
 LDC merge DMD's code. This situation is also sub optimal and 
 requires manual work and has redundancies in the process. E.g. 
 what if the LDC guys fix a bug in the "shared" front-end? 
 Should this be integrated back to DMD? What if Walter comes up 
 with a different fix?

It will take me some time to read and digest all of the points you've made, so with that in mind I have some reading and thinking to do, but I am aware that LDC, GDC and possibly some others are poorly supported through whatever process we have going on at the moment, and you are raising another perfectly valid point that should be listed in the wiki page (if you have not done so already). For example, one of our goals should be that the process will include specifics that are intended to support the various DMD front ends that are out there. For supporting various front ends better, there's of course much more that can be done than adjusting the process, but as you know that's off topic and for later consideration, however some of it may have to be considered in order to fully understand what will eventually have to be done which may have an effect on the process we're conjuring up. It would be great if the LDC and and GDC guys could toss in their thoughts on this subject with respect to what the process must do in order to support them better. --rt
Dec 17 2012
prev sibling next sibling parent "SomeDude" <lovelydear mailmetrash.com> writes:
On Monday, 17 December 2012 at 17:45:12 UTC, David Nadlinger 
wrote:
 On Monday, 17 December 2012 at 17:31:45 UTC, foobar wrote:
 Huh?
 Both LLVM and KDE are developed on *subversion* and as such 
 their work-flows are not applicable. Not to mention that KDE 
 is vastly different in concept and goals than a programming 
 language.

 Subversion is conceptually very different from git and its 
 model imposes practical restrictions that are not relevant for 
 git, mostly with regards to branches, merging, etc. Actions 
 which are first class and trivial to accomplish in git. This 
 is analogous to designing highways based on the speed 
 properties of bicycles.

Guess what, I know that. The post by SomeDude just claimed that release branches in general are impractical and not used by open source projects, which is wrong. David

I was actually correct, saying that it can only work with very few releases, and indeed, for KDE, there are at most 2 releases per year and usually less. For instance, the 4.10 is scheduled to span from october 2012 to july 2013, with numerous patches and corrective releases in between. So it's roughly the same release process as the one branch/year I advocated. And it's not the same at all as creating one branch every month.
Dec 18 2012
prev sibling next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 12/18/2012 09:48 PM, SomeDude wrote:
 And it's not the same at all as creating one branch every month.

But why would you do that? The general discussion seems to have been around the idea of 1 or 2 stable releases per year, 1 every 3 months max.
Dec 18 2012
prev sibling next sibling parent "SomeDude" <lovelydear mailmetrash.com> writes:
On Tuesday, 18 December 2012 at 20:55:34 UTC, Joseph Rushton 
Wakeling wrote:
 On 12/18/2012 09:48 PM, SomeDude wrote:
 And it's not the same at all as creating one branch every 
 month.

But why would you do that? The general discussion seems to have been around the idea of 1 or 2 stable releases per year, 1 every 3 months max.

THat's what I understood from Andrei's post:
 Just one tidbit of information: I talked to Walter and we want 
 to build into the process the ability to modify any particular 
 release. (One possibility is to do so as part of paid support 
 for large corporate users.) That means there needs to be one 
 branch per release.

 Andrei

Maybe I misunderstood him, I don't know.
Dec 18 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Tuesday, 18 December 2012 at 22:37:10 UTC, SomeDude wrote:
 On Tuesday, 18 December 2012 at 20:55:34 UTC, Joseph Rushton 
 Wakeling wrote:
 On 12/18/2012 09:48 PM, SomeDude wrote:
 And it's not the same at all as creating one branch every 
 month.

But why would you do that? The general discussion seems to have been around the idea of 1 or 2 stable releases per year, 1 every 3 months max.

THat's what I understood from Andrei's post:
 Just one tidbit of information: I talked to Walter and we want 
 to build into the process the ability to modify any particular 
 release. (One possibility is to do so as part of paid support 
 for large corporate users.) That means there needs to be one 
 branch per release.

 Andrei

Maybe I misunderstood him, I don't know.

He seemed to imply that, but to me it makes little sense to support older versions of a stable release. The support comes from the latest stable bug fix update, and you normally don't bother propagating a bug fix any further down stream. The best you may do, is after a new release is made, continue supporting the latest older release for a limited period of time until the newest release is considered completely free of any major bugs. If there's a code breaking release, I can see the previous older stable version being supported for a fairly long period of time. --rt
Dec 18 2012
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 18 December 2012 at 20:55:34 UTC, Joseph Rushton 
Wakeling wrote:
 On 12/18/2012 09:48 PM, SomeDude wrote:
 And it's not the same at all as creating one branch every 
 month.

But why would you do that? The general discussion seems to have been around the idea of 1 or 2 stable releases per year, 1 every 3 months max.

This is probably too slow for revision and too fast for new versions. This is yet another point where the distro process don't apply to us : distro update a given version on a per package basis, which make absolute no sense for us.
Dec 18 2012
prev sibling next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 12/19/2012 01:00 AM, deadalnix wrote:
 This is probably too slow for revision and too fast for new versions.

I'm not counting minor-point bugfix updates as a "release" in this context, as I doubt you'd have a separate branch for those.
 This is yet another point where the distro process don't apply to us : distro
 update a given version on a per package basis, which make absolute no sense
for us.

....? I don't think this has any relation to anything that anyone has proposed.
Dec 18 2012
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 19 December 2012 at 01:08:28 UTC, Joseph Rushton 
Wakeling wrote:
 On 12/19/2012 01:00 AM, deadalnix wrote:
 This is probably too slow for revision and too fast for new 
 versions.

I'm not counting minor-point bugfix updates as a "release" in this context, as I doubt you'd have a separate branch for those.

I don't see how they aren't release. They are released. But they have branches, they are tags. Each time you package the software to put it in download somewhere for users to use, you do a release. I you want to talk about branch or versions talk about branch or versions, not release please.
Dec 18 2012
prev sibling next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 12/19/2012 02:22 AM, deadalnix wrote:
 I don't see how they aren't release. They are released. But they have branches,
 they are tags. Each time you package the software to put it in download
 somewhere for users to use, you do a release.

 I you want to talk about branch or versions talk about branch or versions, not
 release please.

OK, let me rephrase it. I doubt that it makes sense to have a separate branch for minor point releases (i.e. going from N.x.y --> N.x.y+1, a bugfix release). You'd have an N.x branch and you'd tag the .y minor point releases. I don't think this is in contradiction with Andrei's intentions.
Dec 18 2012
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 19 December 2012 at 01:38:16 UTC, Joseph Rushton 
Wakeling wrote:
 On 12/19/2012 02:22 AM, deadalnix wrote:
 I don't see how they aren't release. They are released. But 
 they have branches,
 they are tags. Each time you package the software to put it in 
 download
 somewhere for users to use, you do a release.

 I you want to talk about branch or versions talk about branch 
 or versions, not
 release please.

OK, let me rephrase it. I doubt that it makes sense to have a separate branch for minor point releases (i.e. going from N.x.y --> N.x.y+1, a bugfix release). You'd have an N.x branch and you'd tag the .y minor point releases. I don't think this is in contradiction with Andrei's intentions.

No indeed. I think that this way make sense. But I'm afraid that releasing a new version (note version, not revision) every 6 month or so will lead to an important number of version to maintain simultaneously if we want to ensure some stability. I think the smart move here is to release new feature less often, but to release more at once, and at the same time go fast on bug fixes. I frankly more worried about the fact that D has feature that don't integrate nicely with each other right now than by D not having enough features.
Dec 18 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Wednesday, 19 December 2012 at 01:48:49 UTC, deadalnix wrote:
 I think the smart move here is to release new feature less 
 often, but to release more at once, and at the same time go 
 fast on bug fixes.

I'm not a a fan of time line based releases (snapshots), it makes no sense to me at all other than for producing a testing version, and even then I'm not so sure. Why release a new stable branch every 6 mo's or whatever? Why not release a new stable branch when there's one worth releasing? Why not instead focus on releasing "revisions" (bug fixes) instead, up until there's enough of something new worth releasing as a package. New stable releases, should be major version number increments and they should not happen very often. Bug fixes should happen much more often, and if they are released as new branches, then the old ones should not get support, as it's pointless, that's what the most recent stable release branch provides. I can see us supporting a previous major release version, but not any of the minor bug fix increments that appeared along the way. Perhaps there is resistance to changing from the current "snapshot" process which tends to produce meaningless buggy/breaking releases, to one that is a "feature" based release.
 I frankly more worried about the fact that D has feature that 
 don't integrate nicely with each other right now than by D not 
 having enough features.

I sure hope everyone can agree with that. --rt
Dec 18 2012
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 19 December 2012 at 06:31:04 UTC, Rob T wrote:
 On Wednesday, 19 December 2012 at 01:48:49 UTC, deadalnix wrote:
 I think the smart move here is to release new feature less 
 often, but to release more at once, and at the same time go 
 fast on bug fixes.

I'm not a a fan of time line based releases (snapshots), it makes no sense to me at all other than for producing a testing version, and even then I'm not so sure. Why release a new stable branch every 6 mo's or whatever? Why not release a new stable branch when there's one worth releasing? Why not instead focus on releasing "revisions" (bug fixes) instead, up until there's enough of something new worth releasing as a package. New stable releases, should be major version number increments and they should not happen very often. Bug fixes should happen much more often, and if they are released as new branches, then the old ones should not get support, as it's pointless, that's what the most recent stable release branch provides. I can see us supporting a previous major release version, but not any of the minor bug fix increments that appeared along the way. Perhaps there is resistance to changing from the current "snapshot" process which tends to produce meaningless buggy/breaking releases, to one that is a "feature" based release.

I can't even say if you agree or disagree with what you quote. If we want to understand each other, it may be nice to start using common vocabulary. In the planned process exposed on the wiki, you'll find no branch called stable (for good reasons). Nobody talked about doing revisions in branches, it make no sense. Again, you should read what is written in the wiki page. Nothing is released as new branch. I don't even understand what it is supposed to mean. Thing are released as .zip/.deb/.rpm/.dmg/whatever packages. Finally, nobody is supposed to support fix, this is the other way around : fix are what you do when a supported version has a bug.
Dec 19 2012
prev sibling next sibling parent "Jesse Phillips" <Jessekphillips+D gmail.com> writes:
On Wednesday, 19 December 2012 at 06:31:04 UTC, Rob T wrote:
 Perhaps there is resistance to changing from the current 
 "snapshot" process which tends to produce meaningless 
 buggy/breaking releases, to one that is a "feature" based 
 release.

I don not see a greater correlation between snapshot releases and buggy/breaking any more than a feature based release being buggy/breaking. To facilitate feature releases a plan for what/how many features will make a release. I'm not against having these, only against requiring it for a release. Having it based on "important" or "enough" changes is subjective and the small things can be very important to someone. And even with just bugs, how many make for a good release/revision (we have way to many names that all seem to mean something different to everyone)? I will agree though, if there isn't anything worth releasing, don't release it. But my threshold for 'worth' is much lower than yours. I'd also say it has little to do with new "features" as it is about completing features and disruptive bugs. I'd think the supported/stable/lts/somethingsomething would be open to Phobos additions, but I'm not too concerned as things can be changed.
Dec 19 2012
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Dec 14, 2012 at 10:16:45AM -0500, Andrei Alexandrescu wrote:
 On 12/14/12 10:02 AM, H. S. Teoh wrote:
A number of us have put up a draft of a proposed release process on
the wiki, based on some of the things discussed in this thread.

	http://wiki.dlang.org/Release_Process

Seen it, will give a thorough read today. Thanks!

Just to follow up, any comments on what's there currently? I fear that some of us on this thread may be over-engineering the whole thing without input from the core devs who will actually be implementing this process. T -- My program has no bugs! Only undocumented features...
Dec 19 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Wednesday, 19 December 2012 at 19:01:56 UTC, Andrei 
Alexandrescu wrote:
 I was hoping for more consensus to build in this thread. Right 
 now it seems there's still quite a bit of controversy about 
 what the best way to go is.

 Andrei

We need to list out all the main points of contention and pick away at them until there's none left I suspect that some of it may be more of a misunderstanding than real contention, but no doubt a few key points are in contention. In terms of "core devs", I know about Walter and Andrei, so who else is there that has yet to give us any input? --rt
Dec 19 2012
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 19 December 2012 at 19:01:56 UTC, Andrei 
Alexandrescu wrote:
 I was hoping for more consensus to build in this thread. Right 
 now it seems there's still quite a bit of controversy about 
 what the best way to go is.

I'm afraid a lot of discussions we see right now are plain useless because overengineered. This shouldn't prevent us from switching some more basics things, like the git workflow and see how it work. This process need some actual practical use for the reflection to go forward, as it has gone way too far in the stratosphere.
Dec 19 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Wednesday, 19 December 2012 at 19:26:48 UTC, deadalnix wrote:
 On Wednesday, 19 December 2012 at 19:01:56 UTC, Andrei 
 Alexandrescu wrote:
 I was hoping for more consensus to build in this thread. Right 
 now it seems there's still quite a bit of controversy about 
 what the best way to go is.

I'm afraid a lot of discussions we see right now are plain useless because overengineered. This shouldn't prevent us from switching some more basics things, like the git workflow and see how it work. This process need some actual practical use for the reflection to go forward, as it has gone way too far in the stratosphere.

Yes, overengineering is pointless, and doing something basic is better than doing nothing at all, and knowing what will work and what won't and what the challenges are means that we have to try something out or we'll never learn anything. There are some main points that I think everyone agrees with that could be implemented immediately. Do we all agree that MASTER becomes the "Dev" branch? Do we all agree that we need a "testing" (aka staging) branch? Do we all agree that we need a "stable" branch? --rt
Dec 19 2012
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Dec 19, 2012 at 02:01:57PM -0500, Andrei Alexandrescu wrote:
 On 12/19/12 1:45 PM, H. S. Teoh wrote:
On Fri, Dec 14, 2012 at 10:16:45AM -0500, Andrei Alexandrescu wrote:
On 12/14/12 10:02 AM, H. S. Teoh wrote:
A number of us have put up a draft of a proposed release process on
the wiki, based on some of the things discussed in this thread.

	http://wiki.dlang.org/Release_Process

Seen it, will give a thorough read today. Thanks!

Just to follow up, any comments on what's there currently? I fear that some of us on this thread may be over-engineering the whole thing without input from the core devs who will actually be implementing this process.

I was hoping for more consensus to build in this thread. Right now it seems there's still quite a bit of controversy about what the best way to go is.

I think we need more input from the core devs before any further discussions make sense. Like deadalnix said, we've gone way overboard with the over-engineering; can we at least get the core devs to evaluate the most basic tenets of the proposal first? Others can correct me if I'm wrong, but the main points as I understand them are: - Continue using master for development, as before. - New features will be developed in their own dedicated branches (branched from master), NOT on master directly. When the new feature is deemed ready, by some criterion to be decided on, then it's merged into master. - At some point in the process, perhaps after a release is made from staging (see below), the new changes in master are folded into the staging branch (we can argue about the exact name of this later). - The intent of staging is to test new features and possibly breaking changes by a wider audience before the official release. It is to be used by people who want bleeding-edge features and can live with existing code breaking once in a while. - Staging only receives bugfixes and no new features until it's deemed release-ready. It may receive breaking changes if a particular feature is deemed unsuitable in its present form and needs to be revised. (Said changes are also applied to master, of course.) This is where new features are refined from their initial design to their final form. - Once staging is release-ready, a release branch is made from staging, then the latest stuff from master is folded into staging. - A release branch is used for applying critical bugfixes and showstopper fixes after a release is made. It does not receive low-priority bug fixes, and certainly no new features and no breaking changes. I deliberately left out things like version numbering and timelines, etc., as those are just bikeshedding. The exact criteria under which master is folded into staging, and staging is deemed release-ready, are intentionally left vague. We need the core devs to evaluate whether these high-level steps in the process are actually workable first, before we even think about timelines, version numberings, and merging/branching criteria. T -- All problems are easy in retrospect.
Dec 19 2012
prev sibling next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 19 December 2012 at 19:56:47 UTC, Rob T wrote:
 On Wednesday, 19 December 2012 at 19:26:48 UTC, deadalnix wrote:
 On Wednesday, 19 December 2012 at 19:01:56 UTC, Andrei 
 Alexandrescu wrote:
 I was hoping for more consensus to build in this thread. 
 Right now it seems there's still quite a bit of controversy 
 about what the best way to go is.

I'm afraid a lot of discussions we see right now are plain useless because overengineered. This shouldn't prevent us from switching some more basics things, like the git workflow and see how it work. This process need some actual practical use for the reflection to go forward, as it has gone way too far in the stratosphere.

Yes, overengineering is pointless, and doing something basic is better than doing nothing at all, and knowing what will work and what won't and what the challenges are means that we have to try something out or we'll never learn anything. There are some main points that I think everyone agrees with that could be implemented immediately. Do we all agree that MASTER becomes the "Dev" branch?

Yes
 Do we all agree that we need a "testing" (aka staging) branch?

Yes
 Do we all agree that we need a "stable" branch?

No. Stable isn't a boolean criteria. You'll find different degree of stability going from not so stable (dev version) to very stable (dead project). The wiki already mention a process with a branch per version of the software.
Dec 19 2012
next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Dec 20, 2012 at 05:48:13AM +0100, deadalnix wrote:
 On Thursday, 20 December 2012 at 04:11:00 UTC, Jesse Phillips wrote:
On Wednesday, 19 December 2012 at 23:05:59 UTC, deadalnix wrote:

master : used as a base for development. New feature are merged
here.
staging : used to provide a view of what the next version will
look like. Regular snapshot of that branch are made so public
can use the last features.
version : used to contain a version that will have a support for
an extended period of time.



From the sound of it this request is pulled into master. We
continue to pull many of these changes in. How do we decide they
should be placed into staging, when we pull them into master?. If
we wait for some 'magic time' how do we pull it into staging, does
that mean we now cherry pick commits of master?


In my mind, after a release, the contents of staging are updated to be exactly the same as master. This can be done either via a merge, or simply deleting the current staging and making a new one by branching from master.
Another issue is it sounds like master becomes a "phantom" branch.
At no point in time would master resemble what is released. I see
this as a problem because it is the branch people are developing off
of, it means nothing to test in master as staging has the actual
state that will be released.

That is very true. With the given modification master and stagging become redundant with one another. Why not get rid of stagging ?

No, staging does not receive new features except immediately after release. But it will get bugfixes that do not introduce new things. Example timeline: - time=0: we just made a release, so staging := master. - time=1: a crash bug is discovered in master. Walter checks in a fix. Fix is propagated to staging because it doesn't introduce new things, only fixes existing problems. - time=2: Walter decides to merge UDA implementation into master (for example). Staging does NOT get UDA merged in, because UDA is new. - time=3: more bugs are found in master. Fixes are checked in. Fixes get propagated to staging. - time=4: a UDA bug is fixed. Fix is NOT propagated to staging, because staging doesn't have UDA (yet). - time=5: we decide that it's time to release. So: - we tag a release on current staging - we merge master into staging, so now staging gets UDA - time=6: Walter adds HalfFloat to master. HalfFloat is NOT propagated to staging, because it's new. - time=7: users running staging find a nasty design flaw in UDA. Walter makes a (breaking) fix in master, and the fix is propagated to staging, because UDA is already in staging. - time=8: Walter finds bugs in HalfFloat. Fixes are checked into master, but NOT staging. - time=9: Walter adds more new features to master. These are NOT propagated to staging, because they're new. - time=10: it is decided that UDA design is stable now, and we're ready for release. So: - we tag a new release on current staging - we merge master into staging, now staging gets HalfFloat. - etc. Hopefully this makes it clearer what we're trying to achieve. T -- Arise, you prisoners of Windows Arise, you slaves of Redmond, Wash, The day and hour soon are coming When all the IT folks say "Gosh!" It isn't from a clever lawsuit That Windowsland will finally fall, But thousands writing open source code Like mice who nibble through a wall. -- The Linux-nationale by Greg Baker
Dec 19 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Thursday, 20 December 2012 at 05:32:30 UTC, H. S. Teoh wrote:
 On Thu, Dec 20, 2012 at 05:48:13AM +0100, deadalnix wrote:
 On Thursday, 20 December 2012 at 04:11:00 UTC, Jesse Phillips

updated to be exactly the same as master. This can be done either via a merge, or simply deleting the current staging and making a new one by branching from master.

Yes, exactly!
 No, staging does not receive new features except immediately 
 after
 release. But it will get bugfixes that do not introduce new 
 things.

 Example timeline:
 - time=0: we just made a release, so staging := master.

 - time=1: a crash bug is discovered in master. Walter checks in 
 a fix.
   Fix is propagated to staging because it doesn't introduce new 
 things,
   only fixes existing problems.

 - time=2: Walter decides to merge UDA implementation into 
 master (for
   example). Staging does NOT get UDA merged in, because UDA is 
 new.

 - time=3: more bugs are found in master. Fixes are checked in. 
 Fixes get
   propagated to staging.

 - time=4: a UDA bug is fixed. Fix is NOT propagated to staging, 
 because
   staging doesn't have UDA (yet).

 - time=5: we decide that it's time to release. So:
    - we tag a release on current staging
    - we merge master into staging, so now staging gets UDA

 - time=6: Walter adds HalfFloat to master. HalfFloat is NOT 
 propagated
   to staging, because it's new.

 - time=7: users running staging find a nasty design flaw in 
 UDA. Walter
   makes a (breaking) fix in master, and the fix is propagated to
   staging, because UDA is already in staging.

 - time=8: Walter finds bugs in HalfFloat. Fixes are checked 
 into master,
   but NOT staging.

 - time=9: Walter adds more new features to master. These are NOT
   propagated to staging, because they're new.

 - time=10: it is decided that UDA design is stable now, and 
 we're ready
   for release. So:
    - we tag a new release on current staging
    - we merge master into staging, now staging gets HalfFloat.

 - etc.

 Hopefully this makes it clearer what we're trying to achieve.

Yes, fully agreeable with me!
 T

Thanks for writing that out so well. Excellent! --rt
Dec 19 2012
prev sibling parent "Jesse Phillips" <Jessekphillips+D gmail.com> writes:
On Thursday, 20 December 2012 at 05:32:30 UTC, H. S. Teoh wrote:
On Wednesday, 19 December 2012 at 23:05:59 UTC, deadalnix 
wrote:

master : used as a base for development. New feature are 
merged
here.
staging : used to provide a view of what the next version 
will
look like. Regular snapshot of that branch are made so public
can use the last features.
version : used to contain a version that will have a support 
for
an extended period of time.



In my mind, after a release, the contents of staging are updated to be exactly the same as master. This can be done either via a merge, or simply deleting the current staging and making a new one by branching from master.

I can get behind this. deadalnix, can we please go back to calling "version" as "release" that is what it is and what everyone else uses I don't see a need make a new name because you see "release" as something different.
Dec 20 2012
prev sibling next sibling parent "foobar" <foo bar.com> writes:
On Wednesday, 19 December 2012 at 20:51:57 UTC, deadalnix wrote:
 On Wednesday, 19 December 2012 at 19:56:47 UTC, Rob T wrote:

 Do we all agree that we need a "stable" branch?

No. Stable isn't a boolean criteria. You'll find different degree of stability going from not so stable (dev version) to very stable (dead project). The wiki already mention a process with a branch per version of the software.

Let's generalize this point for the sake of reaching consensus - we need _at least one_ "stable" branch which is separate from "staging". We are still conflicted as to what should be the maximum amount. For the record, I'm with the camp advocating at most a fixed amount countable on one hand. That's an O(1) with a very small constant as opposed to the O(n) suggestion by Andrei. I hope Andrei appreciates the order of efficiency here.
Dec 19 2012
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 19 December 2012 at 21:30:44 UTC, Andrei 
Alexandrescu wrote:
 On 12/19/12 4:23 PM, foobar wrote:
 On Wednesday, 19 December 2012 at 20:51:57 UTC, deadalnix 
 wrote:
 On Wednesday, 19 December 2012 at 19:56:47 UTC, Rob T wrote:

 Do we all agree that we need a "stable" branch?

No. Stable isn't a boolean criteria. You'll find different degree of stability going from not so stable (dev version) to very stable (dead project). The wiki already mention a process with a branch per version of the software.

Let's generalize this point for the sake of reaching consensus - we need _at least one_ "stable" branch which is separate from "staging". We are still conflicted as to what should be the maximum amount. For the record, I'm with the camp advocating at most a fixed amount countable on one hand. That's an O(1) with a very small constant as opposed to the O(n) suggestion by Andrei. I hope Andrei appreciates the order of efficiency here.

I agree with one "stable" branch.

This does conflict with the requirement you gave before about being able to support anything, as previous stable version cannot be revised. Or does stable here mean supported ? (which means we still have branch per version, but only one version is supported)
Dec 19 2012
prev sibling next sibling parent "foobar" <foo bar.com> writes:
On Wednesday, 19 December 2012 at 21:30:44 UTC, Andrei 
Alexandrescu wrote:
 On 12/19/12 4:23 PM, foobar wrote:
 On Wednesday, 19 December 2012 at 20:51:57 UTC, deadalnix 
 wrote:
 On Wednesday, 19 December 2012 at 19:56:47 UTC, Rob T wrote:

 Do we all agree that we need a "stable" branch?

No. Stable isn't a boolean criteria. You'll find different degree of stability going from not so stable (dev version) to very stable (dead project). The wiki already mention a process with a branch per version of the software.

Let's generalize this point for the sake of reaching consensus - we need _at least one_ "stable" branch which is separate from "staging". We are still conflicted as to what should be the maximum amount. For the record, I'm with the camp advocating at most a fixed amount countable on one hand. That's an O(1) with a very small constant as opposed to the O(n) suggestion by Andrei. I hope Andrei appreciates the order of efficiency here.

I agree with one "stable" branch. Andrei

That's great! What about the other high level points summarized eloquently by H. S. Teoh? Any other high level points or goals the process should incorporate? Personally, I think the whole pre-development stage needs to get looks at - Specifically having some sort of at least high-level *binding* planning - road map, mile stones, todo lists. This should give more weight to DIPs. DIPs at the moment have no weight whatsoever. The attributes showed all the major flows in the decision making process: 1. Feature idea comes up in discussion. 2. Feature was discussed heavily by the community reaching some design consensus. 3. Plan is abandoned/forgotten due to Walter's objections - not to the design but the idea itself. 4. Feature comes up again in discussion, thus returning to point 1 above in infinite loop. ** 5. After many cycles spanning years, the community accepts that the feature will never be added despite a consensus and constant demand. 5.1 Feature is added by surprise with deviations from consensus design, optionally integrates purely with other parts of the language. 5.1.1 Feature is final! Another prime example is the auto-ref feature which we are stuck with because Walter apparently understood differently from what you intended. This is highly coupled with Walter, making it very hard to agree on a high level design and delegate its implementation to other developers. As far as I see, No major feature was ever implemented by someone other than Walter and he has final say on all design decisions. I agree with Walter's motives, he wants to stabilize the language and by resisting to feature suggestions he sets a very high bar for any major change. That is a good thing. The problem is that it all happens in his head and not in plain sight. The community has no idea what his plans are, I'm not convinced that even you get all the details. That means no planning ahead and no delegation. Had we known that Walter sign off featureX but just doesn't have the time to implement it, than it still would be on the official *binding* todo list and someone else would be able to implement this feature. As the situation now stands, no one will bother doing any significant work if it has a high chance of being entirely dismissed by Walter.
Dec 19 2012
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 19 December 2012 at 21:48:22 UTC, Andrei 
Alexandrescu wrote:
 Walter needs to chime in about that. One possibility is to 
 continue using tags for marking releases, and then branch for 
 the few important releases that we want to patch.

Note that what is described on the wiki distinguish new version (that actually bring new stuff) and revision (that contains only bugfixes). Does having branches that are not used anymore is a problem ? They'll not make the repository much bigger because the data is in history anyway.
Dec 19 2012
prev sibling next sibling parent "foobar" <foo bar.com> writes:
On Wednesday, 19 December 2012 at 21:58:12 UTC, foobar wrote:
 On Wednesday, 19 December 2012 at 21:30:44 UTC, Andrei 
 Alexandrescu wrote:
 On 12/19/12 4:23 PM, foobar wrote:
 On Wednesday, 19 December 2012 at 20:51:57 UTC, deadalnix 
 wrote:
 On Wednesday, 19 December 2012 at 19:56:47 UTC, Rob T wrote:

 Do we all agree that we need a "stable" branch?

No. Stable isn't a boolean criteria. You'll find different degree of stability going from not so stable (dev version) to very stable (dead project). The wiki already mention a process with a branch per version of the software.

Let's generalize this point for the sake of reaching consensus - we need _at least one_ "stable" branch which is separate from "staging". We are still conflicted as to what should be the maximum amount. For the record, I'm with the camp advocating at most a fixed amount countable on one hand. That's an O(1) with a very small constant as opposed to the O(n) suggestion by Andrei. I hope Andrei appreciates the order of efficiency here.

I agree with one "stable" branch. Andrei

That's great! What about the other high level points summarized eloquently by H. S. Teoh? Any other high level points or goals the process should incorporate? Personally, I think the whole pre-development stage needs to get looks at - Specifically having some sort of at least high-level *binding* planning - road map, mile stones, todo lists. This should give more weight to DIPs. DIPs at the moment have no weight whatsoever. The attributes showed all the major flows in the decision making process: 1. Feature idea comes up in discussion. 2. Feature was discussed heavily by the community reaching some design consensus. 3. Plan is abandoned/forgotten due to Walter's objections - not to the design but the idea itself. 4. Feature comes up again in discussion, thus returning to point 1 above in infinite loop. ** 5. After many cycles spanning years, the community accepts that the feature will never be added despite a consensus and constant demand. 5.1 Feature is added by surprise with deviations from consensus design, optionally integrates purely with other parts of the language. 5.1.1 Feature is final! Another prime example is the auto-ref feature which we are stuck with because Walter apparently understood differently from what you intended. This is highly coupled with Walter, making it very hard to agree on a high level design and delegate its implementation to other developers. As far as I see, No major feature was ever implemented by someone other than Walter and he has final say on all design decisions. I agree with Walter's motives, he wants to stabilize the language and by resisting to feature suggestions he sets a very high bar for any major change. That is a good thing. The problem is that it all happens in his head and not in plain sight. The community has no idea what his plans are, I'm not convinced that even you get all the details. That means no planning ahead and no delegation. Had we known that Walter sign off featureX but just doesn't have the time to implement it, than it still would be on the official *binding* todo list and someone else would be able to implement this feature. As the situation now stands, no one will bother doing any significant work if it has a high chance of being entirely dismissed by Walter.

The high-level steps as I see them: 1. The community discusses the merits and costs of the suggested feature and a consensus must be reached regarding adding such a feature. 2. An initial design must be provided - also with consensus. including this parts: 2.a. Suggested semantics of the feature - describe as precisely as possible 2.b. All interactions with other language features must be carefully examined and listed. 2.c An initial syntax agreed upon by the community. 3. Feature is implemented and after reaching some agreed upon level of stability is than field tested by the public. 3.1 Based on public feedback, feature design can be refined and bugs are fixed until consensus is reached that feature is ready to be finalized and merged to the next stable release. It's vague on purpose - the exact details of how each point above is approved is to be decided later. The main principle is that approved planned features should be listed on the road map and point to their DIP with description of their design. Rejected features are also listed as DIPs with the rationale for the rejection, this is to avoid wasting time on repeat discussing them and requesting them. Ultimate goal for all this is making the planning process as transparent as possible for the community.
Dec 19 2012
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 19 December 2012 at 22:30:29 UTC, Andrei 
Alexandrescu wrote:
 On 12/19/12 5:07 PM, deadalnix wrote:
 On Wednesday, 19 December 2012 at 21:48:22 UTC, Andrei 
 Alexandrescu wrote:
 Walter needs to chime in about that. One possibility is to 
 continue
 using tags for marking releases, and then branch for the few 
 important
 releases that we want to patch.

Note that what is described on the wiki distinguish new version (that actually bring new stuff) and revision (that contains only bugfixes). Does having branches that are not used anymore is a problem ? They'll not make the repository much bigger because the data is in history anyway.

I kinda like the idea that we have tags for releases not planned for long term support and distinguished branches for those that we do. Clarifies intent a lot.

Should something be released if nobody will support it ? If it is in order to provide overview of what is coming in the next version, I guess the best option is to provide a snapshot of the staging branch on regular basis (and call it beta or whatever). Reading this answer, I start to think that we have something pretty close in mind but fail to explain it to each other.
Dec 19 2012
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 19 December 2012 at 22:30:29 UTC, Andrei 
Alexandrescu wrote:
 On 12/19/12 5:07 PM, deadalnix wrote:
 On Wednesday, 19 December 2012 at 21:48:22 UTC, Andrei 
 Alexandrescu wrote:
 Walter needs to chime in about that. One possibility is to 
 continue
 using tags for marking releases, and then branch for the few 
 important
 releases that we want to patch.

Note that what is described on the wiki distinguish new version (that actually bring new stuff) and revision (that contains only bugfixes). Does having branches that are not used anymore is a problem ? They'll not make the repository much bigger because the data is in history anyway.

I kinda like the idea that we have tags for releases not planned for long term support and distinguished branches for those that we do. Clarifies intent a lot.

Let me propose an updated version of branches w have, according to what I understand here. Note that I avoid using the term stable because I'm pretty sure it is a source of confusion and miscommunication. master : used as a base for development. New feature are merged here. staging : used to provide a view of what the next version will look like. Regular snapshot of that branch are made so public can use the last features. version : used to contain a version that will have a support for an extended period of time. If I understand correctly, version is what you call stable : a branch that is detached to provide long term support.
Dec 19 2012
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 19 December 2012 at 23:08:59 UTC, Andrei 
Alexandrescu wrote:
 My understanding is that's what many projects do. Supporting 
 each minor release would make for a ton of work.

The whole point is to not support all minor release. Usually project supports one or 2 of them, that's it. Nobody win in supporting more. For instance, python is supporting 2.7 and 3.3 . PHP support 5.3 and 5.4 . Ruby support 1.9 . Java support 1.6 and 1.7 . scala 2.8 and 2.9 . As you can see, this is very common way of doing thing in the programming language world. I propose to release snapshot of staging in order to allow for very few version branches to be created, while still alowing people to try the very last D, and so allow to support them for an extended period of time without havin to ends up in a maintenance hell.
 If it is in order to provide overview of what is coming in the 
 next
 version, I guess the best option is to provide a snapshot of 
 the staging
 branch on regular basis (and call it beta or whatever).

No, I'm not talking about previews.

Then call let's call it rolling release, release candidate, bleeding edge D, unstable, latest, whatever may fit depending on how high we raise the bar in terms of stability.
Dec 19 2012
prev sibling next sibling parent "Jesse Phillips" <Jessekphillips+D gmail.com> writes:
On Wednesday, 19 December 2012 at 23:05:59 UTC, deadalnix wrote:

 master : used as a base for development. New feature are merged 
 here.
 staging : used to provide a view of what the next version will 
 look like. Regular snapshot of that branch are made so public 
 can use the last features.
 version : used to contain a version that will have a support 
 for an extended period of time.

I do not see how development in master is moved to staging based on this description. I'll try and be more specific. I realize you don't like features, but we need to talk about Phobos additions, property mechanics, and other highly disruptive bugs. I'll call it the major-change. The major-change branch is generally developed to completion and may just be a pull requests from developer342's master. From the sound of it this request is pulled into master. We continue to pull many of these changes in. How do we decide they should be placed into staging, when we pull them into master?. If we wait for some 'magic time' how do we pull it into staging, does that mean we now cherry pick commits of master? Another issue is it sounds like master becomes a "phantom" branch. At no point in time would master resemble what is released. I see this as a problem because it is the branch people are developing off of, it means nothing to test in master as staging has the actual state that will be released.
Dec 19 2012
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Thursday, 20 December 2012 at 04:11:00 UTC, Jesse Phillips 
wrote:
 On Wednesday, 19 December 2012 at 23:05:59 UTC, deadalnix wrote:

 master : used as a base for development. New feature are 
 merged here.
 staging : used to provide a view of what the next version will 
 look like. Regular snapshot of that branch are made so public 
 can use the last features.
 version : used to contain a version that will have a support 
 for an extended period of time.

I do not see how development in master is moved to staging based on this description. I'll try and be more specific. I realize you don't like features, but we need to talk about Phobos additions, property mechanics, and other highly disruptive bugs. I'll call it the major-change. The major-change branch is generally developed to completion and may just be a pull requests from developer342's master. From the sound of it this request is pulled into master. We continue to pull many of these changes in. How do we decide they should be placed into staging, when we pull them into master?. If we wait for some 'magic time' how do we pull it into staging, does that mean we now cherry pick commits of master? Another issue is it sounds like master becomes a "phantom" branch. At no point in time would master resemble what is released. I see this as a problem because it is the branch people are developing off of, it means nothing to test in master as staging has the actual state that will be released.

That is very true. With the given modification master and stagging become redundant with one another. Why not get rid of stagging ?
Dec 19 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Thursday, 20 December 2012 at 04:11:00 UTC, Jesse Phillips 
wrote:
 From the sound of it this request is pulled into master. We 
 continue to pull many of these changes in. How do we decide 
 they should be placed into staging, when we pull them into 
 master?. If we wait for some 'magic time' how do we pull it 
 into staging, does that mean we now cherry pick commits of 
 master?

 Another issue is it sounds like master becomes a "phantom" 
 branch. At no point in time would master resemble what is 
 released. I see this as a problem because it is the branch 
 people are developing off of, it means nothing to test in 
 master as staging has the actual state that will be released.

Most of the D users are not D devs, and they don't want to be testing highly unstable features, however some of them will want to use reasonably stable new features, even though there may be some bugs and there may be some changes. The reason for using the testing branch at all, is that they need something in it that's not available in the stable branch, so they may take a smallish risk and try out the testing or staging branch (need to decide on the terms here). I placed a note in the wiki about the issue of making sure there are incentives for people to do the things you wish them to do for each major step in the process. If there is no incentives for people to test what comes out of development, few people will test it, but you need plenty of testers. The carrot is that a staging tester gets a snapshot of what is most current that is also reasonably stable, ie., there has to be enough confidence that the code is stable enough to use for real, else the risk will be too high to bother with it. I know this because I'm one of those people. I pick and choose stable vs "reasonably" stable based on my needs and the risk assessment. You need to be quick about supporting the testing branch and careful about not dumping crap into it, it's a delicate balance yes, but you *need* testers, and there's only one good way to get them, you have to dangle a carrot out and hope they take the bait, and then you have to be prepared to help them out when things go wrong. I expect very few people will take a chance on the dev branch other than the devs themselves and a few of the experienced people who know their way around the compiler, that's just the way things will be because it'll be too risky for the regular folks to bother with. The user count will always be lowest for the dev branch, more for staging but you want to encourage highest use possible which means quick support, and the highest use will be for stable, but because it's stable means it's not really being tested because it should be mostly bug free, i.e., it's being "used" as opposed to being "tested". I don't think you can afford to drop the testing branch unless you think there's no need for very many Average Joe testers, but those are the guys who will uncover a ton of unexpected bugs missed by the dev testers. --rt
Dec 19 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Wednesday, 19 December 2012 at 21:58:12 UTC, foobar wrote:
 Personally, I think the whole pre-development stage needs to 
 get looks at -
 Specifically having some sort of at least high-level *binding* 
 planning - road map, mile stones, todo lists. This should give 
 more weight to DIPs.
 DIPs at the moment have no weight whatsoever. The attributes 
 showed all the major flows in the decision making process:

 1. Feature idea comes up in discussion.
 2. Feature was discussed heavily by the community reaching some 
 design consensus.
 3. Plan is abandoned/forgotten due to Walter's objections - not 
 to the design but the idea itself.
 4. Feature comes up again in discussion, thus returning to 
 point 1 above in infinite loop.

A big fix is in order there too, it's actually what should be fixed first but from what I've seen going on in here if we can get something even basic implemented for the development and releases that will be a *huge* step forward. Once we have some kind of formalized process in place and prove to everyone how much better things are because of it, it'll be a first by the looks of things, and from that experience the other big holes will become easier to pick out and deal with. I think we have to keep it simple for now, focus on a dev, staging, release process, implement something, work out the bugs, get people used to having it, and prove the value, then we can get on with tackling the other major issues in a similar way. There's lots of room for improvement ahead, but we need to make at least one significant step forward in a successful way before we can hope to move on to the next one. --rt
Dec 19 2012
prev sibling next sibling parent "foobar" <foo bar.com> writes:
On Thursday, 20 December 2012 at 05:33:27 UTC, Rob T wrote:
 On Wednesday, 19 December 2012 at 21:58:12 UTC, foobar wrote:
 Personally, I think the whole pre-development stage needs to 
 get looks at -
 Specifically having some sort of at least high-level *binding* 
 planning - road map, mile stones, todo lists. This should give 
 more weight to DIPs.
 DIPs at the moment have no weight whatsoever. The attributes 
 showed all the major flows in the decision making process:

 1. Feature idea comes up in discussion.
 2. Feature was discussed heavily by the community reaching 
 some design consensus.
 3. Plan is abandoned/forgotten due to Walter's objections - 
 not to the design but the idea itself.
 4. Feature comes up again in discussion, thus returning to 
 point 1 above in infinite loop.

A big fix is in order there too, it's actually what should be fixed first but from what I've seen going on in here if we can get something even basic implemented for the development and releases that will be a *huge* step forward. Once we have some kind of formalized process in place and prove to everyone how much better things are because of it, it'll be a first by the looks of things, and from that experience the other big holes will become easier to pick out and deal with. I think we have to keep it simple for now, focus on a dev, staging, release process, implement something, work out the bugs, get people used to having it, and prove the value, then we can get on with tackling the other major issues in a similar way. There's lots of room for improvement ahead, but we need to make at least one significant step forward in a successful way before we can hope to move on to the next one. --rt

I see both as going hand-in-hand, otherwise we have chicken-egg problem. We need a better process to allow more developers to contribute code more easily *and* we need better planning to provide incentive for new developer to contribute code. Implementing only the the planning provides incentive but no means. Implementing only the process provides the means but not the incentive. While improving git mastery levels is beneficial of itself, it brings no benefit to the _users_ if only Walter is still the main developer. We end up wasting Walter's time on adjusting to a new system for a hypothetical non existent system of many developers, when he already has a working system for the actual existing situation of being the sole main developer. We need an *initial high-level* design for both parts. We can define and refine the details later and we probably should defer it to later and adjust it as we go along based on experience. Arguing now on the exact time span of a release just wastes mind cycles and detracts from the important bits - reaching a consensus regarding the goals of the project and defining a general workflow that achieves those goals with minimal amount of overhead.
Dec 20 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Thursday, 20 December 2012 at 08:27:22 UTC, foobar wrote:
 I see both as going hand-in-hand, otherwise we have chicken-egg 
 problem.
 We need a better process to allow more developers to contribute 
 code more easily *and* we need better planning to provide 
 incentive for new developer to contribute code.
 Implementing only the the planning provides incentive but no 
 means. Implementing only the process provides the means but not 
 the incentive.

 While improving git mastery levels is beneficial of itself, it 
 brings no benefit to the _users_ if only Walter is still the 
 main developer. We end up wasting Walter's time on adjusting to 
 a new system for a hypothetical non existent system of many 
 developers, when he already has a working system for the actual 
 existing situation of being the sole main developer.

 We need an *initial high-level* design for both parts. We can 
 define and refine the details later and we probably should 
 defer it to later and adjust it as we go along based on 
 experience.
 Arguing now on the exact time span of a release just wastes 
 mind cycles and detracts from the important bits - reaching a 
 consensus regarding the goals of the project and defining a 
 general workflow that achieves those goals with minimal amount 
 of overhead.

I think what was proposed - to fast track this one, and not over engineer - is a good idea. That way we can at least get something that's far better than before, and be able to more quickly move on to the next big problem area which I would agree is what you are mentioning. There's lack of planning and how decisions are made is very inefficient and arbitrary, and I would also say we have a big problem with the specification being poorly managed, eg, it's not even available for download, has no process for revisions and releases, nothing at all - it's worse than the existing development process which we all agree is in need of a big overhaul. Please, let's all agree to deal with that monster next, but it's a big problem to solve, and if we try to bundle it in with this one, we may not get any accomplished at all. I'd prefer the divide and conquer approach. --rt
Dec 20 2012
prev sibling next sibling parent "Joseph Cassman" <jc7919 outlook.com> writes:
On Wednesday, 19 December 2012 at 21:30:44 UTC, Andrei 
Alexandrescu wrote:
 I agree with one "stable" branch.

 Andrei

Just some food for thought. In the section about the "Branching model", the wiki currently has a staging branch in addition to the master branch. From what I understand, the idea seems to be to vet a release on staging until it is considered production level and then marked as the release. Another idea could be to keep the quality of the master branch at a high level so as to be able to branch into a release at any time, directly from master. Before feature branches are merged back into master, their quality is vetted so the quality of master is maintained. This idea seems similar to what is used for the vibe.d project (http://vibed.org/temp/branch-model-small.png). My apologies if I misunderstood their process. It looks like Xamarin has been using this process for a while and it seems to be working for them. http://tirania.org/blog/archive/2011/Oct-14.html Joseph
Dec 20 2012
prev sibling next sibling parent "sclytrack" <sclytrack simplicity.com> writes:
On Thursday, 20 December 2012 at 23:43:12 UTC, Joseph Cassman 
wrote:
 On Wednesday, 19 December 2012 at 21:30:44 UTC, Andrei 
 Alexandrescu wrote:
 I agree with one "stable" branch.

 Andrei

Just some food for thought. In the section about the "Branching model", the wiki currently has a staging branch in addition to the master branch. From what I understand, the idea seems to be to vet a release on staging until it is considered production level and then marked as the release. Another idea could be to keep the quality of the master branch at a high level so as to be able to branch into a release at any time, directly from master. Before feature branches are merged back into master, their quality is vetted so the quality of master is maintained.

I prefer this one. It's simpler, less to memorize. Staging area complicates things. I propose to go for a yearly release of the stable branches with one year support (In the beginning). So development needs to be of super high quality once a year.
 This idea seems similar to what is used for the vibe.d project 
 (http://vibed.org/temp/branch-model-small.png). My apologies if 
 I misunderstood their process.

 It looks like Xamarin has been using this process for a while 
 and it seems to be working for them. 
 http://tirania.org/blog/archive/2011/Oct-14.html

 Joseph

Dec 20 2012
prev sibling next sibling parent "Rob T" <rob ucora.com> writes:
On Thursday, 20 December 2012 at 23:43:12 UTC, Joseph Cassman 
wrote:
 Just some food for thought.

 In the section about the "Branching model", the wiki currently 
 has a staging branch in addition to the master branch. From 
 what I understand, the idea seems to be to vet a release on 
 staging until it is considered production level and then marked 
 as the release.

 Another idea could be to keep the quality of the master branch 
 at a high level so as to be able to branch into a release at 
 any time, directly from master. Before feature branches are 
 merged back into master, their quality is vetted so the quality 
 of master is maintained.

 This idea seems similar to what is used for the vibe.d project 
 (http://vibed.org/temp/branch-model-small.png). My apologies if 
 I misunderstood their process.

 It looks like Xamarin has been using this process for a while 
 and it seems to be working for them. 
 http://tirania.org/blog/archive/2011/Oct-14.html

 Joseph

Doesn't that just turn master into staging, and turn the feature branches into a diluted and distributed version of master? If there's no common development branch to work with that integrates the most current features together, then how will such a thing ever be properly tested before going into a high quality common branch? We also need the ability to stop brand new poorly tested features from making their way into a release, so at some point a common pre-release branch needs to be frozen from receiving any new features so that it can be honed into a high quality product. If you use the master branch for such a thing, then no new features can go into it, so with master frozen, what common branch is available for the devs to merge their new work into? --rt
Dec 21 2012
prev sibling parent "foobar" <foo bar.com> writes:
On Friday, 21 December 2012 at 18:34:12 UTC, Rob T wrote:
 On Thursday, 20 December 2012 at 23:43:12 UTC, Joseph Cassman 
 wrote:
 Just some food for thought.

 In the section about the "Branching model", the wiki currently 
 has a staging branch in addition to the master branch. From 
 what I understand, the idea seems to be to vet a release on 
 staging until it is considered production level and then 
 marked as the release.

 Another idea could be to keep the quality of the master branch 
 at a high level so as to be able to branch into a release at 
 any time, directly from master. Before feature branches are 
 merged back into master, their quality is vetted so the 
 quality of master is maintained.

 This idea seems similar to what is used for the vibe.d project 
 (http://vibed.org/temp/branch-model-small.png). My apologies 
 if I misunderstood their process.

 It looks like Xamarin has been using this process for a while 
 and it seems to be working for them. 
 http://tirania.org/blog/archive/2011/Oct-14.html

 Joseph

Doesn't that just turn master into staging, and turn the feature branches into a diluted and distributed version of master? If there's no common development branch to work with that integrates the most current features together, then how will such a thing ever be properly tested before going into a high quality common branch? We also need the ability to stop brand new poorly tested features from making their way into a release, so at some point a common pre-release branch needs to be frozen from receiving any new features so that it can be honed into a high quality product. If you use the master branch for such a thing, then no new features can go into it, so with master frozen, what common branch is available for the devs to merge their new work into? --rt

Precisely. I think people just don't understand the purpose of these additional branches. The point being - integration. The general flow of events should be: 1. Developer has cool idea/feature which he explores on a separate private "feature branch" 2. During development of the feature the developer can optionally collaborate with other developers. This can be done either by pulling from other developers' repositories directly or by pushing to a branch on github. Either way, this is an ad hoc "my new feature" branch. 3. First level of integration - feature is complete and is merged into official first level of integration - the "dev" branch (consensus was to use master for that) 4. Feature can than be further refined and _integration bugs_ can be fixed by the general dev team. 5. When the "dev" branch is considered stable enough by the team (exact criteria to be defined later), the changes are merged to the _2nd level of integration_ - the "staging" branch. This allows for a wider audience to test and provide real-world feedback. 6. when the 2nd level of integration is complete, the changes are stable enough to be released and the included features finalized. Since git provides each developer with their own private copy of the entire repository there is *no need* to define any official processes prior to initial integration. The developers are free to collaborate by using ad hoc branches. The only common sense recommendation I'd give (again, NOT part of the _official process_) is to use meaningful branch names if they are meant to be shared with other people.
Dec 21 2012