www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Build Master: Scheduling

reply "Tyro[17]" <ridimz yahoo.com> writes:
Greetings,

I have accepted the responsibility of preparing the builds for DMD and 
would like to engage in conversation about the way ahead.

The first concern I have is about the build cycle. Presently, it is 
nonexistent. There is no rhyme or reason regarding when releases are 
produced. The v2.065 agenda (first attempt of its kind) suggests that 
the next release will occur sometime in March 2014. I'm of the opinion, 
however, that the cycle should be six months long. This particular 
schedule is not of my own crafting but I believe it to be sound and 
worthy of emulation:

Schedule
--------

2.064 --- 2.064.1 --- 2.064.2 -- ...
   \
    + -- 2.065beta1 --- 2.065beta2 --- ... --- 2.065rc1 --- 2.065 --- 
2.065.1 --- ...
                                                             \
                                                               + -- 
2.066beta1 --- ...

Starting from the date of a major release (November 5th):

At four-week intervals we make a new beta release. There will be no 
separate release candidate, but if a serious problem is discovered we 
may do the next beta ahead of schedule or make a point release. There 
will be about five or six releases in that series.

In parallel with this, bugs targeted to the previous major release are 
merged into its branch. We will make bugfix releases from that branch as 
appropriate to the accumulation of changes, perhaps monthly, perhaps 
more often if there are serious bugs, perhaps much less often if no new 
changes have landed.

We will then make a release candidate for the next major release, and at 
this point create a release branch for it. We will iterate release 
candidates at approximately weekly intervals until there are no bugs 
blocking the final major release.

Major releases (2.0)
     The big ones, every six months, intended to ship in distributions 
and to be used by stability-oriented users.

Release Candidates (2.065rc1)
     Created one to two weeks before the release branch is created, this 
is a preview of the major release. There should be very few, if any 
changes from the release candidate to the major release.
Bugfix releases (2.064.1)
     Based on the previous major release or bugfix; contains only 
bugfixes and perhaps documentation corrections.

Beta release (2.065beta1)
     Created monthly from master, except for those months in which we 
create a major release. Stable and suitable for users who want the 
latest code and can live with changes from month to month.

Your thoughts and concerns please.

-- 

Andrew Edwards
--------------------
http://www.akeron.co
auto getAddress() {
     string location = " ", period = ".";
     return ("info" ~ location ~ "afidem" ~ period ~ "org");
}
Nov 13 2013
next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Thursday, 14 November 2013 at 00:37:38 UTC, Tyro[17] wrote:
 Greetings,

 I have accepted the responsibility of preparing the builds for 
 DMD and would like to engage in conversation about the way 
 ahead.

 The first concern I have is about the build cycle. Presently, 
 it is nonexistent. There is no rhyme or reason regarding when 
 releases are produced. The v2.065 agenda (first attempt of its 
 kind) suggests that the next release will occur sometime in 
 March 2014. I'm of the opinion, however, that the cycle should 
 be six months long. This particular schedule is not of my own 
 crafting but I believe it to be sound and worthy of emulation:

 Schedule
 --------

 2.064 --- 2.064.1 --- 2.064.2 -- ...
   \
    + -- 2.065beta1 --- 2.065beta2 --- ... --- 2.065rc1 --- 
 2.065 --- 2.065.1 --- ...
                                                             \
                                                               + 
 -- 2.066beta1 --- ...

 Starting from the date of a major release (November 5th):

 At four-week intervals we make a new beta release. There will 
 be no separate release candidate, but if a serious problem is 
 discovered we may do the next beta ahead of schedule or make a 
 point release. There will be about five or six releases in that 
 series.

 In parallel with this, bugs targeted to the previous major 
 release are merged into its branch. We will make bugfix 
 releases from that branch as appropriate to the accumulation of 
 changes, perhaps monthly, perhaps more often if there are 
 serious bugs, perhaps much less often if no new changes have 
 landed.

 We will then make a release candidate for the next major 
 release, and at this point create a release branch for it. We 
 will iterate release candidates at approximately weekly 
 intervals until there are no bugs blocking the final major 
 release.

 Major releases (2.0)
     The big ones, every six months, intended to ship in 
 distributions and to be used by stability-oriented users.

 Release Candidates (2.065rc1)
     Created one to two weeks before the release branch is 
 created, this is a preview of the major release. There should 
 be very few, if any changes from the release candidate to the 
 major release.
 Bugfix releases (2.064.1)
     Based on the previous major release or bugfix; contains 
 only bugfixes and perhaps documentation corrections.

 Beta release (2.065beta1)
     Created monthly from master, except for those months in 
 which we create a major release. Stable and suitable for users 
 who want the latest code and can live with changes from month 
 to month.

 Your thoughts and concerns please.
Awesome. That is perfect.
Nov 13 2013
prev sibling next sibling parent reply Brad Roberts <braddr puremagic.com> writes:
On 11/13/13 4:37 PM, Tyro[17] wrote:
 I'm of the opinion, however, that
 the cycle should be six months long. This particular schedule is not of my own
crafting but I
 believe it to be sound and worthy of emulation:
I think 6 months between releases is entirely too long. I'd really like us to be back closer to the once every month or two rather than only twice a year. The pace of change is high and increasing (which is a good thing). Release early and often yields a smoother rate of introducing those changes to the non-bleeding-edge part of the community. The larger the set of changes landing in a release the more likely it is to be a painful, breaking, experience.
 Schedule
 --------

 2.064 --- 2.064.1 --- 2.064.2 -- ...
    \
     + -- 2.065beta1 --- 2.065beta2 --- ... --- 2.065rc1 --- 2.065 --- 2.065.1
--- ...
                                                              \
                                                                + -- 2.066beta1
--- ...
As drawn, this looks a lot like a branch structure rather than release timing. I dearly hope this isn't intended to represent branching. My few cents, Brad
Nov 13 2013
next sibling parent reply "Brad Anderson" <eco gnuk.net> writes:
On Thursday, 14 November 2013 at 02:47:15 UTC, Brad Roberts wrote:
 I think 6 months between releases is entirely too long.  I'd 
 really like us to be back closer to the once every month or two 
 rather than only twice a year.  The pace of change is high and 
 increasing (which is a good thing).  Release early and often 
 yields a smoother rate of introducing those changes to the 
 non-bleeding-edge part of the community.  The larger the set of 
 changes landing in a release the more likely it is to be a 
 painful, breaking, experience.
I agree. There's really no reason not to release frequently as long as the release process isn't overly burdensome on the Release Czar (Walter's term, not mine :P). Git's cheap and powerful branching means it's easy to work on changes of nearly any size and still have a rapid release cycle. 6 months between releases means a regression that was introduced in the latest version requires you to wait another 6 months for the fix which means you are running a version that is a year out of date.
Nov 13 2013
next sibling parent "Craig Dillabaugh" <cdillaba cg.scs.carleton.ca> writes:
On Thursday, 14 November 2013 at 03:03:14 UTC, Brad Anderson
wrote:
 On Thursday, 14 November 2013 at 02:47:15 UTC, Brad Roberts 
 wrote:
clip
 6 months between releases means a regression that was 
 introduced in the latest version requires you to wait another 6 
 months for the fix which means you are running a version that 
 is a year out of date.
Don't the point releases on the main branch effectively deal with this issue? Craig
Nov 13 2013
prev sibling next sibling parent "Tyro[17]" <ridimz yahoo.com> writes:
On 11/13/13, 10:03 PM, Brad Anderson wrote:
 On Thursday, 14 November 2013 at 02:47:15 UTC, Brad Roberts wrote:
 I think 6 months between releases is entirely too long.  I'd really
 like us to be back closer to the once every month or two rather than
 only twice a year.  The pace of change is high and increasing (which
 is a good thing).  Release early and often yields a smoother rate of
 introducing those changes to the non-bleeding-edge part of the
 community.  The larger the set of changes landing in a release the
 more likely it is to be a painful, breaking, experience.
I agree. There's really no reason not to release frequently as long as the release process isn't overly burdensome on the Release Czar (Walter's term, not mine :P). Git's cheap and powerful branching means it's easy to work on changes of nearly any size and still have a rapid release cycle. 6 months between releases means a regression that was introduced in the latest version requires you to wait another 6 months for the fix which means you are running a version that is a year out of date.
Not so. The very reason for bugfix releases is to ensure that critical fixes are provided as quickly as feasible. On the other hand you will have six months to prod and poke at the betas to ensure that the impending release is void of bugs. And with a monthly release of betas, the opportunity to test out new features and fixes within, at most, one month after their implementation. Again fixes can always be pushed out as bugfix releases on as frequent or infrequent a term as the situation warrants. -- Andrew Edwards -------------------- http://www.akeron.co auto getAddress() { string location = " ", period = "."; return ("info" ~ location ~ "afidem" ~ period ~ "org"); }
Nov 13 2013
prev sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 11/14/13, Brad Anderson <eco gnuk.net> wrote:
 6 months between releases means a regression that was introduced
 in the latest version requires you to wait another 6 months for
 the fix which means you are running a version that is a year out
 of date.
6 months is ridiculously long. The changelog itself will have to span pages. And because a lot of people do not use DMD-head we'll end up with a ton of regressions that are only caught after a release is made. And people who want an important fix will have to wait 6 months for a release. New library features or modules will only be properly tested after a release, so that means potentially waiting 6 months with very little feedback. IMO 6 months is unacceptably long. We're not steering an oil rig here, D is supposed to be a speedboat.
Nov 13 2013
parent reply "Tyro[17]" <ridimz yahoo.com> writes:
On 11/13/13, 11:30 PM, Andrej Mitrovic wrote:
 On 11/14/13, Brad Anderson <eco gnuk.net> wrote:
 6 months between releases means a regression that was introduced
 in the latest version requires you to wait another 6 months for
 the fix which means you are running a version that is a year out
 of date.
6 months is ridiculously long. The changelog itself will have to span pages. And because a lot of people do not use DMD-head we'll end up with a ton of regressions that are only caught after a release is made. And people who want an important fix will have to wait 6 months for a release. New library features or modules will only be properly tested after a release, so that means potentially waiting 6 months with very little feedback. IMO 6 months is unacceptably long. We're not steering an oil rig here, D is supposed to be a speedboat.
It's been approximately six months since the release of 2.063 (alright five+: May 28 to Nov 5). I don't think too many of us lost sleep over that. There is nothing ridiculously long about six months. I doubt your change log would be much longer because of time elapsed. Rather, it would be longer because more people had time to work with the betas and discover the problems contained therein and subsequently got them fixed. What I am proposing is that you get a package every month. That should be enough time to ferry out any regression that may crop up. Use the betas on a monthly basis and you get to ride the bullet train. -- Andrew Edwards -------------------- http://www.akeron.co auto getAddress() { string location = " ", period = "."; return ("info" ~ location ~ "afidem" ~ period ~ "org"); }
Nov 13 2013
next sibling parent "growler" <growlercab gmail.com> writes:
On Thursday, 14 November 2013 at 05:05:39 UTC, Tyro[17] wrote:
 On 11/13/13, 11:30 PM, Andrej Mitrovic wrote:
 On 11/14/13, Brad Anderson <eco gnuk.net> wrote:
 6 months between releases means a regression that was 
 introduced
 in the latest version requires you to wait another 6 months 
 for
 the fix which means you are running a version that is a year 
 out
 of date.
6 months is ridiculously long. The changelog itself will have to span pages. And because a lot of people do not use DMD-head we'll end up with a ton of regressions that are only caught after a release is made. And people who want an important fix will have to wait 6 months for a release. New library features or modules will only be properly tested after a release, so that means potentially waiting 6 months with very little feedback. IMO 6 months is unacceptably long. We're not steering an oil rig here, D is supposed to be a speedboat.
It's been approximately six months since the release of 2.063 (alright five+: May 28 to Nov 5). I don't think too many of us lost sleep over that. There is nothing ridiculously long about six months. I doubt your change log would be much longer because of time elapsed. Rather, it would be longer because more people had time to work with the betas and discover the problems contained therein and subsequently got them fixed. What I am proposing is that you get a package every month. That should be enough time to ferry out any regression that may crop up. Use the betas on a monthly basis and you get to ride the bullet train.
Thanks for taking this on. Monthly beta releases will be great. Please add them to the main download page as they come out.
Nov 13 2013
prev sibling next sibling parent reply =?ISO-8859-1?Q?S=F6nke_Ludwig?= <sludwig outerproduct.org> writes:
Am 14.11.2013 06:05, schrieb Tyro[17]:
 On 11/13/13, 11:30 PM, Andrej Mitrovic wrote:
 On 11/14/13, Brad Anderson <eco gnuk.net> wrote:
 6 months between releases means a regression that was introduced
 in the latest version requires you to wait another 6 months for
 the fix which means you are running a version that is a year out
 of date.
6 months is ridiculously long. The changelog itself will have to span pages. And because a lot of people do not use DMD-head we'll end up with a ton of regressions that are only caught after a release is made. And people who want an important fix will have to wait 6 months for a release. New library features or modules will only be properly tested after a release, so that means potentially waiting 6 months with very little feedback. IMO 6 months is unacceptably long. We're not steering an oil rig here, D is supposed to be a speedboat.
It's been approximately six months since the release of 2.063 (alright five+: May 28 to Nov 5). I don't think too many of us lost sleep over that. There is nothing ridiculously long about six months.
Just a little personal impression - it seems like during 2.064's development, more people than ever have switched to DMD HEAD instead of the last release version. This made it much more burdensome to support public libraries, because compiler induced breakage was much more frequent. So considering the *current* way releases are done, I'd be very much in favor of a shorter release schedule, maybe 3 months (remember that there were times where the cycle was about 1 month), so that there is less pressure to go GIT HEAD. However, regular point releases and betas would arguably achieve this, too, so if the proposed plan is actually really followed like that, I'm all for it.
Nov 14 2013
parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 11/14/13, S=F6nke Ludwig <sludwig outerproduct.org> wrote:
 Just a little personal impression - it seems like during 2.064's
 development, more people than ever have switched to DMD HEAD instead of
 the last release version. This made it much more burdensome to support
 public libraries, because compiler induced breakage was much more frequen=
t. And a 6 month release schedule means you have to keep around and implement workarounds in your libraries for old bugs. You can't just target git-head for your users, you have to target the latest stable version. The release cycle right now is so slow that most of the time I'm just using git-head, but I still end up having to implement workarounds for the last stable version when I publish code. If the release cycles were more frequent then I wouldn't have to use bleeding-edge.
Nov 14 2013
prev sibling parent "Wyatt" <wyatt.epp gmail.com> writes:
On Thursday, 14 November 2013 at 05:05:39 UTC, Tyro[17] wrote:
 It's been approximately six months since the release of 2.063 
 (alright five+: May 28 to Nov 5). I don't think too many of us 
 lost sleep over that. There is nothing ridiculously long about 
 six months.
I'm not using HEAD either and its annoying having to wait for updates with useful fixes and changes. Also, longer cycles means more breakage for people who run the "stable" release.
 What I am proposing is that you get a package every month. That 
 should be enough time to ferry out any regression that may crop 
 up. Use the betas on a monthly basis and you get to ride the 
 bullet train.
The problem here is the beta becomes the de facto stable because people want to be able to use the new things that the documentation says they can and the breakage is amortised over months, rather than coming in a huge spike every half year. This really needs to be avoided. If you look at the movement of active projects in the past few years, most seem to be settling on shorter release cycles for similar reasons. -Wyatt
Nov 14 2013
prev sibling parent reply "Tyro[17]" <ridimz yahoo.com> writes:
On 11/13/13, 9:46 PM, Brad Roberts wrote:
 On 11/13/13 4:37 PM, Tyro[17] wrote:
 I'm of the opinion, however, that
 the cycle should be six months long. This particular schedule is not
 of my own crafting but I
 believe it to be sound and worthy of emulation:
I think 6 months between releases is entirely too long. I'd really like us to be back closer to the once every month or two rather than only twice a year. The pace of change is high and increasing (which is a good thing). Release early and often yields a smoother rate of introducing those changes to the non-bleeding-edge part of the community. The larger the set of changes landing in a release the more likely it is to be a painful, breaking, experience.
Surely for those of us that live on the edge, it is fun to be able to use the latest and greatest. Hence the reason for monthly release of betas. Within a month (sometimes shorter) of any new feature being implemented in the language, you'll be able to download the binaries for your favorite distro and begin testing it. The side effect is that there is more time to flesh out a particular implementation and get it corrected prior to it being an irreversible eyesore in the language. You also have a greater play in the filing bug reports as to aid in minimizing the number of bugs that make it into the final release. Unlike us adventurers however, companies require a more stable environment to work in. As such, the six month cycle provides a dependable time frame in which they can expect to see only bugfixes in to the current release in use. I think this release plan is a reasonable compromise for both parties.
 Schedule
 --------

 2.064 --- 2.064.1 --- 2.064.2 -- ...
    \
     + -- 2.065beta1 --- 2.065beta2 --- ... --- 2.065rc1 --- 2.065 ---
 2.065.1 --- ...
                                                              \
                                                                + --
 2.066beta1 --- ...
As drawn, this looks a lot like a branch structure rather than release timing. I dearly hope this isn't intended to represent branching.
This is in no way intended to represent branching. Simply to indicate that while betas are being released for the upcoming release, it may at times be necessary to release bug fixes.
 My few cents,
 Brad
-- Andrew Edwards -------------------- http://www.akeron.co auto getAddress() { string location = " ", period = "."; return ("info" ~ location ~ "afidem" ~ period ~ "org"); }
Nov 13 2013
parent reply Brad Roberts <braddr puremagic.com> writes:
On 11/13/13 7:13 PM, Tyro[17] wrote:
 On 11/13/13, 9:46 PM, Brad Roberts wrote:
 On 11/13/13 4:37 PM, Tyro[17] wrote:
 I'm of the opinion, however, that
 the cycle should be six months long. This particular schedule is not
 of my own crafting but I
 believe it to be sound and worthy of emulation:
I think 6 months between releases is entirely too long. I'd really like us to be back closer to the once every month or two rather than only twice a year. The pace of change is high and increasing (which is a good thing). Release early and often yields a smoother rate of introducing those changes to the non-bleeding-edge part of the community. The larger the set of changes landing in a release the more likely it is to be a painful, breaking, experience.
Surely for those of us that live on the edge, it is fun to be able to use the latest and greatest. Hence the reason for monthly release of betas. Within a month (sometimes shorter) of any new feature being implemented in the language, you'll be able to download the binaries for your favorite distro and begin testing it. The side effect is that there is more time to flesh out a particular implementation and get it corrected prior to it being an irreversible eyesore in the language. You also have a greater play in the filing bug reports as to aid in minimizing the number of bugs that make it into the final release. Unlike us adventurers however, companies require a more stable environment to work in. As such, the six month cycle provides a dependable time frame in which they can expect to see only bugfixes in to the current release in use. I think this release plan is a reasonable compromise for both parties.
Companies that don't want frequent changes can just skip releases, using whatever update frequency meets their desires. Companies do this already all the time. That only issue there is how long we continue to backport fixes into past releases. So far we've done very minimal backporting.
Nov 13 2013
parent reply "Tyro[17]" <ridimz yahoo.com> writes:
On 11/13/13, 11:06 PM, Brad Roberts wrote:
 On 11/13/13 7:13 PM, Tyro[17] wrote:
 On 11/13/13, 9:46 PM, Brad Roberts wrote:
 On 11/13/13 4:37 PM, Tyro[17] wrote:
 I'm of the opinion, however, that
 the cycle should be six months long. This particular schedule is not
 of my own crafting but I
 believe it to be sound and worthy of emulation:
I think 6 months between releases is entirely too long. I'd really like us to be back closer to the once every month or two rather than only twice a year. The pace of change is high and increasing (which is a good thing). Release early and often yields a smoother rate of introducing those changes to the non-bleeding-edge part of the community. The larger the set of changes landing in a release the more likely it is to be a painful, breaking, experience.
Surely for those of us that live on the edge, it is fun to be able to use the latest and greatest. Hence the reason for monthly release of betas. Within a month (sometimes shorter) of any new feature being implemented in the language, you'll be able to download the binaries for your favorite distro and begin testing it. The side effect is that there is more time to flesh out a particular implementation and get it corrected prior to it being an irreversible eyesore in the language. You also have a greater play in the filing bug reports as to aid in minimizing the number of bugs that make it into the final release. Unlike us adventurers however, companies require a more stable environment to work in. As such, the six month cycle provides a dependable time frame in which they can expect to see only bugfixes in to the current release in use. I think this release plan is a reasonable compromise for both parties.
Companies that don't want frequent changes can just skip releases, using whatever update frequency meets their desires. Companies do this already all the time. That only issue there is how long we continue to backport fixes into past releases. So far we've done very minimal backporting.
And what I am proposing is that we start backporting to stable releases and with subsequent bugfix releases. I'm also suggesting that for people interested in a more frequent release will have at least five, if not more, such releases (betas) prior to the official release. Live on the edge... use the beta. That's what we do now. At the moment there's nothing that make dmd.2.064.2 any more bug free than its previously released counterparts. Very few people participated in the review of the betas which were released arbitrarily (when the time seemed right). We simply call on of those betas dmd.2.064.2 and moved on. It still has a slew of bugs and more are discovered daily as people start to finally use the so called "release". I'm saying we are go about it a little different. We get more people involved in the testing process by providing more frequent release of betas and getting much of the bugs identified fixed before designating a release. To me you get what your are after a faster turnaround on fixes (monthly). And the broader customer base gets a more stable product with less bugs. -- Andrew Edwards -------------------- http://www.akeron.co auto getAddress() { string location = " ", period = "."; return ("info" ~ location ~ "afidem" ~ period ~ "org"); }
Nov 13 2013
parent reply luka8088 <luka8088 owave.net> writes:
On 14.11.2013. 5:29, Tyro[17] wrote:
 On 11/13/13, 11:06 PM, Brad Roberts wrote:
 On 11/13/13 7:13 PM, Tyro[17] wrote:
 On 11/13/13, 9:46 PM, Brad Roberts wrote:
 On 11/13/13 4:37 PM, Tyro[17] wrote:
 I'm of the opinion, however, that
 the cycle should be six months long. This particular schedule is not
 of my own crafting but I
 believe it to be sound and worthy of emulation:
I think 6 months between releases is entirely too long. I'd really like us to be back closer to the once every month or two rather than only twice a year. The pace of change is high and increasing (which is a good thing). Release early and often yields a smoother rate of introducing those changes to the non-bleeding-edge part of the community. The larger the set of changes landing in a release the more likely it is to be a painful, breaking, experience.
Surely for those of us that live on the edge, it is fun to be able to use the latest and greatest. Hence the reason for monthly release of betas. Within a month (sometimes shorter) of any new feature being implemented in the language, you'll be able to download the binaries for your favorite distro and begin testing it. The side effect is that there is more time to flesh out a particular implementation and get it corrected prior to it being an irreversible eyesore in the language. You also have a greater play in the filing bug reports as to aid in minimizing the number of bugs that make it into the final release. Unlike us adventurers however, companies require a more stable environment to work in. As such, the six month cycle provides a dependable time frame in which they can expect to see only bugfixes in to the current release in use. I think this release plan is a reasonable compromise for both parties.
Companies that don't want frequent changes can just skip releases, using whatever update frequency meets their desires. Companies do this already all the time. That only issue there is how long we continue to backport fixes into past releases. So far we've done very minimal backporting.
And what I am proposing is that we start backporting to stable releases and with subsequent bugfix releases. I'm also suggesting that for people interested in a more frequent release will have at least five, if not more, such releases (betas) prior to the official release. Live on the edge... use the beta. That's what we do now. At the moment there's nothing that make dmd.2.064.2 any more bug free than its previously released counterparts. Very few people participated in the review of the betas which were released arbitrarily (when the time seemed right). We simply call on of those betas dmd.2.064.2 and moved on. It still has a slew of bugs and more are discovered daily as people start to finally use the so called "release". I'm saying we are go about it a little different. We get more people involved in the testing process by providing more frequent release of betas and getting much of the bugs identified fixed before designating a release. To me you get what your are after a faster turnaround on fixes (monthly). And the broader customer base gets a more stable product with less bugs.
Just a wild thought... Maybe we can have monthly release and still keep it stable. Imagine this kind of release schedule: Month # 11 12 1 2 3 2.064 2.065 2.066 2.067 2.068 2.065rc2 2.066rc2 2.067rc2 2.068rc2 2.069rc2 2.066rc1 2.067rc1 2.068rc1 2.069rc1 2.070rc1 2.067b2 2.068b2 2.069b2 2.070b2 2.071b2 2.068b1 2.069b1 2.070b1 2.071b1 2.072b1 2.069alpha 2.070alpha 2.071alpha 2.072alpha 2.073alpha Where new features are only added to alpha release. And bug fixes are added to all releases. This way new bug fixes and new features would be released every month but there would be a 5 month delay between the time that feature A is added to the alpha release and the time feature A is propagated to the stable release. But during this period feature A would be propagated through releases and there would be plenty of opportunity to test it and clear it of bugs. I am not a fan of such delay but I don't see any other way new features could be added without higher risk of bugs. Also vote up for daily snapshots.
Nov 14 2013
next sibling parent "qznc" <qznc web.de> writes:
On Thursday, 14 November 2013 at 08:39:36 UTC, luka8088 wrote:
 Also vote up for daily snapshots.
Ack. Ultimately, I could envision a mostly automated release management: 1. Nightly builds aka alpha versions (auto) 2. Nightly builds which pass testsuite = beta versions (auto) 3. Declare major releases (manually) The concept of release candidates is merely a social concept in my view. They are basically used to start a discussion for consensus and get manual processes like Release Notes rolling. This means there is a manual step between 2 and 3, which consists of "declare beta version 42 = release candidate 2".
Nov 14 2013
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-11-14 09:39, luka8088 wrote:

 Just a wild thought...

 Maybe we can have monthly release and still keep it stable. Imagine this
 kind of release schedule:

 Month #  11          12          1           2           3

           2.064       2.065       2.066       2.067       2.068
           2.065rc2    2.066rc2    2.067rc2    2.068rc2    2.069rc2
           2.066rc1    2.067rc1    2.068rc1    2.069rc1    2.070rc1
           2.067b2     2.068b2     2.069b2     2.070b2     2.071b2
           2.068b1     2.069b1     2.070b1     2.071b1     2.072b1
           2.069alpha  2.070alpha  2.071alpha  2.072alpha  2.073alpha

 Where new features are only added to alpha release. And bug fixes are
 added to all releases.

 This way new bug fixes and new features would be released every month
 but there would be a 5 month delay between the time that feature A is
 added to the alpha release and the time feature A is propagated to the
 stable release. But during this period feature A would be propagated
 through releases and there would be plenty of opportunity to test it and
 clear it of bugs. I am not a fan of such delay but I don't see any other
 way new features could be added without higher risk of bugs.

 Also vote up for daily snapshots.
Are you saying we should have six releases going on at the same time? -- /Jacob Carlborg
Nov 14 2013
parent reply luka8088 <luka8088 owave.net> writes:
On 14.11.2013. 10:55, Jacob Carlborg wrote:
 On 2013-11-14 09:39, luka8088 wrote:
 
 Just a wild thought...

 Maybe we can have monthly release and still keep it stable. Imagine this
 kind of release schedule:

 Month #  11          12          1           2           3

           2.064       2.065       2.066       2.067       2.068
           2.065rc2    2.066rc2    2.067rc2    2.068rc2    2.069rc2
           2.066rc1    2.067rc1    2.068rc1    2.069rc1    2.070rc1
           2.067b2     2.068b2     2.069b2     2.070b2     2.071b2
           2.068b1     2.069b1     2.070b1     2.071b1     2.072b1
           2.069alpha  2.070alpha  2.071alpha  2.072alpha  2.073alpha

 Where new features are only added to alpha release. And bug fixes are
 added to all releases.

 This way new bug fixes and new features would be released every month
 but there would be a 5 month delay between the time that feature A is
 added to the alpha release and the time feature A is propagated to the
 stable release. But during this period feature A would be propagated
 through releases and there would be plenty of opportunity to test it and
 clear it of bugs. I am not a fan of such delay but I don't see any other
 way new features could be added without higher risk of bugs.

 Also vote up for daily snapshots.
Are you saying we should have six releases going on at the same time?
Yes. For example, if you have version 0.1, 0.2 and 0.3. And you find and fix a bug in 0.3 but you still wish to support backport for 0.2 and 0.1 that you indeed need to make 3 releases. 0.1.1, 0.2.1 and 0.3.1. But then again having LTS that others have mentioned seems better. So that only each nth release has 2.x.1, 2.x.2, 2.x.3. From my perspective, not separating releases with improvements + bug fixes from releases with only bug fixes is an issue. Because every new improvement implies risk of new bugs and some users just want to have one version that is as stable as possible. What do you all think about http://semver.org/ ? We use this king of versioning notation at work and it turns out to be very good.
Nov 15 2013
parent reply Jacob Carlborg <doob me.com> writes:
On 2013-11-15 10:16, luka8088 wrote:

 Yes. For example, if you have version 0.1, 0.2 and 0.3. And you find and
 fix a bug in 0.3 but you still wish to support backport for 0.2 and 0.1
 that you indeed need to make 3 releases. 0.1.1, 0.2.1 and 0.3.1.
There's a difference in still supporting old releases and working on five different releases at the same time that hasn't been released at all.
 But then again having LTS that others have mentioned seems better. So
 that only each nth release has 2.x.1, 2.x.2, 2.x.3.

  From my perspective, not separating releases with improvements + bug
 fixes from releases with only bug fixes is an issue. Because every new
 improvement implies risk of new bugs and some users just want to have
 one version that is as stable as possible.

 What do you all think about http://semver.org/ ?
 We use this king of versioning notation at work and it turns out to be
 very good.
I like it but I'm not sure who it's applied to applications. It's clear to see how it works for libraries but not for applications. I mean, what is considered an API change for an application? Changing the command line flags? -- /Jacob Carlborg
Nov 15 2013
parent reply luka8088 <luka8088 owave.net> writes:
On 15.11.2013. 11:01, Jacob Carlborg wrote:
 On 2013-11-15 10:16, luka8088 wrote:
 
 Yes. For example, if you have version 0.1, 0.2 and 0.3. And you find and
 fix a bug in 0.3 but you still wish to support backport for 0.2 and 0.1
 that you indeed need to make 3 releases. 0.1.1, 0.2.1 and 0.3.1.
There's a difference in still supporting old releases and working on five different releases at the same time that hasn't been released at all.
Yeah. I agree. Bad idea.
 
 But then again having LTS that others have mentioned seems better. So
 that only each nth release has 2.x.1, 2.x.2, 2.x.3.

  From my perspective, not separating releases with improvements + bug
 fixes from releases with only bug fixes is an issue. Because every new
 improvement implies risk of new bugs and some users just want to have
 one version that is as stable as possible.

 What do you all think about http://semver.org/ ?
 We use this king of versioning notation at work and it turns out to be
 very good.
I like it but I'm not sure who it's applied to applications. It's clear to see how it works for libraries but not for applications. I mean, what is considered an API change for an application? Changing the command line flags?
I think API change could be analog to features change (and the way they are interfaced). So the version consists of x.y.z z increments only on bug fixes. y increments when new features are added, but they are backwards compatable. Incrementing y resets z to 0. x increments when backwards incompatible change are made. Incrementing x resets y and z to 0.
Nov 15 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-11-15 11:51, luka8088 wrote:

 I think API change could be analog to features change (and the way they
 are interfaced).

 So the version consists of x.y.z

 z increments only on bug fixes.
 y increments when new features are added, but they are backwards
 compatable. Incrementing y resets z to 0.
 x increments when backwards incompatible change are made. Incrementing x
 resets y and z to 0.
The problem is that it's very easy to break code, even with bug fixes. -- /Jacob Carlborg
Nov 15 2013
prev sibling parent reply Xavier Bigand <flamaros.xavier gmail.com> writes:
Le 14/11/2013 09:39, luka8088 a écrit :
 On 14.11.2013. 5:29, Tyro[17] wrote:
 On 11/13/13, 11:06 PM, Brad Roberts wrote:
 On 11/13/13 7:13 PM, Tyro[17] wrote:
 On 11/13/13, 9:46 PM, Brad Roberts wrote:
 On 11/13/13 4:37 PM, Tyro[17] wrote:
 I'm of the opinion, however, that
 the cycle should be six months long. This particular schedule is not
 of my own crafting but I
 believe it to be sound and worthy of emulation:
I think 6 months between releases is entirely too long. I'd really like us to be back closer to the once every month or two rather than only twice a year. The pace of change is high and increasing (which is a good thing). Release early and often yields a smoother rate of introducing those changes to the non-bleeding-edge part of the community. The larger the set of changes landing in a release the more likely it is to be a painful, breaking, experience.
Surely for those of us that live on the edge, it is fun to be able to use the latest and greatest. Hence the reason for monthly release of betas. Within a month (sometimes shorter) of any new feature being implemented in the language, you'll be able to download the binaries for your favorite distro and begin testing it. The side effect is that there is more time to flesh out a particular implementation and get it corrected prior to it being an irreversible eyesore in the language. You also have a greater play in the filing bug reports as to aid in minimizing the number of bugs that make it into the final release. Unlike us adventurers however, companies require a more stable environment to work in. As such, the six month cycle provides a dependable time frame in which they can expect to see only bugfixes in to the current release in use. I think this release plan is a reasonable compromise for both parties.
Companies that don't want frequent changes can just skip releases, using whatever update frequency meets their desires. Companies do this already all the time. That only issue there is how long we continue to backport fixes into past releases. So far we've done very minimal backporting.
And what I am proposing is that we start backporting to stable releases and with subsequent bugfix releases. I'm also suggesting that for people interested in a more frequent release will have at least five, if not more, such releases (betas) prior to the official release. Live on the edge... use the beta. That's what we do now. At the moment there's nothing that make dmd.2.064.2 any more bug free than its previously released counterparts. Very few people participated in the review of the betas which were released arbitrarily (when the time seemed right). We simply call on of those betas dmd.2.064.2 and moved on. It still has a slew of bugs and more are discovered daily as people start to finally use the so called "release". I'm saying we are go about it a little different. We get more people involved in the testing process by providing more frequent release of betas and getting much of the bugs identified fixed before designating a release. To me you get what your are after a faster turnaround on fixes (monthly). And the broader customer base gets a more stable product with less bugs.
Just a wild thought... Maybe we can have monthly release and still keep it stable. Imagine this kind of release schedule: Month # 11 12 1 2 3 2.064 2.065 2.066 2.067 2.068 2.065rc2 2.066rc2 2.067rc2 2.068rc2 2.069rc2 2.066rc1 2.067rc1 2.068rc1 2.069rc1 2.070rc1 2.067b2 2.068b2 2.069b2 2.070b2 2.071b2 2.068b1 2.069b1 2.070b1 2.071b1 2.072b1 2.069alpha 2.070alpha 2.071alpha 2.072alpha 2.073alpha Where new features are only added to alpha release. And bug fixes are added to all releases. This way new bug fixes and new features would be released every month but there would be a 5 month delay between the time that feature A is added to the alpha release and the time feature A is propagated to the stable release. But during this period feature A would be propagated through releases and there would be plenty of opportunity to test it and clear it of bugs. I am not a fan of such delay but I don't see any other way new features could be added without higher risk of bugs. Also vote up for daily snapshots.
As pure D user, I am not really comfortable with having to wait 6 months between releases. I like the idea to get new small features regularly (phobos API enhancement,...), and on some release getting a long wait hudge feature like Allocators. Small releases will help to community to live with D. It's important to make noise regularly around the "product". Users will also xp more progressively in their development. There is a lot of development I am waiting for my project, I'll really be happy to see one come before the next summer : - std.logger - std.serialization - Weak Ptr - New signals/slots - RPC (Remote procedure call) - AST Macro
Nov 14 2013
next sibling parent reply luka8088 <luka8088 owave.net> writes:
On 15.11.2013. 0:22, Xavier Bigand wrote:
 Le 14/11/2013 09:39, luka8088 a écrit :
 On 14.11.2013. 5:29, Tyro[17] wrote:
 On 11/13/13, 11:06 PM, Brad Roberts wrote:
 On 11/13/13 7:13 PM, Tyro[17] wrote:
 On 11/13/13, 9:46 PM, Brad Roberts wrote:
 On 11/13/13 4:37 PM, Tyro[17] wrote:
 I'm of the opinion, however, that
 the cycle should be six months long. This particular schedule is not
 of my own crafting but I
 believe it to be sound and worthy of emulation:
I think 6 months between releases is entirely too long. I'd really like us to be back closer to the once every month or two rather than only twice a year. The pace of change is high and increasing (which is a good thing). Release early and often yields a smoother rate of introducing those changes to the non-bleeding-edge part of the community. The larger the set of changes landing in a release the more likely it is to be a painful, breaking, experience.
Surely for those of us that live on the edge, it is fun to be able to use the latest and greatest. Hence the reason for monthly release of betas. Within a month (sometimes shorter) of any new feature being implemented in the language, you'll be able to download the binaries for your favorite distro and begin testing it. The side effect is that there is more time to flesh out a particular implementation and get it corrected prior to it being an irreversible eyesore in the language. You also have a greater play in the filing bug reports as to aid in minimizing the number of bugs that make it into the final release. Unlike us adventurers however, companies require a more stable environment to work in. As such, the six month cycle provides a dependable time frame in which they can expect to see only bugfixes in to the current release in use. I think this release plan is a reasonable compromise for both parties.
Companies that don't want frequent changes can just skip releases, using whatever update frequency meets their desires. Companies do this already all the time. That only issue there is how long we continue to backport fixes into past releases. So far we've done very minimal backporting.
And what I am proposing is that we start backporting to stable releases and with subsequent bugfix releases. I'm also suggesting that for people interested in a more frequent release will have at least five, if not more, such releases (betas) prior to the official release. Live on the edge... use the beta. That's what we do now. At the moment there's nothing that make dmd.2.064.2 any more bug free than its previously released counterparts. Very few people participated in the review of the betas which were released arbitrarily (when the time seemed right). We simply call on of those betas dmd.2.064.2 and moved on. It still has a slew of bugs and more are discovered daily as people start to finally use the so called "release". I'm saying we are go about it a little different. We get more people involved in the testing process by providing more frequent release of betas and getting much of the bugs identified fixed before designating a release. To me you get what your are after a faster turnaround on fixes (monthly). And the broader customer base gets a more stable product with less bugs.
Just a wild thought... Maybe we can have monthly release and still keep it stable. Imagine this kind of release schedule: Month # 11 12 1 2 3 2.064 2.065 2.066 2.067 2.068 2.065rc2 2.066rc2 2.067rc2 2.068rc2 2.069rc2 2.066rc1 2.067rc1 2.068rc1 2.069rc1 2.070rc1 2.067b2 2.068b2 2.069b2 2.070b2 2.071b2 2.068b1 2.069b1 2.070b1 2.071b1 2.072b1 2.069alpha 2.070alpha 2.071alpha 2.072alpha 2.073alpha Where new features are only added to alpha release. And bug fixes are added to all releases. This way new bug fixes and new features would be released every month but there would be a 5 month delay between the time that feature A is added to the alpha release and the time feature A is propagated to the stable release. But during this period feature A would be propagated through releases and there would be plenty of opportunity to test it and clear it of bugs. I am not a fan of such delay but I don't see any other way new features could be added without higher risk of bugs. Also vote up for daily snapshots.
As pure D user, I am not really comfortable with having to wait 6 months between releases. I like the idea to get new small features regularly (phobos API enhancement,...), and on some release getting a long wait hudge feature like Allocators. Small releases will help to community to live with D. It's important to make noise regularly around the "product". Users will also xp more progressively in their development. There is a lot of development I am waiting for my project, I'll really be happy to see one come before the next summer : - std.logger - std.serialization - Weak Ptr - New signals/slots - RPC (Remote procedure call) - AST Macro
Yes but not having a delay between the time a new feature is implemented and the time it is released is very risky in terms of bugs. Because this new features has not been tester properly. And that is a fact. As far as I see it. Nobody is against frequent releases, but some people (including me) are against calling them stable. Because for people who depend on stability they are not stable enough. And calling them a release and not calling them beta (or something like that) implies stability. And especially if there is nothing like LTS then it implies "this is the most stable there is".
Nov 15 2013
parent "Dicebot" <public dicebot.lv> writes:
On Friday, 15 November 2013 at 09:23:27 UTC, luka8088 wrote:
 Yes but not having a delay between the time a new feature is 
 implemented
 and the time it is released is very risky in terms of bugs. 
 Because this
 new features has not been tester properly. And that is a fact.
Common misconception. We have relatively few breaking changes and bugs introduced by new features, much more common are regressions introduced by routine bug fixing (because of all hidden relations) What is bad with calling those fast releases "betas" is that they will be unusable with current approach to betas. Right now beta is just some snapshot from master which can contain any possible amount of regressions. Point of time when beta is released is point of time when people start actively fixing all those issues and that is the key property. If it becomes just another snapshot with no implies efforts to fix all regressions in relatively short term it gains us nothing over nightly master builds. Other than making actual releases delayed even longer.
Nov 15 2013
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-11-15 00:22, Xavier Bigand wrote:

 There is a lot of development I am waiting for my project, I'll really
 be happy to see one come before the next summer :
   - std.logger
   - std.serialization
   - Weak Ptr
   - New signals/slots
   - RPC (Remote procedure call)
   - AST Macro
I'm wondering if we can managed to release the compiler and Phobos and independently. -- /Jacob Carlborg
Nov 15 2013
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, November 15, 2013 11:03:37 Jacob Carlborg wrote:
 On 2013-11-15 00:22, Xavier Bigand wrote:
 There is a lot of development I am waiting for my project, I'll really
 
 be happy to see one come before the next summer :
   - std.logger
   - std.serialization
   - Weak Ptr
   - New signals/slots
   - RPC (Remote procedure call)
   - AST Macro
I'm wondering if we can managed to release the compiler and Phobos and independently.
I don't think so. It's still far too often the case that something has to be tweaked in Phobos because of a compiler change (generally for bug fixes, but that arguably doesn't matter much - it prevents separating their releases regardless). - Jonathan M Davis
Nov 15 2013
prev sibling next sibling parent "Joseph Cassman" <jc7919 outlook.com> writes:
On Thursday, 14 November 2013 at 00:37:38 UTC, Tyro[17] wrote:
 Greetings,

 I have accepted the responsibility of preparing the builds for 
 DMD and would like to engage in conversation about the way 
 ahead.
[...]
 At four-week intervals we make a new beta release. There will 
 be no separate release candidate, but if a serious problem is 
 discovered we may do the next beta ahead of schedule or make a 
 point release. There will be about five or six releases in that 
 series.
[...]
 Beta release (2.065beta1)
     Created monthly from master, except for those months in 
 which we create a major release. Stable and suitable for users 
 who want the latest code and can live with changes from month 
 to month.

 Your thoughts and concerns please.
First congratulations! Appreciate your work on this. For me, I do not really have the time to keep a local build up-to-date to get the latest changes on head. So I would really appreciate having a regular binary release that would give me the latest changes. Being able to schedule in and rely on the proposed monthly beta release would be a big help. Definitely +1 to that part of the proposal. If a daily snapshot could be created reliably then that might work. As long as I can get a regular binary to install. Thanks Joseph
Nov 13 2013
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-11-14 01:37, Tyro[17] wrote:

 Your thoughts and concerns please.
I like that you're doing this. But as others have said, I think the release schedule is too long. Iain has already been complaining several times that the releases is taking too long time. It gets hard them to merge the changes to GDC. I guess LDC has similar problems. -- /Jacob Carlborg
Nov 14 2013
next sibling parent reply "growler" <growlercab gmail.com> writes:
On Thursday, 14 November 2013 at 08:03:40 UTC, Jacob Carlborg 
wrote:
 On 2013-11-14 01:37, Tyro[17] wrote:

 Your thoughts and concerns please.
I like that you're doing this. But as others have said, I think the release schedule is too long. Iain has already been complaining several times that the releases is taking too long time. It gets hard them to merge the changes to GDC. I guess LDC has similar problems.
Would this still be a problem though if GDC and LDC align to the beta release schedule?
Nov 14 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-11-14 10:02, growler wrote:

 Would this still be a problem though if GDC and LDC align to the beta
 release schedule?
I would guess so. The problem is that the longer we wait the more code will have change, which means harder to merge with GC and LDC. Unless they continuously merge. -- /Jacob Carlborg
Nov 14 2013
prev sibling parent reply "qznc" <qznc web.de> writes:
On Thursday, 14 November 2013 at 08:03:40 UTC, Jacob Carlborg 
wrote:
 On 2013-11-14 01:37, Tyro[17] wrote:

 Your thoughts and concerns please.
I like that you're doing this. But as others have said, I think the release schedule is too long. Iain has already been complaining several times that the releases is taking too long time. It gets hard them to merge the changes to GDC. I guess LDC has similar problems.
I assume there still is the possibility of bug-fix-releases between the major releases? The 6 month cycle for *major* releases, means 6 months until a release which might break your code. Less for minor bug fix releases. The Linux kernel has an 8 week cycle, but they aim for forever-backwards-compatible.
Nov 14 2013
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, November 14, 2013 10:18:08 qznc wrote:
 The 6 month cycle for *major* releases, means 6 months until a
 release which might break your code. Less for minor bug fix
 releases.
It's fallacy to think that bug fixes are less likely to break code. In fact, based on dmd's history, I believe that bug fixes are _more_ likely to break code than new features are. - Jonathan M Davis
Nov 14 2013
prev sibling next sibling parent reply "tn" <no email.com> writes:
On Thursday, 14 November 2013 at 00:37:38 UTC, Tyro[17] wrote:
 Greetings,

 I have accepted the responsibility of preparing the builds for 
 DMD and would like to engage in conversation about the way 
 ahead.

 The first concern I have is about the build cycle.

 ... (clip) ...

 Your thoughts and concerns please.
What is wrong with the process that is described in the wiki (and that I thought you already agreed on)? See http://wiki.dlang.org/Development_and_Release_Process and specifically http://wiki.dlang.org/Development_and_Release_Process#Release_Schedule There the release interval is described to be approximately 2 months (which in my opinion is more reasonable in the current state of language developement).
Nov 14 2013
parent reply Martin Nowak <code dawg.eu> writes:
On 11/14/2013 09:18 AM, tn wrote:
 What is wrong with the process that is described in the wiki (and that I
 thought you already agreed on)? See

 http://wiki.dlang.org/Development_and_Release_Process
It's too complex.
Nov 14 2013
parent reply "Jesse Phillips" <Jesse.K.Phillips+D gmail.com> writes:
On Friday, 15 November 2013 at 00:30:09 UTC, Martin Nowak wrote:
 On 11/14/2013 09:18 AM, tn wrote:
 What is wrong with the process that is described in the wiki 
 (and that I
 thought you already agreed on)? See

 http://wiki.dlang.org/Development_and_Release_Process
It's too complex.
How? It is the same as Tyro[17] describes only with 2 month rather than 6 month releases. The page just goes into great detail about how to achieve it, which is complex.
Nov 14 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-11-15 02:33, Jesse Phillips wrote:

 How? It is the same as Tyro[17] describes only with 2 month rather than
 6 month releases. The page just goes into great detail about how to
 achieve it, which is complex.
Yeah, it has all the git commands, you don't have to read those. -- /Jacob Carlborg
Nov 14 2013
prev sibling next sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Thursday, 14 November 2013 at 00:37:38 UTC, Tyro[17] wrote:
 ...

 Your thoughts and concerns please.
Key problem here is that you call by beta something that is really not a beta. It is short term support release similar to ones we currently have with a shorter release schedule. And if it is _really_ supposed to be beta (== with some potentially unfixed regressions remaining) it is not really usable even in bleeding edge code. I have been proposing for ages to take existing http://wiki.dlang.org/Release_Process and define long-term support releases on top of it (== "normal" releases in your proposal). That way one can do releases once in 2 months to keep stuff going and mark ones as LTS once in a 6 months (with intention to backport non-breaking fixes into last 2 LTS) for users that want more stability. It fulfills same goals but does not result in broken releases / confusing naming.
Nov 14 2013
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, November 14, 2013 14:55:44 Dicebot wrote:
 On Thursday, 14 November 2013 at 00:37:38 UTC, Tyro[17] wrote:
 ...
 
 Your thoughts and concerns please.
Key problem here is that you call by beta something that is really not a beta. It is short term support release similar to ones we currently have with a shorter release schedule. And if it is _really_ supposed to be beta (== with some potentially unfixed regressions remaining) it is not really usable even in bleeding edge code. I have been proposing for ages to take existing http://wiki.dlang.org/Release_Process and define long-term support releases on top of it (== "normal" releases in your proposal). That way one can do releases once in 2 months to keep stuff going and mark ones as LTS once in a 6 months (with intention to backport non-breaking fixes into last 2 LTS) for users that want more stability. It fulfills same goals but does not result in broken releases / confusing naming.
I think that it makes some sense to continue to do releases more or less as we have (though hopefully more frequently - monthly or bi-monthly) but to have an "LTS" release every six months and then update that LTS branch and do patch releases for it whenever there's a critical bug that needs to be fixed. So, we have the LTS version which only gets critical patches, and the normal one which gets everything. But we don't try and treat the normal one as a beta or anything like that. That's a horrible idea IMHO, and will just lead to everyone using git head. However, even with the LTS release, we should be _very_ restrictive on what bugs get backported, because it's actually the bug fixes which cause the most bugs and code breakage. But I think that the idea should be that the LTS release is for those who need things to change as little as possible but still get fixes for bugs that really need them, whereas the normal releases should be what most everyone uses and that we not try and avoid having most folks use the releases which can include new features or other major changes. - Jonathan M Davis
Nov 17 2013
prev sibling next sibling parent reply "Rob T" <alanb ucora.com> writes:
On Thursday, 14 November 2013 at 00:37:38 UTC, Tyro[17] wrote:
[...]
 Your thoughts and concerns please.
Please do not mix together bugs fix releases with enhancement/new addition releases, those represent two different types of release that serve two entirely different purposes. Bug fixes should be as frequent as possible. To understand why, just try and find a good reason to artificially hold back a bug fix. As for feature enhancements and new additions to the language, delays tend to be a good thing, allowing time for fine tuning reflection and beta testing - but please take note that the current process lacks a public beta release cycle, and we can see the results each time there's been a new release, with weird version number mix ups, and embarrassing bugs that would have been detected had there been a real public beta release posted on the main download page. --rt
Nov 14 2013
parent reply "Dicebot" <public dicebot.lv> writes:
On Thursday, 14 November 2013 at 20:27:23 UTC, Rob T wrote:
 Bug fixes should be as frequent as possible. To understand why, 
 just try and find a good reason to artificially hold back a bug 
 fix.
Code breakage. DMD has good amount of accepts-invalid bugs.
Nov 14 2013
next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Thursday, 14 November 2013 at 20:30:40 UTC, Dicebot wrote:
 On Thursday, 14 November 2013 at 20:27:23 UTC, Rob T wrote:
 Bug fixes should be as frequent as possible. To understand 
 why, just try and find a good reason to artificially hold back 
 a bug fix.
Code breakage. DMD has good amount of accepts-invalid bugs.
Right now I have 15 tests failing in SDC test suite because of DMD messing up with closures. This is certainly not an option to wait 6 month that way. Accept invalid is certainly not something you can stick in patches release, but invalid codegen is. I fairly bummed by code breakage being raised as a excuse to not do anything. As a matter of fact, I have to work with my own homebrew version of DMD half of the time. One release out of 2 has introduced unacceptable regression. Now I'm ready to hear that people don't want to spend time publishing a minor version of DMD because they don't have time or whatever. But this breaking code bullshit must stop. Every single release of DMD break my code, 1 release out of 2 break it by introducing regression, not because of accept-invalid (which I'm happy to fix). We need these minor release. Anyone that have a reasonably sized D codebase knows it. In fact I'm pretty sure that anyone having a reasonably sized D codebase is using some homebrew version of DMD most of the time, as waiting 6 month is usually not an option.
Nov 14 2013
parent "Dicebot" <public dicebot.lv> writes:
On Thursday, 14 November 2013 at 20:53:05 UTC, deadalnix wrote:
 ...
Hush-hush :) I was simply answering a question "Why may someone refuse to move to minor release that contains only bug fixes". Nothing more, nothing less, no hidden implications :)
Nov 14 2013
prev sibling parent "Rob T" <alanb ucora.com> writes:
On Thursday, 14 November 2013 at 20:30:40 UTC, Dicebot wrote:
 On Thursday, 14 November 2013 at 20:27:23 UTC, Rob T wrote:
 Bug fixes should be as frequent as possible. To understand 
 why, just try and find a good reason to artificially hold back 
 a bug fix.
Code breakage. DMD has good amount of accepts-invalid bugs.
Code that is invalid but compiles is already broken by definition. These are potentially silent bugs in the code base, which are never good for anyone. I'd rather know about accepted-invalid code sooner in a development environment than later in an production environment. Furthermore, holding back on fixing accepts-invalid will allow more invalid code to sneak into the ever expanding code base, making the problem worse over time. --rt
Nov 14 2013
prev sibling next sibling parent "Xinok" <xinok live.com> writes:
On Thursday, 14 November 2013 at 00:37:38 UTC, Tyro[17] wrote:
 Greetings,

 I have accepted the responsibility of preparing the builds for 
 DMD and would like to engage in conversation about the way 
 ahead.

 The first concern I have is about the build cycle. Presently, 
 it is nonexistent. There is no rhyme or reason regarding when 
 releases are produced. The v2.065 agenda (first attempt of its 
 kind) suggests that the next release will occur sometime in 
 March 2014. I'm of the opinion, however, that the cycle should 
 be six months long. This particular schedule is not of my own 
 crafting but I believe it to be sound and worthy of emulation:

 ...

 Your thoughts and concerns please.
I think this is perfect. It's essentially what I would do. Ever since Google defined their six week release schedule for Chrome, people think quick release cycles are the next great thing. It was such an effective marketing strategy that Firefox had to jump the bandwagon. The truth is though, it doesn't cater well to large corporations. That's why Firefox also has ESR (Extended Support Release) for corporations which are older versions maintained for longer, only being updated for new bug and security fixes. I think the majority of people active on these forums are hardcore D enthusiasts so they're eager to have the latest and greatest. However, we're not hearing the voice of those on the outside. I'm willing to bet if we were to post this as news on Reddit, we'd be getting praise from those who aren't actively involved in the development of D. I do have a few ideas: I'm one who favors having different version numbers for beta and stable releases. Given the version number scheme that DMD follows, a simple numbering scheme to follow is odd- and even-numbered versions: odd-numbered versions, starting with 2.065, would be beta, and even-numbered versions, like 2.066, would be stable. I would define two periods during the release cycle: Early on, when it's okay to introduce new features and changes which may break code, and later on when we should be concentrating on testing, fixing bugs, and generally working towards a stable release. Finally, I think after every stable release, we should define an agenda, i.e. what are our goals for the next version? My current impression is that pull requests are incorporated into master whenever they're deemed ready. If we have a clear agenda, then we could collectively focus our efforts into a few areas for development.
Nov 14 2013
prev sibling next sibling parent reply "Nick" <nmsmith65 gmail.com> writes:
I have no idea what everyone is getting worked up about. My 
understanding of the plan is that there are biannual major 
releases, with bug fix releases in-between, for those 
people/companies who need stability/dependability.

In addition to that we have monthly 'beta' releases for those who 
want the latest and greatest but know that a few things may get 
changed around between the releases that break code and that 
there are likely to be more bugs.

So all the passionate D devs here get their monthly releases, and 
the people who want maximum stability get a biannual, LTS release 
along with some bug fix releases.
Nov 14 2013
next sibling parent Martin Nowak <code dawg.eu> writes:
On 11/15/2013 12:36 AM, Nick wrote:
 I have no idea what everyone is getting worked up about. My
 understanding of the plan is that there are biannual major releases,
 with bug fix releases in-between, for those people/companies who need
 stability/dependability.

 In addition to that we have monthly 'beta' releases for those who want
 the latest and greatest but know that a few things may get changed
 around between the releases that break code and that there are likely to
 be more bugs.

 So all the passionate D devs here get their monthly releases, and the
 people who want maximum stability get a biannual, LTS release along with
 some bug fix releases.
I too think it's a good plan. Many new features need a few month until they are stable. Also the 6 month nicely match our deprecation cycle. There is only one maintenance release in parallel to the master/beta branch which reduces the backporting effort. We still need automate more of the release building.
Nov 14 2013
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-11-15 00:36, Nick wrote:
 I have no idea what everyone is getting worked up about. My
 understanding of the plan is that there are biannual major releases,
 with bug fix releases in-between, for those people/companies who need
 stability/dependability.

 In addition to that we have monthly 'beta' releases for those who want
 the latest and greatest but know that a few things may get changed
 around between the releases that break code and that there are likely to
 be more bugs.

 So all the passionate D devs here get their monthly releases, and the
 people who want maximum stability get a biannual, LTS release along with
 some bug fix releases.
The opposite of an LTS release is _not_ a beta release. It's a regular release. It seems it would be better if the beta release were regular releases and the releases were LTS releases. I'm having a hard time requiring my users to use anything that is not a release (that is, a beta). -- /Jacob Carlborg
Nov 14 2013
next sibling parent reply "Martin Nowak" <code dawg.eu> writes:
 I'm having a hard time requiring my users to use anything that 
 is not a release (that is, a beta).
The point is, there has never been a really stable dmd release. Using it requires to read the mailing list just like with most beta software. We won't be able to release a stable compiler every month anytime soon.
Nov 15 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-11-15 14:06, Martin Nowak wrote:

 The point is, there has never been a really stable dmd release.
 Using it requires to read the mailing list just like with most beta
 software.
 We won't be able to release a stable compiler every month anytime soon.
True, but a beta would be even less stable. -- /Jacob Carlborg
Nov 15 2013
prev sibling next sibling parent reply "QAston" <qaston gmail.com> writes:
On Friday, 15 November 2013 at 07:50:28 UTC, Jacob Carlborg wrote:
 The opposite of an LTS release is _not_ a beta release. It's a 
 regular release. It seems it would be better if the beta 
 release were regular releases and the releases were LTS 
 releases.

 I'm having a hard time requiring my users to use anything that 
 is not a release (that is, a beta).
On the other hand, supporting build with 2 versions: latest beta and latest LTS is not a big burden imo (unless you expose bleeding edge features in the api).
Nov 15 2013
parent "eles" <eles eles.com> writes:
On Friday, 15 November 2013 at 15:25:29 UTC, QAston wrote:
 On Friday, 15 November 2013 at 07:50:28 UTC, Jacob Carlborg 
 wrote:
 On the other hand, supporting build with 2 versions: latest 
 beta and latest LTS is not a big burden imo (unless you expose 
 bleeding edge features in the api).
I like the Ubuntu release model. Translated into D would be: - a (regular) release every 2 months * supported until the next (regular) release gets out * point releases will follow every 2 weeks, with bug fixes only (better a short list of bugs well fixed, that is without regressions, than a longer list with plenty of regressions) - each 3rd (regular) release, that is every 6 months, will be a LTS release * supported until the next LTS release gets out (possibly longer) * point releases will follow every 2 weeks, with bug fixes only (unless causing serious code breakage) The first 2 (regular) releases will introduce bugfixes as well as new features, while the LTS release will (aim to) provide the new features from the above regular releases, plus bug fixes. So, for creating a LTS release, the first for months will be of features and bug fixes, while the latter 2 months will be just bug fixes (features could also appear here if judged stable enough).
Nov 15 2013
prev sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 15/11/13 08:50, Jacob Carlborg wrote:
 The opposite of an LTS release is _not_ a beta release. It's a regular release.
 It seems it would be better if the beta release were regular releases and the
 releases were LTS releases.

 I'm having a hard time requiring my users to use anything that is not a release
 (that is, a beta).
Really this is a question of semantics and what usage patterns you want to encourage. If you call something beta (or testing) you probably discourage a lot of people from using it -- that may be a good thing. As far as I can see the _real_ distinction between the 1- and 6-month releases here is less about how bug-free the code is and more about how long you can expect to wait before potentially getting hit with a breaking change.
Nov 19 2013
prev sibling next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 15/11/13 00:36, Nick wrote:
 So all the passionate D devs here get their monthly releases
I would imagine many of the passionate D devs here are already running git HEAD -- 1 month is FAR too long to wait ... :-)
Nov 19 2013
prev sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday, November 19, 2013 11:40:18 Joseph Rushton Wakeling wrote:
 On 15/11/13 00:36, Nick wrote:
 So all the passionate D devs here get their monthly releases
I would imagine many of the passionate D devs here are already running git HEAD -- 1 month is FAR too long to wait ... :-)
Well, if you're working on the code, you pretty much _have_ to be running on git HEAD - though what you use for your projects outside of working on dmd/druntime/Phobos is another matter. I usually do even my own stuff with git HEAD, but I do have one project that's sitting on 2.059, because it hasn't worked on anything since (due to an ICE IIRC) - though once I get time to finish refactoring it, I expect that that won't be a problem anymore, because it'll have been almost completely rewritten. - Jonathan M Davis
Nov 19 2013
prev sibling next sibling parent "ThreeFour" <threefour example.com> writes:
tl;dr thread.

What about release circle as debian project have.

Stable (LTS) release, for example, each year or each 3 versions 
with backports available.

Testing release, for example, each month.

Unstable (head revision), each week?

Naming:

dmd-2.065.0-2013.11.15-unstable.zip,
dmd-2.065.0-2013.11.11-testing.zip,
dmd-2.065.2.zip stable release with 2 update.

dmd-unstable.zip - latest head rev build.
dmd-testing.zip - latest testing/feature build.
dmd-stable.zip - latest stable / LTS.
Nov 15 2013
prev sibling next sibling parent reply "ponce" <contact gam3sfrommars.fr> writes:
On Thursday, 14 November 2013 at 00:37:38 UTC, Tyro[17] wrote:
 
 Bugfix releases (2.064.1)
     Based on the previous major release or bugfix; contains 
 only bugfixes and perhaps documentation corrections.


 Your thoughts and concerns please.
Do you happen to work with me? We have two dozens of such releases branch :) And customers still tend to prefer the slightly bleeding edge ones. While this effectively _does_ work in creating more stable releases, I think that it puts all the burden on the developers in a way that is difficult to ignore. Most of the time backporting is little work, but sometimes you need to redo a fix you already did, for a branch which only exist to be phased out a bit later, in a codebase slightly different in ways nobody can tell anymore. Sometimes it's even very hard to backport a fix, but if you don't do it how to tell which branch has which bugs? Coupled with being forced to backport every bugfix to every branch, this can make a compelling enough reason never to contribute. With a typical bad fix injection rate of ~5%, this also mean regressions will crop up in release branches and never be noticed since they are not introduced at HEAD level but by the very act of backporting. I'm not sure it can be done in a way that feels right. I prefer the N-month release cycle we have, and take as much time and Release Candidates as needed.
Nov 15 2013
parent reply "Martin Nowak" <code dawg.eu> writes:
On Friday, 15 November 2013 at 09:17:45 UTC, ponce wrote:
 Do you happen to work with me? We have two dozens of such 
 releases branch :) And customers still tend to prefer the 
 slightly bleeding edge ones.

 While this effectively _does_ work in creating more stable 
 releases, I think that it puts all the burden on the developers 
 in a way that is difficult to ignore.

 Most of the time backporting is little work, but sometimes you 
 need to redo a fix you already did, for a branch which only 
 exist to be phased out a bit later, in a codebase slightly 
 different in ways nobody can tell anymore. Sometimes it's even 
 very hard to backport a fix, but if you don't do it how to tell 
 which branch has which bugs?

 Coupled with being forced to backport every bugfix to every 
 branch, this can make a compelling enough reason never to 
 contribute.

 With a typical bad fix injection rate of ~5%, this also mean 
 regressions will crop up in release branches and never be 
 noticed since they are not introduced at HEAD level but by the 
 very act of backporting.

 I'm not sure it can be done in a way that feels right. I prefer 
 the N-month release cycle we have, and take as much time and 
 Release Candidates as needed.
That's the exact problem with most of the release ideas proposed here, they are terribly inefficient. The schedule proposed by Andrew only requires one maintenance branch (point releases) besides the regular beta releases from master. Backporting to a single stable branch should be within our budget.
Nov 15 2013
parent "Dicebot" <public dicebot.lv> writes:
On Friday, 15 November 2013 at 12:58:47 UTC, Martin Nowak wrote:
 That's the exact problem with most of the release ideas 
 proposed here, they are terribly inefficient.
 The schedule proposed by Andrew only requires one maintenance 
 branch (point releases) besides the regular beta releases from 
 master. Backporting to a single stable branch should be within 
 our budget.
It is no different than supporting one LTS backporting branch and current actual release branch. Only difference from proposed model is that by calling it "release" you are taking certain responsibilities to get in at least a bit usable shape at the point of release. DMD releases may have never been stable but they are _much_ more stable than usage of git master HEAD.
Nov 15 2013
prev sibling next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 14/11/13 01:37, Tyro[17] wrote:
 Major releases (2.0)
      The big ones, every six months, intended to ship in distributions and to
be
 used by stability-oriented users.

 Release Candidates (2.065rc1)
      Created one to two weeks before the release branch is created, this is a
 preview of the major release. There should be very few, if any changes from the
 release candidate to the major release.
 Bugfix releases (2.064.1)
      Based on the previous major release or bugfix; contains only bugfixes and
 perhaps documentation corrections.

 Beta release (2.065beta1)
      Created monthly from master, except for those months in which we create a
 major release. Stable and suitable for users who want the latest code and can
 live with changes from month to month.

 Your thoughts and concerns please.
** 1 ** It might be a good idea to consider this from an alternate angle -- different user needs. You can probably divide these into three groups: (1) I want stability above all, no updates, only bugfixes. (2) I want to receive new non-breaking functionality as well as bugfixes. (3) I want bleeding-edge everything, I can cope with backwards-incompatible changes. It's not clear to me how (2) is addressed in your proposal. What guarantees are made about the beta releases and backwards compatibility? Can we assume that with beta releases breaking changes are always a possibility? I imagine it may be difficult to separate out breaking and non-breaking new functionality in DMD development, so I can understand if you just want to support (1) and (3). ** 2 ** Is this proposal for DMD only, or does it include druntime and Phobos? Either way there needs to be some consideration that druntime and Phobos need different approaches from DMD itself. For Phobos in particular there seems to me to be much more short-term need for new functionality which does _not_ include breaking changes. ** 3 ** What's the plan for interacting with downstreams (GDC, LDC and potentially others)? A 1-month beta turnaround may be too fast for them to do the work of integrating new releases; 6 months is an awfully long time to wait for updates. Speaking as one who relies on the speed of the executables they produce, I would be quite upset if I had to wait 6 months for Phobos updates with these compilers -- it's already been frustrating enough as it is having to wait 3+ months for them to get Phobos bugfixes that I've provided.
Nov 19 2013
prev sibling next sibling parent reply "Don" <x nospam.com> writes:
On Thursday, 14 November 2013 at 00:37:38 UTC, Tyro[17] wrote:
 Greetings,

 I have accepted the responsibility of preparing the builds for 
 DMD and would like to engage in conversation about the way 
 ahead.

 The first concern I have is about the build cycle. Presently, 
 it is nonexistent. There is no rhyme or reason regarding when 
 releases are produced. The v2.065 agenda (first attempt of its 
 kind) suggests that the next release will occur sometime in 
 March 2014. I'm of the opinion, however, that the cycle should 
 be six months long.
You shouldn't pay much attention to the length of the recent release cycles. The last two release cycles were abnormally long. Historically we had a release about once a month, then that extended to two months. In the last year the cycles became ridiculously long. It's been completely unplanned. But ANY time period you choose would be better than the current situation.
Nov 19 2013
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday, November 19, 2013 12:50:29 Don wrote:
 You shouldn't pay much attention to the length of the recent
 release cycles.
 The last two release cycles were abnormally long. Historically we
 had a release about once a month, then that extended to two
 months. In the last year the cycles became ridiculously long.
 It's been completely unplanned.
 
 But ANY time period you choose would be better than the current
 situation.
Hmmm. So, 7 years would be better? ;) In any case, we definitely should try and have more regular releases in the one to two month range rather than the really long gaps that we've been having lately. - Jonathan M Davis
Nov 19 2013
prev sibling next sibling parent "eles" <eles eles.com> writes:
On Thursday, 14 November 2013 at 00:37:38 UTC, Tyro[17] wrote:
 Greetings,

 I have accepted the responsibility of preparing the builds for 
 DMD and would like to engage in conversation about the way 
 ahead.
 Your thoughts and concerns please.
Stalled?
Dec 03 2013
prev sibling parent reply Andrew Edwards <ridimz yahoo.com> writes:
 From the preceding discussion, it is clear that the community desires 
both a swift release cycle and more stability. No one solution will 
appease the entire community. Therefore, I will take the middle ground. 
The way forward will be as follows:

	Official releases will occur on an 8 week basis. The third such release 
will be LTS. All LTS releases will be supported with back-ports of 
regressions and non-breaking changes (bug fixes only) until the next LTS 
release. Meaning that only LTS releases will be support by point 
releases. No new feature features will be introduced in a point release.

	Betas will be released four weeks after an official release. The intent 
is to afford a thorough review of all features introduced since the last 
official release and allot ample time to remedy any resulting regressions.

	Release candidates will follow naturally from Betas two weeks prior to 
an official release. Once a release candidate is published, no new 
features will be added.

The same schedule will apply to all components of the language (DMD, 
DRUNTIME, and PHOBOS) and any tools that are included in the release 
packages.

Tools currently included in the packages are as follows:

	Linux:
		DDEMANGLE
		DMAN
		DUMPOBJ
		OBJ2ASM
		RMDM

	OSX:
		DDEMANGLE
		DMAN
		DUMPOBJ
		OBJ2ASM
		RMDM
		SHELL

	Windows:
		DDEMANGLE.EXE
		DMAN.EXE
		DUMPOBJ.EXE
		LIB.EXE
		LINK.EXE
		MAKE.EXE
		RDMD.EXE
		REPLACE.EXE
		SHELL.EXE
		WINDBG.EXE (AND OTHER REQUIRED FILES)

This list, at the very least, should include DustMite but there are some 
other tools included in the tools repo on github that might belong here. 
(This will be considered for inclusion in future releases and will have 
no effect on release 2.065)

As I am not skilled at GitHub, I will be following the 
instructions/guidelines exactly as outlined here:

	http://wiki.dlang.org/Development_and_Release_Process.

I am working on a MacMini running OS X v10.9. I have Ubuntu 13.10 Server 
loaded in VirtualBox and will be using Jordi Sayol's script to build 
packages for linux/Windows and Jacob Carlborg script for OSX.

Both of these scripts require an preexisting release zip and as of this 
moment, I am unaware of the steps to create that file. I will need some 
instructions on how to access and run the auto tester if that's what 
generates the zip or, if it's already automatically created, 
instructions on how to retrieve and stage it for the build.

I will be setting up a tag on github today for the first beta release of 
2.065 (2.065-b1).

Request your corporation/support in ensuring a smooth process.

Andrew
Dec 03 2013
next sibling parent reply "eles" <eles eles.com> writes:
On Tuesday, 3 December 2013 at 14:26:07 UTC, Andrew Edwards wrote:
 From the preceding discussion, it is clear that the community 
 desires both a swift release cycle and more stability. No one 
 solution will appease the entire community. Therefore, I will 
 take the middle ground. The way forward will be as follows:
I support this. It is a good start and, if really needed and obvious, changes to the process could be performed later. But, it is quite nice and it is important to have it started. I would vote for maximum automation of the building process (ideally, a single command that will build up everything in a single shot).
 Tools currently included in the packages are as follows:

 		RMDM
??? RDMD? Should DUB be added, too?
Dec 03 2013
parent "Dicebot" <public dicebot.lv> writes:
On Tuesday, 3 December 2013 at 15:36:46 UTC, eles wrote:
 Should DUB be added, too?
Lets wait a bit until it stabilizes a bit before putting it into official package. At future least SDL addition comes to my mind.
Dec 03 2013
prev sibling next sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Tuesday, 3 December 2013 at 14:26:07 UTC, Andrew Edwards wrote:
 Both of these scripts require an preexisting release zip and as 
 of this moment, I am unaware of the steps to create that file. 
 I will need some instructions on how to access and run the auto 
 tester if that's what generates the zip or, if it's already 
 automatically created, instructions on how to retrieve and 
 stage it for the build.
Wasn't Nick working on automation of that part?
Dec 03 2013
next sibling parent "Andrew Edwards" <ridimz yahoo.com> writes:
On Tuesday, 3 December 2013 at 16:04:32 UTC, Dicebot wrote:
 On Tuesday, 3 December 2013 at 14:26:07 UTC, Andrew Edwards 
 wrote:
 Both of these scripts require an preexisting release zip and 
 as of this moment, I am unaware of the steps to create that 
 file. I will need some instructions on how to access and run 
 the auto tester if that's what generates the zip or, if it's 
 already automatically created, instructions on how to retrieve 
 and stage it for the build.
Wasn't Nick working on automation of that part?
I believe you are correct but I don't think it''s ready as yet.
Dec 03 2013
prev sibling parent reply "Martin Nowak" <code dawg.eu> writes:
On Tuesday, 3 December 2013 at 16:04:32 UTC, Dicebot wrote:
 Wasn't Nick working on automation of that part?
It isn't automatable because the zip is assembled from different platforms.
Dec 03 2013
next sibling parent "Martin Nowak" <code dawg.eu> writes:
On Tuesday, 3 December 2013 at 20:51:27 UTC, Martin Nowak wrote:
 On Tuesday, 3 December 2013 at 16:04:32 UTC, Dicebot wrote:
 Wasn't Nick working on automation of that part?
https://github.com/D-Programming-Language/installer/pull/24
Dec 03 2013
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-12-03 21:51, Martin Nowak wrote:

 It isn't automatable because the zip is assembled from different platforms.
Why can't that be automatable? We do have the servers running the tests. The idea is that they should build them. -- /Jacob Carlborg
Dec 03 2013
parent reply "Martin Nowak" <code dawg.eu> writes:
On Tuesday, 3 December 2013 at 22:24:45 UTC, Jacob Carlborg wrote:
 Why can't that be automatable? We do have the servers running 
 the tests. The idea is that they should build them.
Conflating binaries for multiple platforms is the problem here.
Dec 05 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-12-05 09:02, Martin Nowak wrote:

 Conflating binaries for multiple platforms is the problem here.
Just send them to a single machine which assembles the cross-platform zip. -- /Jacob Carlborg
Dec 05 2013
prev sibling next sibling parent reply "tn" <no email.com> writes:
In general this sounds great. However:

On Tuesday, 3 December 2013 at 14:26:07 UTC, Andrew Edwards wrote:
 Betas will be released four weeks after an official release.
Does this mean that a new release branch will be created at that point? I think it makes sense.
 Once a release candidate is published, no new features will be 
 added.
Does this imply that new features are still added in beta phase? That does not fit well with creating the branch in the beginning of the beta phase. My opinion: I don't think new features should be added in beta phase. Instead it should be for bug fixing. When no known blocker bugs are left, a release candidate phase should be started. Relase candidate should be, as the name implies, a candidate for release. Thus, if no new critical bugs are found, then the release can be made in principle just by bumping the version number. More important than defining when the actual releases are made, is to define when branching is done. It is easy to slip release date because of new bugs and that can then delay the next release (depending on how you implement the procedure). Instead, a new release branch should be branched from the master branch exactly every 8 weeks. That should be easy, since there are no such stability requirements as for actual releases. Whether the following stabilisation phase (betas + release candidates) lasts exactly 3 or 4 or 5 weeks is not that important as long as the quality of the release will be acceptable. The phase should last long enough that all blocker bugs are fixed. In this model the time between actual releases can vary a bit, but on average it will be exactly 8 weeks.
Dec 03 2013
parent Andrew Edwards <ridimz yahoo.com> writes:
On 12/3/13, 12:21 PM, tn wrote:> In general this sounds great. However:
 On Tuesday, 3 December 2013 at 14:26:07 UTC, Andrew Edwards wrote:
 Betas will be released four weeks after an official release.
Does this mean that a new release branch will be created at that point? I think it makes sense.
Yes.
 Once a release candidate is published, no new features will be added.
Does this imply that new features are still added in beta phase? That does not fit well with creating the branch in the beginning of the beta phase. My opinion: I don't think new features should be added in beta phase. Instead it should be for bug fixing. When no known blocker bugs are left, a release candidate phase should be started. Relase candidate should be, as the name implies, a candidate for release. Thus, if no new critical bugs are found, then the release can be made in principle just by bumping the version number.
You make a good point. I actually thought about that after I posted but was on my way to work so could not re-post at the time. The beta release will start a new branch and will basically follow the steps you describe. No new features after a beta is released.
 More important than defining when the actual releases are made, is to
 define when branching is done. It is easy to slip release date because
 of new bugs and that can then delay the next release (depending on how
 you implement the procedure). Instead, a new release branch should be
 branched from the master branch exactly every 8 weeks. That should be
 easy, since there are no such stability requirements as for actual
 releases. Whether the following stabilisation phase (betas + release
 candidates) lasts exactly 3 or 4 or 5 weeks is not that important as
 long as the quality of the release will be acceptable. The phase should
 last long enough that all blocker bugs are fixed. In this model the time
 between actual releases can vary a bit, but on average it will be
 exactly 8 weeks.
Make sense. I will take this into consideration.
Dec 03 2013
prev sibling next sibling parent reply "Brad Anderson" <eco gnuk.net> writes:
On Tuesday, 3 December 2013 at 14:26:07 UTC, Andrew Edwards wrote:
 [snip]
 I am working on a MacMini running OS X v10.9. I have Ubuntu 
 13.10 Server loaded in VirtualBox and will be using Jordi 
 Sayol's script to build packages for linux/Windows and Jacob 
 Carlborg script for OSX.
Please use windows/dinstaller.nsi for creating the installer on Windows. linux/win/installer.nsi has some missing features and an issue with deleting the entire directory tree as part of a uninstallation pre-step that has caused problems for some people. I'm going to try to merge these two files to pick up some of the useful features Jordi has added but for the time being dinstaller.nsi is the proper one to use.
 Both of these scripts require an preexisting release zip and as 
 of this moment
dinstaller.nsi defaults to a web-installer (it downloads dmd2.zip and others itself). As such, it doesn't require the dmd zip to be present in order to create the installer executable. It can optionally embed the zip in the installer but that hasn't been how it's been used historically. If you open up dinstaller.nsi it should be fairly obvious how it all works with regard to updating it for a new release. Just change the version number at the top of the file and ensure the url to the dmd zip a bit further down is correct. Have you arranged with Walter or Brad Roberts to get access to upload to the download server? I think the build master having the ability to do uploads is vital.
, I am unaware of the steps to create that file.
I believe Nick's release tool can generate the release zip properly. https://github.com/D-Programming-Language/installer/pull/24 I really recommend looking at it. He spent a lot of time perfecting it.
 I will need some instructions on how to access and run the auto 
 tester if that's what generates the zip or, if it's already 
 automatically created, instructions on how to retrieve and 
 stage it for the build.
Historically, Walter has created the release zips (just using the makefile, I believe). As far as I know, work hasn't begun on getting the autotester to roll releases.
 I will be setting up a tag on github today for the first beta 
 release of 2.065 (2.065-b1).

 Request your corporation/support in ensuring a smooth process.
Good luck. Let me know if you have any questions or problems with the Windows installer script.
 Andrew
Dec 03 2013
next sibling parent reply Andrew Edwards <ridimz yahoo.com> writes:
On 12/3/13, 2:08 PM, Brad Anderson wrote:
 On Tuesday, 3 December 2013 at 14:26:07 UTC, Andrew Edwards wrote:
 [snip]
 I am working on a MacMini running OS X v10.9. I have Ubuntu 13.10
 Server loaded in VirtualBox and will be using Jordi Sayol's script to
 build packages for linux/Windows and Jacob Carlborg script for OSX.
Please use windows/dinstaller.nsi for creating the installer on Windows. linux/win/installer.nsi has some missing features and an issue with deleting the entire directory tree as part of a uninstallation pre-step that has caused problems for some people. I'm going to try to merge these two files to pick up some of the useful features Jordi has added but for the time being dinstaller.nsi is the proper one to use.
Ok, got you. One question, does dinstaller.nsi work on linux/osx? If not I will need your assistance to prepare the package until such time as I can obtain an image for VirtualBox.
 Both of these scripts require an preexisting release zip and as of
 this moment
dinstaller.nsi defaults to a web-installer (it downloads dmd2.zip and others itself). As such, it doesn't require the dmd zip to be present in order to create the installer executable. It can optionally embed the zip in the installer but that hasn't been how it's been used historically.
Which is basically a staged zip file and the same problem as stated. If the zip file doesn't exist locally or on the web site, then these tools do not work.
 If you open up dinstaller.nsi it should be fairly obvious how it all
 works with regard to updating it for a new release. Just change the
 version number at the top of the file and ensure the url to the dmd zip
 a bit further down is correct.
Understood.
 Have you arranged with Walter or Brad Roberts to get access to upload to
 the download server? I think the build master having the ability to do
 uploads is vital.
I have not. Did for the GitHub repository but forgot about that. Thanks for the reminder.
 , I am unaware of the steps to create that file.
I believe Nick's release tool can generate the release zip properly. https://github.com/D-Programming-Language/installer/pull/24 I really recommend looking at it. He spent a lot of time perfecting it.
Will try it but I thought it was platform specific. Meaning, I thought it created a OSX specific archive if ran on OSX and so on.
 I will need some instructions on how to access and run the auto tester
 if that's what generates the zip or, if it's already automatically
 created, instructions on how to retrieve and stage it for the build.
Historically, Walter has created the release zips (just using the makefile, I believe). As far as I know, work hasn't begun on getting the autotester to roll releases.
That will be great when it's completed.
 I will be setting up a tag on github today for the first beta release
 of 2.065 (2.065-b1).

 Request your corporation/support in ensuring a smooth process.
Good luck. Let me know if you have any questions or problems with the Windows installer script.
Thanks.
 Andrew
Dec 03 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-12-04 02:30, Andrew Edwards wrote:

 Ok, got you. One question, does dinstaller.nsi work on linux/osx? If not
 I will need your assistance to prepare the package until such time as I
 can obtain an image for VirtualBox.
Everything should be built using the autotester.
 Which is basically a staged zip file and the same problem as stated. If
 the zip file doesn't exist locally or on the web site, then these tools
 do not work.
Then the tools need to be changed to be workable from source.
 Will try it but I thought it was platform specific. Meaning, I thought
 it created a OSX specific archive if ran on OSX and so on.
It need to compile everything on each platform. We don't support true cross-platform builds yet.
 Historically, Walter has created the release zips (just using the
 makefile, I believe). As far as I know, work hasn't begun on getting the
 autotester to roll releases.
That will be great when it's completed.
That should be our primary goal now. -- /Jacob Carlborg
Dec 04 2013
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-12-03 20:08, Brad Anderson wrote:

 Historically, Walter has created the release zips (just using the
 makefile, I believe). As far as I know, work hasn't begun on getting the
 autotester to roll releases.
I doubt he has a completely automated process of doing it. It has failed too many times. Having the autotester building and handling everything should be our primary goal. -- /Jacob Carlborg
Dec 04 2013
next sibling parent reply Brad Roberts <braddr puremagic.com> writes:
On 12/4/13 12:18 AM, Jacob Carlborg wrote:
 On 2013-12-03 20:08, Brad Anderson wrote:

 Historically, Walter has created the release zips (just using the
 makefile, I believe). As far as I know, work hasn't begun on getting the
 autotester to roll releases.
I doubt he has a completely automated process of doing it. It has failed too many times. Having the autotester building and handling everything should be our primary goal.
I'm not sure this is really a good goal any more. The overlap between testing and releasing isn't as big as I originally thought. Add to that that the very definition of what a release is is in question. The only overlap that really exists is that there's a set of boxes that exist and that they build in somewhat similar ways. But there's also a hell of a lot of differences. Ignoring pulls for the most part, there's little coordination between the platforms in terms of what to build. It's always "whatever is current" which changes over time. There's no current automation for how to package a release that's part of the packages being tested (there's some fledgling support inside the packages, there's some tooling that lives off in other places). There's parts of the releases that have no source and aren't part of what's being tested. If we stick to having a zip that contains multiple platforms, there's no code to rendezvous multiple platforms together. The auto-tester covers a smaller set of platforms than release builds are currently done for (example, there's several linux distros that have distro specific packages, not all of which are represented by test boxes). All in all, while I was of the opinion that the test fleet could serve the role of release builders and pushed that with Walter and Andrei, I'm much less convinced that's the right short term answer right now. It might become the right answer once a number of the open questions are answered and once the automation for building the releases are improved, but not yet. I guess the 'have the auto-tester do it' mentality really hand waves away that most of the work still has to be done to make it something that might one day be run automatically. The auto-tester isn't nearly as magic as I think a lot of people think it is. It's very little more than a set of clients that: 0) ask a server for what package/branches to checkout and optionally what package/branch to merge 1) checkout a set of git repos 2) for a pull test: merge a branch in a particular repo 3) run make in each 4) run make unittest in each 5) for each of the actions in 1-4, ship the success or failure status and the log file to the results server 6) Repeat forever. And a server that collects the results and answers the question asked in #0 above. There's some interesting logic in deciding how to answer that question and a little bit of fun logic to sync between state between github and the server. But it's not particularly fancy or complicated at the component level. My current 2 cents, Brad
Dec 04 2013
parent reply Jacob Carlborg <doob me.com> writes:
On 2013-12-04 20:52, Brad Roberts wrote:

 I'm not sure this is really a good goal any more.  The overlap between
 testing and releasing isn't as big as I originally thought.  Add to that
 that the very definition of what a release is is in question.  The only
 overlap that really exists is that there's a set of boxes that exist and
 that they build in somewhat similar ways.
I have only ever thought that the autotester provides multiple different platforms which we could build the tools and create the releases[1] on. What I see as the advantage is: * The autotester is always available * Not a single person need to have all different platforms to create a release * We don't need to rely on someone else to build packages for a given platform. He/she may be away for a few days and suddenly we can't get a release for a given platform * It's reproducible. We're creating the release on the same machines every time Just look at the last release. It doesn't include pre-compiled binaries for FreeBSD, because Walter's machine apparently can't build it anymore without running out of memory. That must never happen.
 But there's also a hell of a lot of differences.[SNIP]
I'm not saying that the autotester will just magically be able to build releases. I'm saying it provides the platforms we need (ok, seems to be missing one or two Linux distributions) to be able to build releases. As you say, we don't yet have an automated process of creating releases. But when we're creating that (writing a script/tool) it need to support building on several different machines, then be able to collect the result (cross-platform zip). We don't want to create a script that automates the release, then change all that when we won't it to run on the autotester machines. BTW, I'm not sure we even can create an automate process without something like the autotester. We need to be able to build on all supported platforms. Sure, someone else might have a complete fleet of machines. It's not the autotester itself that is important, it's all the machines it runs on. [1] When I say "release", in this case I'm referring to building the compiler, all the tools and libraries, packaging it in a platform specific installer and cross-platform zip. -- /Jacob Carlborg
Dec 04 2013
parent reply "Martin Nowak" <code dawg.eu> writes:
On Thursday, 5 December 2013 at 07:58:10 UTC, Jacob Carlborg 
wrote:
 * Not a single person need to have all different platforms to 
 create a release
We'll need to move to building linux distribution specific build to fix issues with shared library dependencies, so it's even more platforms.
 * We don't need to rely on someone else to build packages for a 
 given platform. He/she may be away for a few days and suddenly 
 we can't get a release for a given platform
Automation is definitely a good solution.
 * It's reproducible. We're creating the release on the same 
 machines every time
I think a good solution would be to prepare clean virtualbox images that are used for building.
Dec 05 2013
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-12-05 09:09, Martin Nowak wrote:

 We'll need to move to building linux distribution specific build to fix
 issues with shared library dependencies, so it's even more platforms.
Yes. Can we manage with Debian and Red Hat based system? Do we need others as well?
 I think a good solution would be to prepare clean virtualbox images that
 are used for building.
1. I don't think it matters that much if the releases are created on virtual machines or physical machines. As long as they're available 2. Due to EULA, you're only allowed to run Mac OS X as a virtual guest on a Mac OS X host. It's technically possible to run it on any host -- /Jacob Carlborg
Dec 05 2013
prev sibling parent reply Andrew Edwards <ridimz yahoo.com> writes:
On 12/5/13, 3:09 AM, Martin Nowak wrote:
 On Thursday, 5 December 2013 at 07:58:10 UTC, Jacob Carlborg wrote:
 * Not a single person need to have all different platforms to create a
 release
We'll need to move to building linux distribution specific build to fix issues with shared library dependencies, so it's even more platforms.
Which ones specifically? Does Ubuntu and Fedora suffice or are there others that need to be included?
 * We don't need to rely on someone else to build packages for a given
 platform. He/she may be away for a few days and suddenly we can't get
 a release for a given platform
Automation is definitely a good solution.
 * It's reproducible. We're creating the release on the same machines
 every time
I think a good solution would be to prepare clean virtualbox images that are used for building.
I crashed my system so I had to start over with a fresh installation/configuration. The following images are installed within VirtualBox: OS X 10.9 Ubuntu 12.04 Fedora 19 FreeBSD 9.2 Windows 7 [1] All images are 64 bits and allotted 20.00 GB fixed storage, 32 MB Video Memory, and enabled Bridge network adapter. The base system is currently a MacMini running OSX 10.9 but will be upgraded to a Mac Pro in the near future. Are there any other OS's that should be included? Not sure what the correct configuration should be but I do know that, at my current level of expertise, configuring them can be a pain in the arse.... Are there any scripts available to configure/secure these operating systems? Any suggestions regarding configuration to eliminate problems during build process? [1] There is not yet a windows image installed as I do not have an installation CD. Thanks, Andrew
Dec 06 2013
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-12-06 17:45, Andrew Edwards wrote:

 I crashed my system so I had to start over with a fresh
 installation/configuration. The following images are installed within
 VirtualBox:

      OS X 10.9
      Ubuntu 12.04
      Fedora 19
      FreeBSD 9.2
      Windows 7 [1]

 All images are 64 bits and allotted 20.00 GB fixed storage, 32 MB Video
 Memory, and enabled Bridge network adapter. The base system is currently
 a MacMini running OSX 10.9 but will be upgraded to a Mac Pro in the near
 future. Are there any other OS's that should be included?
I would have chosen Debian instead of Ubuntu and something like Red Hat or CentOS instead of Fedora. The reason for this is that they're older and more likely to work on more systems. Have you verified that you can compile for 32bit on all of these systems. I tried PC-BSD (based on FreeBSD) and it had problems compiling 32bit. It can actually correctly generate code but the header files can't handle both 32 and 64bit. That means you can compile D code for 32bit but not C++.
 Not sure what the correct configuration should be but I do know that, at
 my current level of expertise, configuring them can be a pain in the
 arse.... Are there any scripts available to configure/secure these
 operating systems? Any suggestions regarding configuration to eliminate
 problems during build process?

 [1] There is not yet a windows image installed as I do not have an
 installation CD.
I'm wondering if you can use Wine or ReactOS. But if would be preferred to use real Windows. -- /Jacob Carlborg
Dec 06 2013
parent "Dicebot" <public dicebot.lv> writes:
On Friday, 6 December 2013 at 17:09:34 UTC, Jacob Carlborg wrote:
 On 2013-12-06 17:45, Andrew Edwards wrote:

 I crashed my system so I had to start over with a fresh
 installation/configuration. The following images are installed 
 within
 VirtualBox:

     OS X 10.9
     Ubuntu 12.04
     Fedora 19
     FreeBSD 9.2
     Windows 7 [1]

 All images are 64 bits and allotted 20.00 GB fixed storage, 32 
 MB Video
 Memory, and enabled Bridge network adapter. The base system is 
 currently
 a MacMini running OSX 10.9 but will be upgraded to a Mac Pro 
 in the near
 future. Are there any other OS's that should be included?
I would have chosen Debian instead of Ubuntu and something like Red Hat or CentOS instead of Fedora. The reason for this is that they're older and more likely to work on more systems.
Unfortunately it does not provide hard guarantees. Major version change may imply ABI breakage so using older distro / libraries buys you nothing.
Dec 06 2013
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-12-06 17:45, Andrew Edwards wrote:

 I crashed my system so I had to start over with a fresh
 installation/configuration. The following images are installed within
 VirtualBox:

      OS X 10.9
      Ubuntu 12.04
      Fedora 19
      FreeBSD 9.2
      Windows 7 [1]

 All images are 64 bits and allotted 20.00 GB fixed storage, 32 MB Video
 Memory, and enabled Bridge network adapter. The base system is currently
 a MacMini running OSX 10.9 but will be upgraded to a Mac Pro in the near
 future. Are there any other OS's that should be included?
Gentoo and Arch Linux perhaps? I don't remember if Arch uses its own package manager or not. Dicebot can answer this. -- /Jacob Carlborg
Dec 06 2013
parent reply "Dicebot" <public dicebot.lv> writes:
On Friday, 6 December 2013 at 17:10:58 UTC, Jacob Carlborg wrote:
 Gentoo and Arch Linux perhaps? I don't remember if Arch uses 
 its own package manager or not. Dicebot can answer this.
Gentoo does not have binary packages. Arch Linux has own package manager / package format but I'd prefer it to be ignored by official releases while I am maintainer - it will just cause me more troubles. I think Fedora and Ubuntu LTS should cover majority of users for practical reasons. We can't support everything anyway so lets stop at some minimal base.
Dec 06 2013
parent reply Jacob Carlborg <doob me.com> writes:
On 2013-12-06 18:15, Dicebot wrote:

 Gentoo does not have binary packages.
Is Gentoo needed to create package, although it isn't binary packages? -- /Jacob Carlborg
Dec 06 2013
parent reply "Dicebot" <public dicebot.lv> writes:
I see no reason in try to provide packages for a platforms that 
are already maintained by people that actually use those 
platforms. Such attempts are doomed to be inferior to native 
ones. It makes sense to take over support if it was abandoned for 
that specific platform but not before.
Dec 06 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-12-06 18:29, Dicebot wrote:
 I see no reason in try to provide packages for a platforms that are
 already maintained by people that actually use those platforms. Such
 attempts are doomed to be inferior to native ones. It makes sense to
 take over support if it was abandoned for that specific platform but not
 before.
Fair enough. -- /Jacob Carlborg
Dec 06 2013
prev sibling parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
06-Dec-2013 20:45, Andrew Edwards пишет:
 On 12/5/13, 3:09 AM, Martin Nowak wrote:
[snip]
 [1] There is not yet a windows image installed as I do not have an
 installation CD.
Well, there is a temporary option to get an 90-day evaluation version of enterprise edition ISO. Here is 8.1, building DMD/druntime/phobos works for me: http://technet.microsoft.com/en-us/evalcenter/hh699156.aspx You may snatch Windows 7 from them by navigating a bunch of links with "Evaluation" and "Older products" somewhere on that page. Might require registering Windows Live account as well. In the longterm, well, they do offer digital editions (ISO images) for quite some time. I do suspect I can actually share ISO myself legally, as the only thing you do buy from MS is LICENSE (i.e. the key). -- Dmitry Olshansky
Dec 06 2013
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/4/2013 12:18 AM, Jacob Carlborg wrote:
 I doubt he has a completely automated process of doing it. It has failed too
 many times.
The cause of those failures is that the product changes from release to release. There are new files to be added, old ones to be removed, directories change, targets change, everything is in flux. A typical problem is adding a source file to a component but forgetting to add it to the manifest.
Dec 05 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-12-05 09:32, Walter Bright wrote:

 The cause of those failures is that the product changes from release to
 release. There are new files to be added, old ones to be removed,
 directories change, targets change, everything is in flux.
What about all the times when the git tag and the release didn't contain the same code?
 A typical problem is adding a source file to a component but forgetting
 to add it to the manifest.
Is a manifest really needed? -- /Jacob Carlborg
Dec 05 2013
prev sibling next sibling parent reply "Nick" <nmsmith65 gmail.com> writes:
On Tuesday, 3 December 2013 at 14:26:07 UTC, Andrew Edwards wrote:
 	Betas will be released four weeks after an official release. 
 The intent is to afford a thorough review of all features 
 introduced since the last official release and allot ample time 
 to remedy any resulting regressions.

 	Release candidates will follow naturally from Betas two weeks 
 prior to an official release. Once a release candidate is 
 published, no new features will be added.
I don't understand the necessity of a beta release if it doesn't guarantee anything with respect to the codebase. Wouldn't it make more sense to have a "mid-release review" after 4 weeks where everyone discusses the state of the release and the targets for the release candidate? The release candidate seems to handle locking down the addition of new features.
Dec 03 2013
parent Andrew Edwards <ridimz yahoo.com> writes:
On 12/3/13, 3:32 PM, Nick wrote:
 On Tuesday, 3 December 2013 at 14:26:07 UTC, Andrew Edwards wrote:
     Betas will be released four weeks after an official release. The
 intent is to afford a thorough review of all features introduced since
 the last official release and allot ample time to remedy any resulting
 regressions.

     Release candidates will follow naturally from Betas two weeks
 prior to an official release. Once a release candidate is published,
 no new features will be added.
I don't understand the necessity of a beta release if it doesn't guarantee anything with respect to the codebase. Wouldn't it make more sense to have a "mid-release review" after 4 weeks where everyone discusses the state of the release and the targets for the release candidate? The release candidate seems to handle locking down the addition of new features.
The beta release will the signify the end of new features for that particular release cycle. I was concerned about the same thing after posting this morning but was on my way out the door so could not address earlier.
Dec 03 2013
prev sibling next sibling parent reply "Martin Nowak" <code dawg.eu> writes:
On Tuesday, 3 December 2013 at 14:26:07 UTC, Andrew Edwards wrote:
 Tools currently included in the packages are as follows:
We need to discuss the list of included tools. Some of them are only available as binaries, i.e. only Walter knows how to build them. Also DUMPOBJ and OBJ2ASM aren't needed on linux or OSX. The tools strictly related to D are, dman, ddmangle and rdmd.
 This list, at the very least, should include DustMite but there 
 are some other tools included in the tools repo on github that 
 might belong here. (This will be considered for inclusion in 
 future releases and will have no effect on release 2.065)

 As I am not skilled at GitHub, I will be following the 
 instructions/guidelines exactly as outlined here:

 	http://wiki.dlang.org/Development_and_Release_Process.
This does not reflect how we handle things currently. It would be good to start from the current process (version branch + cherry-picking from master) and incrementally improve it towards what we want to achieve. The wiki depicts an idealistic process that requires a lot of awareness by every dev and doesn't map good onto the GitHub workflow.
 I am working on a MacMini running OS X v10.9. I have Ubuntu 
 13.10 Server loaded in VirtualBox and will be using Jordi 
 Sayol's script to build packages for linux/Windows and Jacob 
 Carlborg script for OSX.

 Both of these scripts require an preexisting release zip and as 
 of this moment, I am unaware of the steps to create that file. 
 I will need some instructions on how to access and run the auto 
 tester if that's what generates the zip or, if it's already 
 automatically created, instructions on how to retrieve and 
 stage it for the build.
Only Walter knows how to build the zip. Nick wrote a build script the produces a similar zip file, https://github.com/D-Programming-Language/installer/pull/24. Maybe we can get that in shape for the next few releases. In the mid-term we should replace all installer scripts that depend on the dmd2.zip with scripts that build a platform specific release from source. Work is being done to add install targets to the Makefiles of each project. I'm currently working on a spec file to build RPMs for Fedora.
 I will be setting up a tag on github today for the first beta 
 release of 2.065 (2.065-b1).
It would be helpful to post a short notice in advance on the dmd-beta mailing list so that the devs can coordinate their work accordingly.
Dec 03 2013
next sibling parent "Martin Nowak" <code dawg.eu> writes:
On Tuesday, 3 December 2013 at 21:23:49 UTC, Martin Nowak wrote:
 The tools strictly related to D are, dman, ddmangle and rdmd.
And link on windows. Thanks for doing this Andrew.
Dec 03 2013
prev sibling parent reply Andrew Edwards <ridimz yahoo.com> writes:
On 12/3/13, 4:23 PM, Martin Nowak wrote:
 On Tuesday, 3 December 2013 at 14:26:07 UTC, Andrew Edwards wrote:
 Tools currently included in the packages are as follows:
We need to discuss the list of included tools. Some of them are only available as binaries, i.e. only Walter knows how to build them. Also DUMPOBJ and OBJ2ASM aren't needed on linux or OSX. The tools strictly related to D are, dman, ddmangle and rdmd.
 This list, at the very least, should include DustMite but there are
 some other tools included in the tools repo on github that might
 belong here. (This will be considered for inclusion in future releases
 and will have no effect on release 2.065)

 As I am not skilled at GitHub, I will be following the
 instructions/guidelines exactly as outlined here:

     http://wiki.dlang.org/Development_and_Release_Process.
This does not reflect how we handle things currently. It would be good to start from the current process (version branch + cherry-picking from master) and incrementally improve it towards what we want to achieve. The wiki depicts an idealistic process that requires a lot of awareness by every dev and doesn't map good onto the GitHub workflow.
Understood. I would however, suggest that it is a habit that we should get into. One of the first things that we should have devs review when they want to know how to contribute is to lead them to this page. As I matter of fact, I think it might even be a good idea to include it in the CONTRIBUTING.md file on GitHub or, at the very least, insert a link to it.
 I am working on a MacMini running OS X v10.9. I have Ubuntu 13.10
 Server loaded in VirtualBox and will be using Jordi Sayol's script to
 build packages for linux/Windows and Jacob Carlborg script for OSX.

 Both of these scripts require an preexisting release zip and as of
 this moment, I am unaware of the steps to create that file. I will
 need some instructions on how to access and run the auto tester if
 that's what generates the zip or, if it's already automatically
 created, instructions on how to retrieve and stage it for the build.
Only Walter knows how to build the zip. Nick wrote a build script the produces a similar zip file, https://github.com/D-Programming-Language/installer/pull/24. Maybe we can get that in shape for the next few releases.
I'll check it out. Hopefully it does what I need.
 In the mid-term we should replace all installer scripts that depend on
 the dmd2.zip with scripts that build a platform specific release from
 source.
 Work is being done to add install targets to the Makefiles of each project.

 I'm currently working on a spec file to build RPMs for Fedora.

 I will be setting up a tag on github today for the first beta release
 of 2.065 (2.065-b1).
It would be helpful to post a short notice in advance on the dmd-beta mailing list so that the devs can coordinate their work accordingly.
Will do.
Dec 03 2013
next sibling parent reply "Jesse Phillips" <Jesse.K.Phillips+D gmail.com> writes:
On Wednesday, 4 December 2013 at 02:15:23 UTC, Andrew Edwards 
wrote:
    http://wiki.dlang.org/Development_and_Release_Process.
This does not reflect how we handle things currently. It would be good to start from the current process (version branch + cherry-picking from master) and incrementally improve it towards what we want to achieve. The wiki depicts an idealistic process that requires a lot of awareness by every dev and doesn't map good onto the GitHub workflow.
Understood. I would however, suggest that it is a habit that we should get into. One of the first things that we should have devs review when they want to know how to contribute is to lead them to this page. As I matter of fact, I think it might even be a good idea to include it in the CONTRIBUTING.md file on GitHub or, at the very least, insert a link to it.
It's not the new contributors which is the problem. It is the existing contributors. Every dev has to be retrained on how to handling patching multiple branches (in fact, it is less important contributors know, much more that the people merging know). It is fine to move in this direction, but since you aren't well versed in git workflow it may be too much to take on for the first release. Also, at some point Walter said in a random forum post that he wanted this next release to be bug fixes only and rather quick... You're plan and Walter's don't seem to match exactly.
Dec 03 2013
next sibling parent Andrew Edwards <ridimz yahoo.com> writes:
On 12/3/13, 9:26 PM, Jesse Phillips wrote:
 On Wednesday, 4 December 2013 at 02:15:23 UTC, Andrew Edwards wrote:
    http://wiki.dlang.org/Development_and_Release_Process.
This does not reflect how we handle things currently. It would be good to start from the current process (version branch + cherry-picking from master) and incrementally improve it towards what we want to achieve. The wiki depicts an idealistic process that requires a lot of awareness by every dev and doesn't map good onto the GitHub workflow.
Understood. I would however, suggest that it is a habit that we should get into. One of the first things that we should have devs review when they want to know how to contribute is to lead them to this page. As I matter of fact, I think it might even be a good idea to include it in the CONTRIBUTING.md file on GitHub or, at the very least, insert a link to it.
It's not the new contributors which is the problem. It is the existing contributors. Every dev has to be retrained on how to handling patching multiple branches (in fact, it is less important contributors know, much more that the people merging know).
Point taken. It is hard to break old habits and from new ones but sometimes it is absolutely necessary. IMHO this is one of those times.
 It is fine to move in this direction, but since you aren't well versed
 in git workflow it may be too much to take on for the first release.
The ball is rolling and everything worked as documented with one minor exception (where it says git:// replace with https://). I have created the branch/tag and am waiting for a message to be released to the dmd-beta mailing list. Once devs are alerted and the binaries are prepared, a more formal message will be placed in the announce group.
 Also, at some point Walter said in a random forum post that he wanted
 this next release to be bug fixes only and rather quick... You're plan
 and Walter's don't seem to match exactly.
Unfortunately I did not see that post. And while our thoughts/plans may be different, Walter does have the option to speed up or slow down the process should it be warranted. As such, I will proceed according to my plan until informed otherwise.
Dec 03 2013
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-12-04 03:26, Jesse Phillips wrote:

 Also, at some point Walter said in a random forum post that he wanted
 this next release to be bug fixes only and rather quick... You're plan
 and Walter's don't seem to match exactly.
Walter has also said it's up to the release manager (build master) to decide this. -- /Jacob Carlborg
Dec 04 2013
prev sibling parent reply Andrew Edwards <ridimz yahoo.com> writes:
On 12/3/13, 9:15 PM, Andrew Edwards wrote:
 On 12/3/13, 4:23 PM, Martin Nowak wrote:
 Only Walter knows how to build the zip.
 Nick wrote a build script the produces a similar zip file,
 https://github.com/D-Programming-Language/installer/pull/24.
 Maybe we can get that in shape for the next few releases.
I'll check it out. Hopefully it does what I need.
Tried it but came across the following errors: andrews-mini:osx-release ace$ ~/create_dmd_release v2.065-b1 --extras=$HOME/localextras-osx --archive-zip --archive-7z Cloning: git github.com:D-Programming-Language/dmd.git The authenticity of host 'github.com (192.30.252.128)' can't be established. RSA key fingerprint is 16:27:ac:a5:76:28:2d:36:63:1b:56:4d:eb:df:a6:48. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'github.com,192.30.252.128' (RSA) to the list of known hosts. Permission denied (publickey). fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists. Couldn't do git protocol, falling back to 'https://'... Cloning: https://github.com/D-Programming-Language/dmd.git Cloning: https://github.com/D-Programming-Language/druntime.git error: pathspec 'v2.065-b1' did not match any file(s) known to git. create_dmd_release: Error: Command failed (ran from dir '/private/var/folders/40/ks15z23971qfm9vsn2c2mz640000gn/T/.create_dmd release/druntime'): git checkout -q 'v2.065-b1' I created the v2.065-b1 tag and verified that it actually exists. If I use 2.065 as indicated in the instructions for making betas, it results in the exact same error. Any thoughts? Thanks, Andrew
Dec 03 2013
next sibling parent reply "Craig Dillabaugh" <craig.dillabaugh gmail.com> writes:
On Wednesday, 4 December 2013 at 04:29:19 UTC, Andrew Edwards 
wrote:
 On 12/3/13, 9:15 PM, Andrew Edwards wrote:
 On 12/3/13, 4:23 PM, Martin Nowak wrote:
 Only Walter knows how to build the zip.
 Nick wrote a build script the produces a similar zip file,
 https://github.com/D-Programming-Language/installer/pull/24.
 Maybe we can get that in shape for the next few releases.
I'll check it out. Hopefully it does what I need.
Tried it but came across the following errors: andrews-mini:osx-release ace$ ~/create_dmd_release v2.065-b1 --extras=$HOME/localextras-osx --archive-zip --archive-7z Cloning: git github.com:D-Programming-Language/dmd.git The authenticity of host 'github.com (192.30.252.128)' can't be established. RSA key fingerprint is 16:27:ac:a5:76:28:2d:36:63:1b:56:4d:eb:df:a6:48. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'github.com,192.30.252.128' (RSA) to the list of known hosts. Permission denied (publickey). fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists. Couldn't do git protocol, falling back to 'https://'... Cloning: https://github.com/D-Programming-Language/dmd.git Cloning: https://github.com/D-Programming-Language/druntime.git error: pathspec 'v2.065-b1' did not match any file(s) known to git. create_dmd_release: Error: Command failed (ran from dir '/private/var/folders/40/ks15z23971qfm9vsn2c2mz640000gn/T/.create_dmd release/druntime'): git checkout -q 'v2.065-b1' I created the v2.065-b1 tag and verified that it actually exists. If I use 2.065 as indicated in the instructions for making betas, it results in the exact same error. Any thoughts? Thanks, Andrew
Maybe you have already looked here, but it may help you with your first error: https://help.github.com/articles/error-permission-denied-publickey Good luck. Craig
Dec 03 2013
parent Andrew Edwards <ridimz yahoo.com> writes:
On 12/3/13, 11:38 PM, Craig Dillabaugh wrote:
 On Wednesday, 4 December 2013 at 04:29:19 UTC, Andrew Edwards wrote:
 On 12/3/13, 9:15 PM, Andrew Edwards wrote:
 On 12/3/13, 4:23 PM, Martin Nowak wrote:
 Only Walter knows how to build the zip.
 Nick wrote a build script the produces a similar zip file,
 https://github.com/D-Programming-Language/installer/pull/24.
 Maybe we can get that in shape for the next few releases.
I'll check it out. Hopefully it does what I need.
Tried it but came across the following errors:
...
 I created the v2.065-b1 tag and verified that it actually exists. If I
 use 2.065 as indicated in the instructions for making betas, it
 results in the exact same error.

 Any thoughts?

 Thanks,
 Andrew
Maybe you have already looked here, but it may help you with your first error: https://help.github.com/articles/error-permission-denied-publickey Good luck. Craig
I didn't look into that as the there is a built in fallback for https. I will need to resolve the issue eventually though so thanks for the link. Original problem solved. Andrew
Dec 03 2013
prev sibling next sibling parent reply Andrew Edwards <ridimz yahoo.com> writes:
On 12/3/13, 11:29 PM, Andrew Edwards wrote:
 I created the v2.065-b1 tag and verified that it actually exists. If I
 use 2.065 as indicated in the instructions for making betas, it results
 in the exact same error.

 Any thoughts?
OK the problem was that I didn't finish branching all the components. I did DMD but not DRUNTIME, PHOBOS, INSTALLER, TOOLS, and DLANG.ORG. They are now complete and I successfully built the binaries file for OSX. My next question is whether or not this can be used on my platform (OSX) to build binaries for the other platforms (Linux, Windows, FreeBSD) or do I have to be on those platforms to do so? If that is the case, then I will need assistance getting for Windows binaries created and it will require more time than anticipated to get the the FreeBSD and Linux binaries complete because I will be teaching at 8AM and have not yet look at the material. Thanks, Andrew
Dec 03 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-12-04 06:25, Andrew Edwards wrote:

 My next question is whether or not this can be used on my platform (OSX)
 to build binaries for the other platforms (Linux, Windows, FreeBSD) or
 do I have to be on those platforms to do so? If that is the case, then I
 will need assistance getting for Windows binaries created and it will
 require more time than anticipated to get the the FreeBSD and Linux
 binaries complete because I will be teaching at 8AM and have not yet
 look at the material.
Everything needs to be built on each platform. We don't support true cross-platform builds. The autotester should handle this. -- /Jacob Carlborg
Dec 04 2013
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-12-04 05:29, Andrew Edwards wrote:

 Tried it but came across the following errors:


 andrews-mini:osx-release ace$ ~/create_dmd_release v2.065-b1
 --extras=$HOME/localextras-osx --archive-zip --archive-7z

 Cloning: git github.com:D-Programming-Language/dmd.git

 The authenticity of host 'github.com (192.30.252.128)' can't be
 established.

 RSA key fingerprint is 16:27:ac:a5:76:28:2d:36:63:1b:56:4d:eb:df:a6:48.

 Are you sure you want to continue connecting (yes/no)? yes

 Warning: Permanently added 'github.com,192.30.252.128' (RSA) to the list
 of known hosts.

 Permission denied (publickey).

 fatal: Could not read from remote repository.



 Please make sure you have the correct access rights

 and the repository exists.

 Couldn't do git protocol, falling back to 'https://'...
You need to have your SSH key uploaded to github. See: https://help.github.com/articles/generating-ssh-keys -- /Jacob Carlborg
Dec 04 2013
parent Andrew Edwards <ridimz yahoo.com> writes:
On 12/4/13, 3:21 AM, Jacob Carlborg wrote:
 On 2013-12-04 05:29, Andrew Edwards wrote:

 Couldn't do git protocol, falling back to 'https://'...
You need to have your SSH key uploaded to github. See: https://help.github.com/articles/generating-ssh-keys
I did this but it is not working for me. I probably did something wrong.
Dec 04 2013
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-12-03 15:25, Andrew Edwards wrote:

 I am working on a MacMini running OS X v10.9. I have Ubuntu 13.10 Server
 loaded in VirtualBox and will be using Jordi Sayol's script to build
 packages for linux/Windows and Jacob Carlborg script for OSX.

 Both of these scripts require an preexisting release zip and as of this
 moment, I am unaware of the steps to create that file. I will need some
 instructions on how to access and run the auto tester if that's what
 generates the zip or, if it's already automatically created,
 instructions on how to retrieve and stage it for the build.
Unfortunately there is no publicly available script (or similar) that creates the zip. It's Walter how always creates it. As others have said, the closest you get it probably the script Nick is working on: https://github.com/D-Programming-Language/installer/pull/24
 I will be setting up a tag on github today for the first beta release of
 2.065 (2.065-b1).

 Request your corporation/support in ensuring a smooth process.
Ideally there should be a "one-click" tool/script that manages the complete release of a new version, including beta and release candidates. With one command it should: * Create a git tag for all involved repositories * Instruct the autotester to build the compiler, libraries, installers and all tools for each platform based on the newly created tag * Assemble the cross-platform zip on one of the autotester machines * Upload zip and all installers to the Digital Mars FTP and/or Amazon or wherever these things are currently hosted * Generate the changelog from bugzilla. The overview/general information should already be present in the changelog at this time * Updates the website with the new changelog and links to the new release * Posts a message to the announce newsgroup, twitter, reddit? and any social networks we would like to post to Preferably there should be options to disable any of these steps make it possible to just assembly the compiler and tools but not make a proper release (update website post to social networks) -- /Jacob Carlborg
Dec 04 2013
parent reply Andrew Edwards <ridimz yahoo.com> writes:
On 12/4/13, 3:09 AM, Jacob Carlborg wrote:
 On 2013-12-03 15:25, Andrew Edwards wrote:

 I am working on a MacMini running OS X v10.9. I have Ubuntu 13.10 Server
 loaded in VirtualBox and will be using Jordi Sayol's script to build
 packages for linux/Windows and Jacob Carlborg script for OSX.

 Both of these scripts require an preexisting release zip and as of this
 moment, I am unaware of the steps to create that file. I will need some
 instructions on how to access and run the auto tester if that's what
 generates the zip or, if it's already automatically created,
 instructions on how to retrieve and stage it for the build.
Unfortunately there is no publicly available script (or similar) that creates the zip. It's Walter how always creates it. As others have said, the closest you get it probably the script Nick is working on: https://github.com/D-Programming-Language/installer/pull/24
 I will be setting up a tag on github today for the first beta release of
 2.065 (2.065-b1).

 Request your corporation/support in ensuring a smooth process.
Ideally there should be a "one-click" tool/script that manages the complete release of a new version, including beta and release candidates. With one command it should: * Create a git tag for all involved repositories
Working on that.
 * Instruct the autotester to build the compiler, libraries, installers
 and all tools for each platform based on the newly created tag
Will need access from Brad Roberts. Have yet to receive any kind of response to date.
 * Assemble the cross-platform zip on one of the autotester machines
ditto
 * Upload zip and all installers to the Digital Mars FTP and/or Amazon or
 wherever these things are currently hosted
ditto
 * Generate the changelog from bugzilla. The overview/general information
 should already be present in the changelog at this time
Will need help on this one.
 * Updates the website with the new changelog and links to the new release
ditto
 * Posts a message to the announce newsgroup, twitter, reddit? and any
 social networks we would like to post to
This should be separate. Previous experience (not my own) shows that some days are better posting than others.
 Preferably there should be options to disable any of these steps make it
 possible to just assembly the compiler and tools but not make a proper
 release (update website post to social networks)
Makes sense. Will be great when this is complete. In the interim though, we need to be able to perform manual semi-manual production so that the release cycle is not delayed.
Dec 04 2013
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-12-04 09:23, Andrew Edwards wrote:

 Working on that.

 * Instruct the autotester to build the compiler, libraries, installers
 and all tools for each platform based on the newly created tag
Will need access from Brad Roberts. Have yet to receive any kind of response to date.
 * Assemble the cross-platform zip on one of the autotester machines
ditto
 * Upload zip and all installers to the Digital Mars FTP and/or Amazon or
 wherever these things are currently hosted
ditto
I'm hoping it would be possible to create a tool/script without access to the all this.
 Will need help on this one.
I'm pretty sure Walter has a script for this.
 This should be separate. Previous experience (not my own) shows that
 some days are better posting than others.
Ok.
 Makes sense. Will be great when this is complete. In the interim though,
 we need to be able to perform manual semi-manual production so that the
 release cycle is not delayed.
We really need to push for the automate release process. Either you're doing it yourself or delegating to others. -- /Jacob Carlborg
Dec 04 2013
prev sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
04-Dec-2013 12:23, Andrew Edwards пишет:
 On 12/4/13, 3:09 AM, Jacob Carlborg wrote:
 On 2013-12-03 15:25, Andrew Edwards wrote:
[snip]
 * Generate the changelog from bugzilla. The overview/general information
 should already be present in the changelog at this time
Will need help on this one.
There is a tool for that. Nothing stellar but it does the job: https://github.com/D-Programming-Language/tools/blob/master/changed.d Also be sure to contact Andrej Mitrovic he is a mastermind behind nice changelogs we had for the last couple of releases. -- Dmitry Olshansky
Dec 04 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-12-04 20:28, Dmitry Olshansky wrote:

 There is a tool for that. Nothing stellar but it does the job:
 https://github.com/D-Programming-Language/tools/blob/master/changed.d
Great.
 Also be sure to contact Andrej Mitrovic he is a mastermind behind nice
 changelogs we had for the last couple of releases.
Yeah, that still needs to be manually written. -- /Jacob Carlborg
Dec 04 2013