www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Handling of compiler patches

reply "bearophile" <bearophileHUGS lycos.com> writes:
For DMD in GitHub there are more than one hundred open pull 
requests (currently 111). So far people have created more than 
one thousand patches for DMD (currently 1022):

https://github.com/D-Programming-Language/dmd/pulls

Among the list of open pull requests there are some bugs and 
enhancement requests that I'd really like to see applied & fixed 
in not too much time.

I am not a compiler writer, but I have two suggestions for Walter 
(or at least to open a discussion, if the things I am saying are 
wrong):

1) Currently the patches that Walter applies written by other 
people come mostly from the most recent ones. So the list of the 
open pull requests is managed almost as a stack (last in - fist 
out). I think this is not good.

I suggest to do the opposite, this means to manage that list more 
like a queue (first in - first out), so I suggest Walter to look 
for patches to apply starting from the oldest ones. This has the 
advantage that the patches don't get too much old, so they rot 
less.

An additional strategy to choose what patches to apply is to 
choose first the ones that fix bugs that are most significant. 
Bugs are not all equal, many bugs don't cause significant 
problems, while others are quite annoying. Significant bugs often 
are fixed by small patches. I see lot of not so significant 
patches applied, while I see patches that fix bugs that I hit 
every other day sleep in that list for weeks or months. If you 
want I will list some significant patches currently open.


2) I suggest Walter to increase a little more the openness of the 
way the front-end is developed. There are many kinds of compiler 
patches:

- Simple localized change;
- Complex localized change;
- Widespread trivial changes;
- Widespread complex changes;
- Back-end changes Vs front-end changes;
- That fixes a well defined and localized bug;
- That introduces a new feature, or a small part of a new feature.

The harder patches are better left to Walter, but I suggest 
Walter to eventually let few other people merge the simpler 
patches, after a review by one or two more persons.

In large projects, with skilled helper people, the leader 
eventually must learn to trust other people and to  delegate some 
of the work, building a pyramid of trust. I think that if Hara 
writes a simple 5-lines long patch for the DMD front-end, and Don 
or someone else equally skilled reviews and accepts that patch, 
there is no need for Walter to merge that little patch himself.

Bye,
bearophile
Jun 21 2012
next sibling parent reply "Andrea Fontana" <nospam example.com> writes:
+1

On Thursday, 21 June 2012 at 07:40:35 UTC, bearophile wrote:
 For DMD in GitHub there are more than one hundred open pull 
 requests (currently 111). So far people have created more than 
 one thousand patches for DMD (currently 1022):

 https://github.com/D-Programming-Language/dmd/pulls

 Among the list of open pull requests there are some bugs and 
 enhancement requests that I'd really like to see applied & 
 fixed in not too much time.

 I am not a compiler writer, but I have two suggestions for 
 Walter (or at least to open a discussion, if the things I am 
 saying are wrong):

 1) Currently the patches that Walter applies written by other 
 people come mostly from the most recent ones. So the list of 
 the open pull requests is managed almost as a stack (last in - 
 fist out). I think this is not good.

 I suggest to do the opposite, this means to manage that list 
 more like a queue (first in - first out), so I suggest Walter 
 to look for patches to apply starting from the oldest ones. 
 This has the advantage that the patches don't get too much old, 
 so they rot less.

 An additional strategy to choose what patches to apply is to 
 choose first the ones that fix bugs that are most significant. 
 Bugs are not all equal, many bugs don't cause significant 
 problems, while others are quite annoying. Significant bugs 
 often are fixed by small patches. I see lot of not so 
 significant patches applied, while I see patches that fix bugs 
 that I hit every other day sleep in that list for weeks or 
 months. If you want I will list some significant patches 
 currently open.


 2) I suggest Walter to increase a little more the openness of 
 the way the front-end is developed. There are many kinds of 
 compiler patches:

 - Simple localized change;
 - Complex localized change;
 - Widespread trivial changes;
 - Widespread complex changes;
 - Back-end changes Vs front-end changes;
 - That fixes a well defined and localized bug;
 - That introduces a new feature, or a small part of a new 
 feature.

 The harder patches are better left to Walter, but I suggest 
 Walter to eventually let few other people merge the simpler 
 patches, after a review by one or two more persons.

 In large projects, with skilled helper people, the leader 
 eventually must learn to trust other people and to  delegate 
 some of the work, building a pyramid of trust. I think that if 
 Hara writes a simple 5-lines long patch for the DMD front-end, 
 and Don or someone else equally skilled reviews and accepts 
 that patch, there is no need for Walter to merge that little 
 patch himself.

 Bye,
 bearophile

Jun 21 2012
parent Jacob Carlborg <doob me.com> writes:
On 2012-06-21 09:57, Andrea Fontana wrote:
 +1

 On Thursday, 21 June 2012 at 07:40:35 UTC, bearophile wrote:
 For DMD in GitHub there are more than one hundred open pull requests
 (currently 111). So far people have created more than one thousand
 patches for DMD (currently 1022):

 https://github.com/D-Programming-Language/dmd/pulls

 Among the list of open pull requests there are some bugs and
 enhancement requests that I'd really like to see applied & fixed in
 not too much time.


+1 -- /Jacob Carlborg
Jun 21 2012
prev sibling next sibling parent Trass3r <un known.com> writes:
 1) Currently the patches that Walter applies written by other people  
 come mostly from the most recent ones. I think this is not good.

Yep, github's default sort method sucks. They should really implement some sort of voting scheme or something like a "ready to merge" button for prolific contributors. Interestingly enough they even seem to have a 'votes' field for pull requests internally (http://develop.github.com/p/pulls.html) but I haven't seen any voting system yet.
 I suggest to do the opposite, this means to manage that list more like a  
 queue (first in - first out), so I suggest Walter to look for patches to  
 apply starting from the oldest ones. This has the advantage that the  
 patches don't get too much old, so they rot less.

+1 It's just annoying having to keep pull requests up-to-date forever. AND it definitely pisses contributors off. At least I won't submit anything anymore if the pulls just rot anyways. Also I really wonder if Walter's using the autotester. Most patches are ridiculously small and/or non-critical, so reviewing the code by hand can't take that long. And if you properly use the autotester instead of manually running the testsuite to verify it doesn't break anything you should really be able to merge a whole bunch of pulls in no time. Why is the process still so slow?
Jun 21 2012
prev sibling next sibling parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 21-Jun-12 11:40, bearophile wrote:
 For DMD in GitHub there are more than one hundred open pull requests
 (currently 111). So far people have created more than one thousand
 patches for DMD (currently 1022):

 https://github.com/D-Programming-Language/dmd/pulls

 Among the list of open pull requests there are some bugs and enhancement
 requests that I'd really like to see applied & fixed in not too much time.

 I am not a compiler writer, but I have two suggestions for Walter (or at
 least to open a discussion, if the things I am saying are wrong):

bandwidth I'd suggest to implement proper caching first. What I suggest is to finally organize multi-stage development process that loosely models military command chain :) OK. Walter is the general at HQ ( dlang master branch) no doubt. And some of core contributors are assigned as lieutenants for feature oriented branches. The structure I imagine is: Walter (master branch) |--- CTFE feature branch, Don Clugston |--- Backend bugfix & feature branch, Brad Roberts |--- Language bugfix & feature, Kenji Hara / Daniel Murphy |--- Miscellaneous & ICE bugs, Martin Nowak (we can start with a few less or more of these) *Keep in mind these assignments are purely my speculation. And sure thing it doesn't prevent anybody from working on "someone else" feature branch. All these assignments mean is responsibility for pulling stuff and forming separate (or on the contrary - cumulative) pulls on them for HQ. Also allows them to polish and refactor things on the way up. Then the model works as follows: HQ repo gets pull requests prepared by lieutenants *only* thus *insuring* it's not some "for the **ck of it pull request" and lifting ton of work for reviewing it. Pulls for features are targeted at specific repo thus *insuring* there is at least one "major" guy how it supposed to work in this field (and in the know of proper DMD's conventions). All dirty hack pulls get stopped/re-written somewhere here. Also all of lieutenants repos should be kept rebased on top of master (HQ) at all times. Thus Walter may even choose to cherry pick some commits skipping the usual pull request process. -- Dmitry Olshansky
Jun 21 2012
prev sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Thu, 21 Jun 2012 10:51:03 -0700, Dmitry Olshansky  
<dmitry.olsh gmail.com> wrote:

 On 21-Jun-12 11:40, bearophile wrote:
 For DMD in GitHub there are more than one hundred open pull requests
 (currently 111). So far people have created more than one thousand
 patches for DMD (currently 1022):

 https://github.com/D-Programming-Language/dmd/pulls

 Among the list of open pull requests there are some bugs and enhancement
 requests that I'd really like to see applied & fixed in not too much  
 time.

 I am not a compiler writer, but I have two suggestions for Walter (or at
 least to open a discussion, if the things I am saying are wrong):

bandwidth I'd suggest to implement proper caching first. What I suggest is to finally organize multi-stage development process that loosely models military command chain :) OK. Walter is the general at HQ ( dlang master branch) no doubt. And some of core contributors are assigned as lieutenants for feature oriented branches. The structure I imagine is: Walter (master branch) |--- CTFE feature branch, Don Clugston |--- Backend bugfix & feature branch, Brad Roberts |--- Language bugfix & feature, Kenji Hara / Daniel Murphy |--- Miscellaneous & ICE bugs, Martin Nowak (we can start with a few less or more of these) *Keep in mind these assignments are purely my speculation. And sure thing it doesn't prevent anybody from working on "someone else" feature branch. All these assignments mean is responsibility for pulling stuff and forming separate (or on the contrary - cumulative) pulls on them for HQ. Also allows them to polish and refactor things on the way up. Then the model works as follows: HQ repo gets pull requests prepared by lieutenants *only* thus *insuring* it's not some "for the **ck of it pull request" and lifting ton of work for reviewing it. Pulls for features are targeted at specific repo thus *insuring* there is at least one "major" guy how it supposed to work in this field (and in the know of proper DMD's conventions). All dirty hack pulls get stopped/re-written somewhere here. Also all of lieutenants repos should be kept rebased on top of master (HQ) at all times. Thus Walter may even choose to cherry pick some commits skipping the usual pull request process.

I like this. It neatly solves a whole lot of issues. Also, this is precisely how Linus INTENDED Git to be used. -- Adam Wilson IRC: LightBender Project Coordinator The Horizon Project http://www.thehorizonproject.org/
Jun 21 2012