www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - [Issue 11792] New: Investigate migrating to a meta repo

reply d-bugmail puremagic.com writes:
https://d.puremagic.com/issues/show_bug.cgi?id=11792

           Summary: Investigate migrating to a meta repo
           Product: D
           Version: D2
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: normal
          Priority: P2
         Component: DMD
        AssignedTo: nobody puremagic.com
        ReportedBy: andrei erdani.com



PST ---
Currently our github projects assume they live in the same directory and access
each other by using ../projname/stuff. We should formalize this by migrating
everything to a meta repo and making them as submodules.

The migration should be minimally disruptive and preserve history etc.

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Dec 21 2013
next sibling parent d-bugmail puremagic.com writes:
https://d.puremagic.com/issues/show_bug.cgi?id=11792


Martin Nowak <code dawg.eu> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |code dawg.eu



I think it's pretty simple.

- Create a 'DPL' or 'dlang' repo at D-Programming-Language
- Include all submodule
- Write a README.md on how to checkout and git init submodule everything
- Write github hooks that update the meta repo branches for every submodule
commit.
- Start to populate the meta repo with meta Makefiles :).

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Dec 21 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
https://d.puremagic.com/issues/show_bug.cgi?id=11792


Mathias LANG <pro.mathias.lang gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |pro.mathias.lang gmail.com



19:01:53 PST ---
That's simple to setup but tedious to use.
I made some tests in https://github.com/Geod24/MetaD
This is my feedback:

1) Starting from 1.8.2, git can track a branch, not only a commit. When you
checkout it's still set to a specific commit though, you need to pass the
`--remote` option to `git submodule update`
So I think we could make the repo point to the latest release, and our
contributor just need to keep in mind to passe `--remote`. This design void the
necessity of a hook.

2) I was wondering what the workflow would be with submodules.
I'm currently still testing and updating the README accordingly.
The most annoyance so far is that submodule HEAD gets detached quite easily.
The other problem is that your repository gets "dirty" by using --remote. I
think we should try to lower the need of git submodule commands, as using them
is tricky and often leads to confusion, like commiting in the wrong order. We
should also take care not to accept a PR with a modif to .gitmodules unless
it's a stable version update.

3) I'll take a look at the merging strategy soon. By default submodules are
just binary object, but git provide a way to merge/rebase them too as part of
the update process.

I'll keep experimenting with it, any comment / example of your normal workflow
is very welcome.

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jan 03 2014
prev sibling next sibling parent d-bugmail puremagic.com writes:
https://d.puremagic.com/issues/show_bug.cgi?id=11792


Vladimir Panteleev <thecybershadow gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |thecybershadow gmail.com



19:26:47 EET ---
This sounds similar to my D.git experiment:

http://forum.dlang.org/post/sgkzpaqkfihoajpehauo forum.dlang.org

The repository is automatically generated from the component repos' histories
and periodically updated (currently hourly).


 - Write a README.md on how to checkout and git init submodule everything


"git clone --recursive" will fetch all submodules as well, so there's no need
to use "git submodule init".


 1) Starting from 1.8.2, git can track a branch, not only a commit.


That would preclude importing history, and would make it unusable for
regression testing.


 We should also take care not to accept a PR with a modif to .gitmodules
 unless it's a stable version update.


Why would anyone send a pull request to the meta repository, unless it's e.g.
to add/fix makefiles?

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jan 04 2014
prev sibling next sibling parent d-bugmail puremagic.com writes:
https://d.puremagic.com/issues/show_bug.cgi?id=11792




08:01:51 PST ---


 This sounds similar to my D.git experiment:
 
 http://forum.dlang.org/post/sgkzpaqkfihoajpehauo forum.dlang.org
 
 The repository is automatically generated from the component repos' histories
 and periodically updated (currently hourly).
 


I am getting 404 on the mentioned repository (but D-dot-git works).



 1) Starting from 1.8.2, git can track a branch, not only a commit.


 
 That would preclude importing history, and would make it unusable for
 regression testing.
 


Submodules are not really meant for importing history. Actually, looking at the
man page kind of advise you against doing this:
https://www.kernel.org/pub/software/scm/git/docs/git-submodule.html
(2nd paragraph of the description).



 We should also take care not to accept a PR with a modif to .gitmodules
 unless it's a stable version update.


 
 Why would anyone send a pull request to the meta repository, unless it's e.g.
 to add/fix makefiles?


In my approach, that would be to update the pointed commit (you still have a
pointed commit with --remote).

Now, how do we handle testing PR that require Makefiles changes ? If I do a PR
to Phobos, and one to the superproject which modify Makefiles, how does the
auto-tester will possibly handle it ?

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jan 06 2014
prev sibling parent d-bugmail puremagic.com writes:
https://d.puremagic.com/issues/show_bug.cgi?id=11792




18:36:22 EET ---


 I am getting 404 on the mentioned repository (but D-dot-git works).


As mentioned later in the thread, the repository was moved to Bitbucket to
prevent GitHub from "helpfully" generating mention notifications:

https://bitbucket.org/cybershadow/d/commits


 Submodules are not really meant for importing history. Actually, looking at the
 man page kind of advise you against doing this:
 https://www.kernel.org/pub/software/scm/git/docs/git-submodule.html
 (2nd paragraph of the description).


I don't see how this applies to a meta-repository. It is also not possible to
meaningfully merge the history of several projects with a non-linear history.


 In my approach, that would be to update the pointed commit (you still have a
 pointed commit with --remote).


In what circumstances would that happen?


 Now, how do we handle testing PR that require Makefiles changes ? If I do a PR
 to Phobos, and one to the superproject which modify Makefiles, how does the
 auto-tester will possibly handle it ?


This problem is no different from any changes that require patches on multiple
components at once. It is currently not handled.

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jan 06 2014