www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Deimos need some works

reply "bioinfornatics" <bioinfornatics fedoraproject.org> writes:
to package deimos project that is not really easy
   each module has his repo
   each module should to be builded not in same way

That is a good project but it need more standard

firstly you can to provide a makefile by example this one
https://github.com/bioinfornatics/MakefileForD like this you
provide a uniq way to build it and tosupport all compiller. All
os provide a make build system somewhere.

You can choose an other way the main idea that is to help
packager to package this huge project. Because once time is
packaged you need to maintain it!

Please do anything in this way

thanks
Jul 17 2013
next sibling parent "bioinfornatics" <bioinfornatics fedoraproject.org> writes:
use a repo named deimos in more of projectname and to use git 
submodule + a build system to build all deimos module sound like 
a good thing
Jul 17 2013
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, July 17, 2013 22:40:58 bioinfornatics wrote:
 to package deimos project that is not really easy
 each module has his repo
 each module should to be builded not in same way

While I don't necessarily disagree that how deimos is done needs some work, I don't see why you would ever need to package deimos. It contains disparate libraries that aren't necessarily intended to be used together at all (and potentially can't be - e.g. if one were Linux-specific and another were Windows-specific). Having the deimos project is useful, because it gives us one place to look for D bindings for C libraries, but the libraries in there are otherwise completely unrelated to each other and thus packaging them together really doesn't make sense IMHO. Several of them are set up to use dub though, which makes grabbing them much easier. I think that my biggest gripe with how Deimos is set up is that the folks who manage the projects in there aren't in control of them. So, they can't merge anything themselves or do other git commands such as tagging or branching. Some of the Phobos devs do have merge rights, but that doesn't make it so that they can do other stuff like tagging or branching. So, the result is that only a few people (like Walter and probably Andrei) can do all of the various stuff to those repos that a repo maintainer might want to do (like branching), and yet the only reason that they're grouped together in the first place is searchability. So, we've gained the searchability at the cost of flexibility. But I don't know how to fix that aside maybe from Walter giving full permissions/access to a deimos repo to its primary maintainer (without giving them permissions to other deimos repos), but I don't know what github will or won't allow you to do in that regard. - Jonathan M Davis
Jul 17 2013
prev sibling next sibling parent "bioinfornatics" <bioinfornatics fedoraproject.org> writes:
On Wednesday, 17 July 2013 at 21:01:16 UTC, Jonathan M Davis 
wrote:
 On Wednesday, July 17, 2013 22:40:58 bioinfornatics wrote:
 to package deimos project that is not really easy
 each module has his repo
 each module should to be builded not in same way

While I don't necessarily disagree that how deimos is done needs some work, I don't see why you would ever need to package deimos. It contains disparate libraries that aren't necessarily intended to be used together at all (and potentially can't be - e.g. if one were Linux-specific and another were Windows-specific). Having the deimos project is useful, because it gives us one place to look for D bindings for C libraries, but the libraries in there are otherwise completely unrelated to each other and thus packaging them together really doesn't make sense IMHO. Several of them are set up to use dub though, which makes grabbing them much easier. I think that my biggest gripe with how Deimos is set up is that the folks who manage the projects in there aren't in control of them. So, they can't merge anything themselves or do other git commands such as tagging or branching. Some of the Phobos devs do have merge rights, but that doesn't make it so that they can do other stuff like tagging or branching. So, the result is that only a few people (like Walter and probably Andrei) can do all of the various stuff to those repos that a repo maintainer might want to do (like branching), and yet the only reason that they're grouped together in the first place is searchability. So, we've gained the searchability at the cost of flexibility. But I don't know how to fix that aside maybe from Walter giving full permissions/access to a deimos repo to its primary maintainer (without giving them permissions to other deimos repos), but I don't know what github will or won't allow you to do in that regard. - Jonathan M Davis

I agree with your point of view. I package for a linux distro so maintain one package which sync each module to current stable is more easy to maintain versus maintain x module . I do not know dub i will take a look if is an irtaller system for user or it could to be used to do a package
Jul 17 2013
prev sibling next sibling parent "bioinfornatics" <bioinfornatics fedoraproject.org> writes:
example:
if they are one meta project :
- create 1 package with sub-package (or meta-package)
- git pull or downlobd latest archive once time
- building in 1 shot whole linux deimos project
- push 1 update

package separately each package will eat all my free time to 
maintain it
Jul 17 2013
prev sibling next sibling parent "bioinfornatics" <bioinfornatics fedoraproject.org> writes:
Firstly dub is not into fedora

# yum provides /usr/bin/dub
No matches found

# yum provides */dub
No matches found

So i need to find the project build it provide a package submit a 
review to go to official repo before to start to package one 
module
Jul 17 2013
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, July 17, 2013 23:10:00 bioinfornatics wrote:
 I package for a linux distro so maintain one package which sync
 each module to current stable is more easy to maintain versus
 maintain x module .
 I do not know dub i will take a look if is an irtaller system for
 user or it could to be used to do a package

But the deimos project just provides bindings, not libraries, so the only reason to have them is for development, and there's generally no reason to build them separate from your program. As such, I see little point in including them in a distro's packaging system, not when we're already looking to have a package manager for D development packages. And I really don't agree with the idea of grouping them all together to be grabbed together anyway, because they're all completely unrelated. You wouldn't create a single rpm or deb for all of the dev packages on a Linux distro would you? Granted, that would be larger than packaging deimos (even if deimos had bindings for all of the same dev packages), because it would include more than just bindings, but it's the same concept. This is a prime example of where a package manager for D makes sense and distro packages do not, and many of the deimos projects are included in the dub registry, so whether or not dub ever becomes either the actual or de facto standard for D packages, it's at least usable at this point and is definitely gaining adoption. - Jonathan M Davis
Jul 17 2013
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, July 17, 2013 23:25:23 bioinfornatics wrote:
 Firstly dub is not into fedora
 
 # yum provides /usr/bin/dub
 No matches found
 
 # yum provides */dub
 No matches found
 
 So i need to find the project build it provide a package submit a
 review to go to official repo before to start to package one
 module

https://github.com/rejectedsoftware/dub - Jonathan M Davis
Jul 17 2013
prev sibling next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 07/17/2013 11:00 PM, Jonathan M Davis wrote:
 I think that my biggest gripe with how Deimos is set up is that the folks 
 who manage the projects in there aren't in control of them. So, they can't 
 merge anything themselves or do other git commands such as tagging or 
 branching.

Is it not possible to set up something where (i) Deimos library devs create a 'home repo' for their project, which can be a personal repo, a group repo, or even something not on GitHub if they prefer (ii) Deimos project repos just do a regular, automated clone/pull from the specified home repo.
Jul 17 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, July 18, 2013 08:10:35 Joseph Rushton Wakeling wrote:
 On 07/17/2013 11:00 PM, Jonathan M Davis wrote:
 I think that my biggest gripe with how Deimos is set up is that the folks
 who manage the projects in there aren't in control of them. So, they can't
 merge anything themselves or do other git commands such as tagging or
 branching.

Is it not possible to set up something where (i) Deimos library devs create a 'home repo' for their project, which can be a personal repo, a group repo, or even something not on GitHub if they prefer

As it stands, Deimos is a github organization (named D-Programming-Deimos to be precise) and for anything to be in Deimos, it must be in that github organization, which means that the repo is owned by that organization, and only members of that organization have any permissions for it (and the permissions can vary from member to member). So, while someone can certainly manage D bindings for a C library in their own repo, it's not really part of Deimos unless it's in a repo in D-Programming-Deimos. We could certainly change what it means for something to be in deimos (e.g. Jakob Ovrum suggested recently that we turn it into a wiki page which refers to repos which are not part of any particular github organization and which could belong to any developer, not just those picked by Walter or the other main dmd/druntime/Phobos devs), but at present, what deimos is is defined by the github organization D-Programming-Deimos.
   (ii) Deimos project repos just do a regular, automated clone/pull from the
 specified home repo.

It might be possible to create a tool to do something like that, but AFAIK, nothing like that currently exists. But if you were going to treat a particular person's repo as essentially the official one (as it's where the deimos one gets its stuff), then you probably might as well just take the approach of giving that person full access to the deimos repo in question rather than bothering with such a tool. I really don't know what the best approach would be for deimos. What we have currently is working on some level, but clearly there's some room for improvement. I think that there's plenty of room for suggestions. - Jonathan M Davis
Jul 17 2013
prev sibling next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 07/18/2013 08:35 AM, Jonathan M Davis wrote:
 As it stands, Deimos is a github organization (named D-Programming-Deimos to 
 be precise) and for anything to be in Deimos, it must be in that github 
 organization, which means that the repo is owned by that organization, and 
 only members of that organization have any permissions for it (and the 
 permissions can vary from member to member).

Yes, I understand that.
 So, while someone can certainly manage D bindings for a C library in their
 own repo, it's not really part of Deimos unless it's in a repo in
 D-Programming-Deimos.

Yes, but there's a difference between "an up-to-date clone of the project repo is stored in D-Programming-Deimos" versus "the merging and pull requests and other project management have to be carried out in D-Programming-Deimos."
   (ii) Deimos project repos just do a regular, automated clone/pull from the
 specified home repo.

It might be possible to create a tool to do something like that, but AFAIK, nothing like that currently exists. But if you were going to treat a particular person's repo as essentially the official one (as it's where the deimos one gets its stuff), then you probably might as well just take the approach of giving that person full access to the deimos repo in question rather than bothering with such a tool.

Depends on what options GitHub gives you. If you can't give someone control over a single repo in Deimos without giving them effective control over all the others, then independent project repos + clone to D-Programming-Deimos might be an OK way to organize it. Seems better to me than a Wiki page list, because this way you have direct access to the code from the Deimos project list. It also gives Deimos admins a way to easily exclude projects that are proving problematic -- just delete the Deimos clone of the project repo.
Jul 18 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, July 18, 2013 09:09:09 Joseph Rushton Wakeling wrote:
 Depends on what options GitHub gives you.  If you can't give someone control
 over a single repo in Deimos without giving them effective control over all
 the others, then independent project repos + clone to D-Programming-Deimos
 might be an OK way to organize it.

I'm sure that it's possible to give per-repo permissions, because I have full commit access to Phobos and druntime but no commit access whatsoever to dmd, and they're all in the same github org. But I don't know how fine-grained the permissions controls are or how much they can ultimately do. - Jonathan M Davis
Jul 18 2013
prev sibling next sibling parent "bioinfornatics" <bioinfornatics fedoraproject.org> writes:
after taking a look about dub source code. I do not think dun is
the way to go. Maybe i misunderstanding something.  This project
do not follow FHS: https://wiki.linuxfoundation.org/en/FHS.

at line 73
https://github.com/rejectedsoftware/dub/blob/master/source/dub/dub.d#L73
dub path are set to "/var/lib/dub/"

and at line 195
https://github.com/rejectedsoftware/dub/blob/master/source/dub/dub.d#L195
   install path is set to "/var/lib/dub/packages/"

Any serious linux distro should to accept this to their official
repo. Then all d project which provides only dub as installer
will fail grow.

Maybe i misunderstood somewhere just show me
Jul 18 2013
prev sibling next sibling parent "bioinfornatics" <bioinfornatics fedoraproject.org> writes:
On Thursday, 18 July 2013 at 07:36:48 UTC, bioinfornatics wrote:
 after taking a look about dub source code. I do not think dun is
 the way to go. Maybe i misunderstanding something.  This project
 do not follow FHS: https://wiki.linuxfoundation.org/en/FHS.

 at line 73
 https://github.com/rejectedsoftware/dub/blob/master/source/dub/dub.d#L73
 dub path are set to "/var/lib/dub/"

 and at line 195
 https://github.com/rejectedsoftware/dub/blob/master/source/dub/dub.d#L195
   install path is set to "/var/lib/dub/packages/"

 Any serious linux distro should to accept this to their official
 repo. Then all d project which provides only dub as installer
 will fail grow.

 Maybe i misunderstood somewhere just show me

Any serious linux distro should not* to accept this to their official repo.
Jul 18 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, July 18, 2013 09:37:52 bioinfornatics wrote:
 On Thursday, 18 July 2013 at 07:36:48 UTC, bioinfornatics wrote:
 after taking a look about dub source code. I do not think dun is
 the way to go. Maybe i misunderstanding something.  This project
 do not follow FHS: https://wiki.linuxfoundation.org/en/FHS.
 
 at line 73
 https://github.com/rejectedsoftware/dub/blob/master/source/dub/dub.d#L73
 dub path are set to "/var/lib/dub/"
 
 and at line 195
 https://github.com/rejectedsoftware/dub/blob/master/source/dub/dub.d#L195
 
   install path is set to "/var/lib/dub/packages/"
 
 Any serious linux distro should to accept this to their official
 repo. Then all d project which provides only dub as installer
 will fail grow.
 
 Maybe i misunderstood somewhere just show me

Any serious linux distro should not* to accept this to their official repo.

Except that none of this is supposed to even be in a Linux distro. None of it would ever be in a Linux package or be installed in /usr or /var or any of that. These are development packages, not actual programs, so there's no reason to even have them installed by the distro in the main system. You only need them to build your own software. Now, that's a bit different if you're dealing with shared libraries (which we'll have soon), and at that point, dub (or any other D package manager) would potentially have to worry about looking at what's installed on the system in some standard manner, and shared D libraries would be installed with Linux packages. But for anything you'd be doing now, that's completely irrelevant, because there's no benefit in installing any of it in the system as a whole. This is particularly true for deimos projects which don't even generate static libraries. They've basically just the D equivalent of a bunch of include files. So, I don't think that discussing installing deimos with a linux package (such as an rpm or deb) makes any sense at all. At this point, the only D stuff that it makes sense to make Linux packages for is applications. In the future, it'll make sense for shared libraries, and package tools like dub will have to adapt deal with that, but for the moment, since shared library support in D is in its infancy, it's pretty much a non-issue. We _could_ make linux packages for static libraries, but there's really not much benefit in doing so, particularly when D compiles so quickly, and you pretty much have to have built everything with the same version of dmd. But not even that applies to deimos, since it just provides bindings. - Jonathan M Davis
Jul 18 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, July 18, 2013 01:59:54 Jonathan M Davis wrote:
 On Thursday, July 18, 2013 09:37:52 bioinfornatics wrote:
 On Thursday, 18 July 2013 at 07:36:48 UTC, bioinfornatics wrote:
 after taking a look about dub source code. I do not think dun is
 the way to go. Maybe i misunderstanding something.  This project
 do not follow FHS: https://wiki.linuxfoundation.org/en/FHS.
 
 at line 73
 https://github.com/rejectedsoftware/dub/blob/master/source/dub/dub.d#L73
 dub path are set to "/var/lib/dub/"
 
 and at line 195
 https://github.com/rejectedsoftware/dub/blob/master/source/dub/dub.d#L19
 5
 
   install path is set to "/var/lib/dub/packages/"
 
 Any serious linux distro should to accept this to their official
 repo. Then all d project which provides only dub as installer
 will fail grow.
 
 Maybe i misunderstood somewhere just show me

Any serious linux distro should not* to accept this to their official repo.

Except that none of this is supposed to even be in a Linux distro. None of it would ever be in a Linux package or be installed in /usr or /var or any of that. These are development packages, not actual programs, so there's no reason to even have them installed by the distro in the main system. You only need them to build your own software. Now, that's a bit different if you're dealing with shared libraries (which we'll have soon), and at that point, dub (or any other D package manager) would potentially have to worry about looking at what's installed on the system in some standard manner, and shared D libraries would be installed with Linux packages. But for anything you'd be doing now, that's completely irrelevant, because there's no benefit in installing any of it in the system as a whole. This is particularly true for deimos projects which don't even generate static libraries. They've basically just the D equivalent of a bunch of include files. So, I don't think that discussing installing deimos with a linux package (such as an rpm or deb) makes any sense at all. At this point, the only D stuff that it makes sense to make Linux packages for is applications. In the future, it'll make sense for shared libraries, and package tools like dub will have to adapt deal with that, but for the moment, since shared library support in D is in its infancy, it's pretty much a non-issue. We _could_ make linux packages for static libraries, but there's really not much benefit in doing so, particularly when D compiles so quickly, and you pretty much have to have built everything with the same version of dmd. But not even that applies to deimos, since it just provides bindings.

I would also point out that a number of other languages (e.g. ruby, haskell, and perl) have package managers for downloading code to use and which may or may not be integrated into Linux packages. And it's that sort of thing that we're looking to emulate. Then you can just build your program without having to track all of the dependencies down. The D package manager would just do that for you. - Jonathan M Davis
Jul 18 2013
prev sibling next sibling parent "bioinfornatics" <bioinfornatics fedoraproject.org> writes:
firstly fedora provide D shared lib since 2 years and static lib 
is not allowed.
We want to get one lib and share it to each application.

Secondly deimos provide include file these should not to be embed 
by each application which need it. that duplicate file in more 
some same file will not be at same revision that is not linux 
philosophy.

Third, about package manager:
  - this is for user not distro linux
  - setuptools from python use FHS
  - cpan from perl it is used by user not by packager
  - rvm (ruby ) ditto

so they are 2 things:
  - builder should to build
  - package manager should to package

problem here that is dub is showed to be the universal D way to 
build and manage packaging. that is really a problem to continue 
to support D project into offcial linux repository.

I can package dub and user use it for any reason but i need to be 
able to build and install a project by following FHS and other 
linux rules
Jul 18 2013
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Thursday, 18 July 2013 at 09:00:23 UTC, Jonathan M Davis wrote:
 ...

There are two distinct issues here: 1) The fact that stuff dub installs stuff without the help of package manager does not mean it can ignore FHS guidelines - otherwise one will have a hard time adopting dub itself into repository. At the very least its place is /opt/dub but I'd actually expect it to split stuff over /usr/local. Will likely need to have a look at it if ever want to create such package. 2) Convenience for packager. However, I don't think this is much of an issue. One can always clone Deimos repository and enhance DFLAGS accordingly in a packaging script. As Deimos modules are used only during the build process and are not supposed to be provided to the end user, it is just a minor maintenance inconvenience. Nothing critical. There are, of course, library packages like vibe.d which do include full source code and require availability of Deimos bindings to be used. But I think packing a bindings copy with the library itself is a proper way to go here, unless they actually get exposed to programmer as part of library API (have not met a single example so far).
Jul 18 2013
prev sibling next sibling parent "Wyatt" <wyatt.epp gmail.com> writes:
On Thursday, 18 July 2013 at 09:47:27 UTC, bioinfornatics wrote:
 problem here that is dub is showed to be the universal D way to 
 build and manage packaging. that is really a problem to 
 continue to support D project into offcial linux repository.

adopt it officially, but for the time being, it's third-party code maintained as part of vibe.d.
 I can package dub and user use it for any reason but i need to 
 be able to build and install a project by following FHS and 
 other linux rules

Your job as a distro package maintainer is to package software for your distro; make sure the package interacts well with the system, such as by making sure it builds with the proper CFLAGS, works with --as-needed, installs files to the proper place, etc.; and, within a reasonable degree of certainty, ensure it won't break users' systems. If it's not installing files in the right places, patch it. Then, contact the maintainers to see if you can have that patch upstreamed. It's not a project's job to package software-- those that manage to do it and do it well are fortunate; in my experience, many of them are awful at it (especially LSI). Sometimes you'll need to help them understand what distros need so they can keep distro needs in mind going forward. It's a two-way street. -Wyatt
Jul 18 2013
prev sibling next sibling parent "bioinfornatics" <bioinfornatics fedoraproject.org> writes:
 Your job as a distro package maintainer is to package software 
 for your distro; make sure the package interacts well with the 
 system, such as by making sure it builds with the proper 
 CFLAGS, works with --as-needed, installs files to the proper 
 place, etc.; and, within a reasonable degree of certainty, 
 ensure it won't break users' systems.

 If it's not installing files in the right places, patch it.  
 Then, contact the maintainers to see if you can have that patch 
 upstreamed.  It's not a project's job to package software-- 
 those that manage to do it and do it well are fortunate; in my 
 experience, many of them are awful at it (especially LSI).  
 Sometimes you'll need to help them understand what distros need 
 so they can keep distro needs in mind going forward.  It's a 
 two-way street.

 -Wyatt

I know my jobs ;-) if we can start something which satisfy everyone that is good (dev / user / distro ). I exposing what could to be a problem. If D lib start to be a nightmare to package that will not help to continue to package it. Lib should not to embed deimos import file we wills pent lot of time to patch every build system. So the original question about deimos is closed i will package - vibe.d ( building manually erf ... ), - corresponding deimos dependencies - dub to be used as cpan
Jul 18 2013
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, July 18, 2013 11:47:26 bioinfornatics wrote:
 firstly fedora provide D shared lib since 2 years and static lib
 is not allowed.
 We want to get one lib and share it to each application.

How on earth has it been providing shared libraries for any D packages when we only just started supporting shared libraries with the most recent release? Any shared library prior to now would definitely have been buggy, and even now, it's highly experimental. No one who wants a low chance of bugs should be using shared libraries in D yet, let alone for the last 2 years. - Jonathan M Davis
Jul 18 2013
prev sibling parent "Rikki Cattermole" <alphaglosined gmail.com> writes:
On Thursday, 18 July 2013 at 06:10:45 UTC, Joseph Rushton 
Wakeling wrote:
 On 07/17/2013 11:00 PM, Jonathan M Davis wrote:
 I think that my biggest gripe with how Deimos is set up is 
 that the folks who manage the projects in there aren't in 
 control of them. So, they can't merge anything themselves or 
 do other git commands such as tagging or branching.

Is it not possible to set up something where (i) Deimos library devs create a 'home repo' for their project, which can be a personal repo, a group repo, or even something not on GitHub if they prefer (ii) Deimos project repos just do a regular, automated clone/pull from the specified home repo.

An alternative method is to use git submodules and let the people who wish to maintain bindings to have it as a submodule of a deimos repo. That way they are linked in nice forms from a central location.
Jul 19 2013