www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Deimos need some works

reply "bioinfornatics" <bioinfornatics fedoraproject.org> writes:
to package deimos project that is not really easy
   each module has his repo
   each module should to be builded not in same way

That is a good project but it need more standard

firstly you can to provide a makefile by example this one
https://github.com/bioinfornatics/MakefileForD like this you
provide a uniq way to build it and tosupport all compiller. All
os provide a make build system somewhere.

You can choose an other way the main idea that is to help
packager to package this huge project. Because once time is
packaged you need to maintain it!

Please do anything in this way

thanks
Jul 17 2013
next sibling parent "bioinfornatics" <bioinfornatics fedoraproject.org> writes:
use a repo named deimos in more of projectname and to use git 
submodule + a build system to build all deimos module sound like 
a good thing
Jul 17 2013
prev sibling next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, July 17, 2013 22:40:58 bioinfornatics wrote:

 to package deimos project that is not really easy
 each module has his repo
 each module should to be builded not in same way


While I don't necessarily disagree that how deimos is done needs some work, I 
don't see why you would ever need to package deimos. It contains disparate 
libraries that aren't necessarily intended to be used together at all (and 
potentially can't be - e.g. if one were Linux-specific and another were 
Windows-specific). Having the deimos project is useful, because it gives us one 
place to look for D bindings for C libraries, but the libraries in there are 
otherwise completely unrelated to each other and thus packaging them together 
really doesn't make sense IMHO.

Several of them are set up to use dub though, which makes grabbing them much 
easier. I think that my biggest gripe with how Deimos is set up is that the 
folks who manage the projects in there aren't in control of them. So, they 
can't merge anything themselves or do other git commands such as tagging or 
branching. Some of the Phobos devs do have merge rights, but that doesn't make 
it so that they can do other stuff like tagging or branching. So, the result is 
that only a few people (like Walter and probably Andrei) can do all of the 
various stuff to those repos that a repo maintainer might want to do (like 
branching), and yet the only reason that they're grouped together in the first 
place is searchability. So, we've gained the searchability at the cost of 
flexibility. But I don't know how to fix that aside maybe from Walter giving 
full permissions/access to a deimos repo to its primary maintainer (without 
giving them permissions to other deimos repos), but I don't know what github 
will or won't allow you to do in that regard.

- Jonathan M Davis
Jul 17 2013
parent reply "bioinfornatics" <bioinfornatics fedoraproject.org> writes:
On Wednesday, 17 July 2013 at 21:01:16 UTC, Jonathan M Davis 
wrote:

 On Wednesday, July 17, 2013 22:40:58 bioinfornatics wrote:

 to package deimos project that is not really easy
 each module has his repo
 each module should to be builded not in same way


 While I don't necessarily disagree that how deimos is done 
 needs some work, I
 don't see why you would ever need to package deimos. It 
 contains disparate
 libraries that aren't necessarily intended to be used together 
 at all (and
 potentially can't be - e.g. if one were Linux-specific and 
 another were
 Windows-specific). Having the deimos project is useful, because 
 it gives us one
 place to look for D bindings for C libraries, but the libraries 
 in there are
 otherwise completely unrelated to each other and thus packaging 
 them together
 really doesn't make sense IMHO.

 Several of them are set up to use dub though, which makes 
 grabbing them much
 easier. I think that my biggest gripe with how Deimos is set up 
 is that the
 folks who manage the projects in there aren't in control of 
 them. So, they
 can't merge anything themselves or do other git commands such 
 as tagging or
 branching. Some of the Phobos devs do have merge rights, but 
 that doesn't make
 it so that they can do other stuff like tagging or branching. 
 So, the result is
 that only a few people (like Walter and probably Andrei) can do 
 all of the
 various stuff to those repos that a repo maintainer might want 
 to do (like
 branching), and yet the only reason that they're grouped 
 together in the first
 place is searchability. So, we've gained the searchability at 
 the cost of
 flexibility. But I don't know how to fix that aside maybe from 
 Walter giving
 full permissions/access to a deimos repo to its primary 
 maintainer (without
 giving them permissions to other deimos repos), but I don't 
 know what github
 will or won't allow you to do in that regard.

 - Jonathan M Davis


I agree with your point of view.

I package for a linux distro so maintain one package which sync 
each module to current stable is more easy to maintain versus 
maintain x module .
I do not know dub i will take a look if is an irtaller system for 
user or it could to be used to do a package
Jul 17 2013
next sibling parent reply "bioinfornatics" <bioinfornatics fedoraproject.org> writes:
example:
if they are one meta project :
- create 1 package with sub-package (or meta-package)
- git pull or downlobd latest archive once time
- building in 1 shot whole linux deimos project
- push 1 update

package separately each package will eat all my free time to 
maintain it
Jul 17 2013
parent reply "bioinfornatics" <bioinfornatics fedoraproject.org> writes:
Firstly dub is not into fedora


No matches found


No matches found

So i need to find the project build it provide a package submit a 
review to go to official repo before to start to package one 
module
Jul 17 2013
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, July 17, 2013 23:25:23 bioinfornatics wrote:

 Firstly dub is not into fedora
 

 No matches found
 

 No matches found
 
 So i need to find the project build it provide a package submit a
 review to go to official repo before to start to package one
 module


https://github.com/rejectedsoftware/dub

- Jonathan M Davis
Jul 17 2013
parent reply "bioinfornatics" <bioinfornatics fedoraproject.org> writes:
after taking a look about dub source code. I do not think dun is
the way to go. Maybe i misunderstanding something.  This project
do not follow FHS: https://wiki.linuxfoundation.org/en/FHS.

at line 73
https://github.com/rejectedsoftware/dub/blob/master/source/dub/dub.d#L73
dub path are set to "/var/lib/dub/"

and at line 195
https://github.com/rejectedsoftware/dub/blob/master/source/dub/dub.d#L195
   install path is set to "/var/lib/dub/packages/"

Any serious linux distro should to accept this to their official
repo. Then all d project which provides only dub as installer
will fail grow.

Maybe i misunderstood somewhere just show me
Jul 18 2013
parent reply "bioinfornatics" <bioinfornatics fedoraproject.org> writes:
On Thursday, 18 July 2013 at 07:36:48 UTC, bioinfornatics wrote:

 after taking a look about dub source code. I do not think dun is
 the way to go. Maybe i misunderstanding something.  This project
 do not follow FHS: https://wiki.linuxfoundation.org/en/FHS.

 at line 73
 https://github.com/rejectedsoftware/dub/blob/master/source/dub/dub.d#L73
 dub path are set to "/var/lib/dub/"

 and at line 195
 https://github.com/rejectedsoftware/dub/blob/master/source/dub/dub.d#L195
   install path is set to "/var/lib/dub/packages/"

 Any serious linux distro should to accept this to their official
 repo. Then all d project which provides only dub as installer
 will fail grow.

 Maybe i misunderstood somewhere just show me


Any serious linux distro should not* to accept this to their
official repo.
Jul 18 2013
next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, July 18, 2013 09:37:52 bioinfornatics wrote:

 On Thursday, 18 July 2013 at 07:36:48 UTC, bioinfornatics wrote:

 after taking a look about dub source code. I do not think dun is
 the way to go. Maybe i misunderstanding something.  This project
 do not follow FHS: https://wiki.linuxfoundation.org/en/FHS.
 
 at line 73
 https://github.com/rejectedsoftware/dub/blob/master/source/dub/dub.d#L73
 dub path are set to "/var/lib/dub/"
 
 and at line 195
 https://github.com/rejectedsoftware/dub/blob/master/source/dub/dub.d#L195
 
   install path is set to "/var/lib/dub/packages/"
 
 Any serious linux distro should to accept this to their official
 repo. Then all d project which provides only dub as installer
 will fail grow.
 
 Maybe i misunderstood somewhere just show me


 
 Any serious linux distro should not* to accept this to their
 official repo.


Except that none of this is supposed to even be in a Linux distro. None of it 
would ever be in a Linux package or be installed in /usr or /var or any of 
that. These are development packages, not actual programs, so there's no 
reason to even have them installed by the distro in the main system. You only 
need them to build your own software.

Now, that's a bit different if you're dealing with shared libraries (which 
we'll have soon), and at that point, dub (or any other D package manager) 
would potentially have to worry about looking at what's installed on the 
system in some standard manner, and shared D libraries would be installed with 
Linux packages. But for anything you'd be doing now, that's completely 
irrelevant, because there's no benefit in installing any of it in the system as 
a whole. This is particularly true for deimos projects which don't even 
generate static libraries. They've basically just the D equivalent of a bunch 
of include files.

So, I don't think that discussing installing deimos with a linux package (such 
as an rpm or deb) makes any sense at all. At this point, the only D stuff that 
it makes sense to make Linux packages for is applications. In the future, 
it'll make sense for shared libraries, and package tools like dub will have to 
adapt deal with that, but for the moment, since shared library support in D is 
in its infancy, it's pretty much a non-issue.

We _could_ make linux packages for static libraries, but there's really not 
much benefit in doing so, particularly when D compiles so quickly, and you 
pretty much have to have built everything with the same version of dmd. But 
not even that applies to deimos, since it just provides bindings.

- Jonathan M Davis
Jul 18 2013
parent "Dicebot" <public dicebot.lv> writes:
On Thursday, 18 July 2013 at 09:00:23 UTC, Jonathan M Davis wrote:

 ...


There are two distinct issues here:

1) The fact that stuff dub installs stuff without the help of 
package manager does not mean it can ignore FHS guidelines - 
otherwise one will have a hard time adopting dub itself into 
repository. At the very least its place is /opt/dub but I'd 
actually expect it to split stuff over /usr/local. Will likely 
need to have a look at it if ever want to create such package.

2) Convenience for packager. However, I don't think this is much 
of an issue. One can always clone Deimos repository and enhance 
DFLAGS accordingly in a packaging script. As Deimos modules are 
used only during the build process and are not supposed to be 
provided to the end user, it is just a minor maintenance 
inconvenience. Nothing critical.

There are, of course, library packages like vibe.d which do 
include full source code and require availability of Deimos 
bindings to be used. But I think packing a bindings copy with the 
library itself is a proper way to go here, unless they actually 
get exposed to programmer as part of library API (have not met a 
single example so far).
Jul 18 2013
prev sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, July 18, 2013 01:59:54 Jonathan M Davis wrote:

 On Thursday, July 18, 2013 09:37:52 bioinfornatics wrote:

 On Thursday, 18 July 2013 at 07:36:48 UTC, bioinfornatics wrote:

 after taking a look about dub source code. I do not think dun is
 the way to go. Maybe i misunderstanding something.  This project
 do not follow FHS: https://wiki.linuxfoundation.org/en/FHS.
 
 at line 73
 https://github.com/rejectedsoftware/dub/blob/master/source/dub/dub.d#L73
 dub path are set to "/var/lib/dub/"
 
 and at line 195
 https://github.com/rejectedsoftware/dub/blob/master/source/dub/dub.d#L19
 5
 
   install path is set to "/var/lib/dub/packages/"
 
 Any serious linux distro should to accept this to their official
 repo. Then all d project which provides only dub as installer
 will fail grow.
 
 Maybe i misunderstood somewhere just show me


 
 Any serious linux distro should not* to accept this to their
 official repo.


 
 Except that none of this is supposed to even be in a Linux distro. None of
 it would ever be in a Linux package or be installed in /usr or /var or any
 of that. These are development packages, not actual programs, so there's no
 reason to even have them installed by the distro in the main system. You
 only need them to build your own software.
 
 Now, that's a bit different if you're dealing with shared libraries (which
 we'll have soon), and at that point, dub (or any other D package manager)
 would potentially have to worry about looking at what's installed on the
 system in some standard manner, and shared D libraries would be installed
 with Linux packages. But for anything you'd be doing now, that's completely
 irrelevant, because there's no benefit in installing any of it in the
 system as a whole. This is particularly true for deimos projects which
 don't even generate static libraries. They've basically just the D
 equivalent of a bunch of include files.
 
 So, I don't think that discussing installing deimos with a linux package
 (such as an rpm or deb) makes any sense at all. At this point, the only D
 stuff that it makes sense to make Linux packages for is applications. In
 the future, it'll make sense for shared libraries, and package tools like
 dub will have to adapt deal with that, but for the moment, since shared
 library support in D is in its infancy, it's pretty much a non-issue.
 
 We _could_ make linux packages for static libraries, but there's really not
 much benefit in doing so, particularly when D compiles so quickly, and you
 pretty much have to have built everything with the same version of dmd. But
 not even that applies to deimos, since it just provides bindings.


I would also point out that a number of other languages (e.g. ruby, haskell, 
and perl) have package managers for downloading code to use and which may or 
may not be integrated into Linux packages. And it's that sort of thing that 
we're looking to emulate. Then you can just build your program without having 
to track all of the dependencies down. The D package manager would just do 
that for you.

- Jonathan M Davis
Jul 18 2013
parent reply "bioinfornatics" <bioinfornatics fedoraproject.org> writes:
firstly fedora provide D shared lib since 2 years and static lib 
is not allowed.
We want to get one lib and share it to each application.

Secondly deimos provide include file these should not to be embed 
by each application which need it. that duplicate file in more 
some same file will not be at same revision that is not linux 
philosophy.

Third, about package manager:
  - this is for user not distro linux
  - setuptools from python use FHS
  - cpan from perl it is used by user not by packager
  - rvm (ruby ) ditto

so they are 2 things:
  - builder should to build
  - package manager should to package

problem here that is dub is showed to be the universal D way to 
build and manage packaging. that is really a problem to continue 
to support D project into offcial linux repository.

I can package dub and user use it for any reason but i need to be 
able to build and install a project by following FHS and other 
linux rules
Jul 18 2013
next sibling parent reply "Wyatt" <wyatt.epp gmail.com> writes:
On Thursday, 18 July 2013 at 09:47:27 UTC, bioinfornatics wrote:

 problem here that is dub is showed to be the universal D way to 
 build and manage packaging. that is really a problem to 
 continue to support D project into offcial linux repository.


It really isn't.  It may become that if the community decides to 
adopt it officially, but for the time being, it's third-party 
code maintained as part of vibe.d.


 I can package dub and user use it for any reason but i need to 
 be able to build and install a project by following FHS and 
 other linux rules


Your job as a distro package maintainer is to package software 
for your distro; make sure the package interacts well with the 
system, such as by making sure it builds with the proper CFLAGS, 
works with --as-needed, installs files to the proper place, etc.; 
and, within a reasonable degree of certainty, ensure it won't 
break users' systems.

If it's not installing files in the right places, patch it.  
Then, contact the maintainers to see if you can have that patch 
upstreamed.  It's not a project's job to package software-- those 
that manage to do it and do it well are fortunate; in my 
experience, many of them are awful at it (especially LSI).  
Sometimes you'll need to help them understand what distros need 
so they can keep distro needs in mind going forward.  It's a 
two-way street.

-Wyatt
Jul 18 2013
parent "bioinfornatics" <bioinfornatics fedoraproject.org> writes:
 Your job as a distro package maintainer is to package software 
 for your distro; make sure the package interacts well with the 
 system, such as by making sure it builds with the proper 
 CFLAGS, works with --as-needed, installs files to the proper 
 place, etc.; and, within a reasonable degree of certainty, 
 ensure it won't break users' systems.

 If it's not installing files in the right places, patch it.  
 Then, contact the maintainers to see if you can have that patch 
 upstreamed.  It's not a project's job to package software-- 
 those that manage to do it and do it well are fortunate; in my 
 experience, many of them are awful at it (especially LSI).  
 Sometimes you'll need to help them understand what distros need 
 so they can keep distro needs in mind going forward.  It's a 
 two-way street.

 -Wyatt


I know my jobs ;-)

if we can start something which satisfy everyone that is good 
(dev / user / distro ).

I exposing what could to be a problem. If D lib start to be a 
nightmare to package that will not help to continue to package it.

Lib should not to embed deimos import file we wills pent lot of 
time to patch every build system.

So the original question about deimos is closed i will package
- vibe.d ( building manually erf ... ),
- corresponding deimos dependencies
- dub to be used as cpan
Jul 18 2013
prev sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, July 18, 2013 11:47:26 bioinfornatics wrote:

 firstly fedora provide D shared lib since 2 years and static lib
 is not allowed.
 We want to get one lib and share it to each application.


How on earth has it been providing shared libraries for any D packages when we 
only just started supporting shared libraries with the most recent release? 
Any shared library prior to now would definitely have been buggy, and even now, 
it's highly experimental. No one who wants a low chance of bugs should be 
using shared libraries in D yet, let alone for the last 2 years.

- Jonathan M Davis
Jul 18 2013
prev sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, July 17, 2013 23:10:00 bioinfornatics wrote:

 I package for a linux distro so maintain one package which sync
 each module to current stable is more easy to maintain versus
 maintain x module .
 I do not know dub i will take a look if is an irtaller system for
 user or it could to be used to do a package


But the deimos project just provides bindings, not libraries, so the only 
reason to have them is for development, and there's generally no reason to 
build them separate from your program. As such, I see little point in 
including them in a distro's packaging system, not when we're already looking 
to have a package manager for D development packages. And I really don't agree 
with the idea of grouping them all together to be grabbed together anyway, 
because they're all completely unrelated. You wouldn't create a single rpm or 
deb for all of the dev packages on a Linux distro would you? Granted, that 
would be larger than packaging deimos (even if deimos had bindings for all of 
the same dev packages), because it would include more than just bindings, but 
it's the same concept.

This is a prime example of where a package manager for D makes sense and 
distro packages do not, and many of the deimos projects are included in the 
dub registry, so whether or not dub ever becomes either the actual or de facto 
standard for D packages, it's at least usable at this point and is definitely 
gaining adoption.

- Jonathan M Davis
Jul 17 2013
prev sibling next sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 07/17/2013 11:00 PM, Jonathan M Davis wrote:

 I think that my biggest gripe with how Deimos is set up is that the folks 
 who manage the projects in there aren't in control of them. So, they can't 
 merge anything themselves or do other git commands such as tagging or 
 branching.


Is it not possible to set up something where

   (i) Deimos library devs create a 'home repo' for their project, which can
       be a personal repo, a group repo, or even something not on GitHub if
       they prefer

  (ii) Deimos project repos just do a regular, automated clone/pull from the
       specified home repo.
Jul 17 2013
parent "Rikki Cattermole" <alphaglosined gmail.com> writes:
On Thursday, 18 July 2013 at 06:10:45 UTC, Joseph Rushton 
Wakeling wrote:

 On 07/17/2013 11:00 PM, Jonathan M Davis wrote:

 I think that my biggest gripe with how Deimos is set up is 
 that the folks who manage the projects in there aren't in 
 control of them. So, they can't merge anything themselves or 
 do other git commands such as tagging or branching.


 Is it not possible to set up something where

    (i) Deimos library devs create a 'home repo' for their 
 project, which can
        be a personal repo, a group repo, or even something not 
 on GitHub if
        they prefer

   (ii) Deimos project repos just do a regular, automated 
 clone/pull from the
        specified home repo.


An alternative method is to use git submodules and let the people 
who wish to maintain bindings to have it as a submodule of a 
deimos repo.

That way they are linked in nice forms from a central location.
Jul 19 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, July 18, 2013 08:10:35 Joseph Rushton Wakeling wrote:

 On 07/17/2013 11:00 PM, Jonathan M Davis wrote:

 I think that my biggest gripe with how Deimos is set up is that the folks
 who manage the projects in there aren't in control of them. So, they can't
 merge anything themselves or do other git commands such as tagging or
 branching.


 
 Is it not possible to set up something where
 
    (i) Deimos library devs create a 'home repo' for their project, which can
 be a personal repo, a group repo, or even something not on GitHub if they
 prefer


As it stands, Deimos is a github organization (named D-Programming-Deimos to 
be precise) and for anything to be in Deimos, it must be in that github 
organization, which means that the repo is owned by that organization, and 
only members of that organization have any permissions for it (and the 
permissions can vary from member to member). So, while someone can certainly 
manage D bindings for a C library in their own repo, it's not really part of 
Deimos unless it's in a repo in D-Programming-Deimos. We could certainly 
change what it means for something to be in deimos (e.g. Jakob Ovrum suggested 
recently that we turn it into a wiki page which refers to repos which are not 
part of any particular github organization and which could belong to any 
developer, not just those picked by Walter or the other main 
dmd/druntime/Phobos devs), but at present, what deimos is is defined by the 
github organization D-Programming-Deimos.


   (ii) Deimos project repos just do a regular, automated clone/pull from the
 specified home repo.


It might be possible to create a tool to do something like that, but AFAIK, 
nothing like that currently exists. But if you were going to treat a 
particular person's repo as essentially the official one (as it's where the 
deimos one gets its stuff), then you probably might as well just take the 
approach of giving that person full access to the deimos repo in question 
rather than bothering with such a tool.

I really don't know what the best approach would be for deimos. What we have 
currently is working on some level, but clearly there's some room for 
improvement. I think that there's plenty of room for suggestions.

- Jonathan M Davis
Jul 17 2013
prev sibling next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 07/18/2013 08:35 AM, Jonathan M Davis wrote:

 As it stands, Deimos is a github organization (named D-Programming-Deimos to 
 be precise) and for anything to be in Deimos, it must be in that github 
 organization, which means that the repo is owned by that organization, and 
 only members of that organization have any permissions for it (and the 
 permissions can vary from member to member).


Yes, I understand that.


 So, while someone can certainly manage D bindings for a C library in their
 own repo, it's not really part of Deimos unless it's in a repo in
 D-Programming-Deimos.


Yes, but there's a difference between "an up-to-date clone of the project repo
is stored in D-Programming-Deimos" versus "the merging and pull requests and
other project management have to be carried out in D-Programming-Deimos."


   (ii) Deimos project repos just do a regular, automated clone/pull from the
 specified home repo.


 
 It might be possible to create a tool to do something like that, but AFAIK, 
 nothing like that currently exists. But if you were going to treat a 
 particular person's repo as essentially the official one (as it's where the 
 deimos one gets its stuff), then you probably might as well just take the 
 approach of giving that person full access to the deimos repo in question 
 rather than bothering with such a tool.


Depends on what options GitHub gives you.  If you can't give someone control
over a single repo in Deimos without giving them effective control over all the
others, then independent project repos + clone to D-Programming-Deimos might be
an OK way to organize it.  Seems better to me than a Wiki page list, because
this way you have direct access to the code from the Deimos project list.

It also gives Deimos admins a way to easily exclude projects that are proving
problematic -- just delete the Deimos clone of the project repo.
Jul 18 2013
prev sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, July 18, 2013 09:09:09 Joseph Rushton Wakeling wrote:

 Depends on what options GitHub gives you.  If you can't give someone control
 over a single repo in Deimos without giving them effective control over all
 the others, then independent project repos + clone to D-Programming-Deimos
 might be an OK way to organize it.


I'm sure that it's possible to give per-repo permissions, because I have full 
commit access to Phobos and druntime but no commit access whatsoever to dmd, 
and they're all in the same github org. But I don't know how fine-grained the 
permissions controls are or how much they can ultimately do.

- Jonathan M Davis
Jul 18 2013