www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - a cabal for D ?

reply "Jason E. Aten" <j.e.aten gmail.com> writes:
Please correct me if I'm wrong, but I observe that there doesn't appear 
to be a package management system / standard repository for D libraries.  
Or is there?

I'm talking about something as easy to use as R's CRAN,
 install.packages("rforest")

or cpan for perl, ctan for latex, dpgk/apt for debian, cabal for Haskell/ Hackage, etc. If there's not a commonly utilized one currently, perhaps we could "borrow" cabal, with a trivial port. cabal is Haskell's package manager. Not only does having a standard package install system facilitate adoption, it greatly facilitates code sharing and library maturation.
Mar 17 2011
next sibling parent reply Tomek =?ISO-8859-2?B?U293afFza2k=?= <just ask.me> writes:
Jason E. Aten napisa=B3:

 Please correct me if I'm wrong, but I observe that there doesn't appear=20
 to be a package management system / standard repository for D libraries. =

 Or is there?

No, there isn't.
 I'm talking about something as easy to use as R's CRAN,
 install.packages("rforest")

or cpan for perl, ctan for latex, dpgk/apt for debian, cabal for Haskell/ Hackage, etc. =20 If there's not a commonly utilized one currently, perhaps we could=20 "borrow" cabal, with a trivial port. cabal is Haskell's package manager. =20 Not only does having a standard package install system facilitate=20 adoption, it greatly facilitates code sharing and library maturation.

Yes, we need it badly. I think it's a good moment to start a discussion. First off, what exactly d= o we want from a package management system? --=20 Tomek
Mar 17 2011
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 3/17/11 4:00 PM, Tomek Sowiński wrote:
 Jason E. Aten napisał:

 Please correct me if I'm wrong, but I observe that there doesn't appear
 to be a package management system / standard repository for D libraries.
 Or is there?

No, there isn't.
 I'm talking about something as easy to use as R's CRAN,
 install.packages("rforest")

or cpan for perl, ctan for latex, dpgk/apt for debian, cabal for Haskell/ Hackage, etc. If there's not a commonly utilized one currently, perhaps we could "borrow" cabal, with a trivial port. cabal is Haskell's package manager. Not only does having a standard package install system facilitate adoption, it greatly facilitates code sharing and library maturation.

Yes, we need it badly. I think it's a good moment to start a discussion. First off, what exactly do we want from a package management system?

Yah, would be great. It would be awesome if an expert in e.g. apt would join D and create the design of a package management system. Andrei
Mar 17 2011
next sibling parent reply Chris Manning <cmanning999 gmail.com> writes:
On 17/03/2011 22:49, Jason E. Aten wrote:
 Somewhat tongue in cheek, we could call it dabal.

 As in, "get on dabal!"  :-)

If D gets accepted for Google Summer of Code, I think this would be a great idea for a project and I would be interested in implementing it as a student. Although, it does seem overly ambitious so maybe only some of this could be for the gsoc (and if I do this It'd be great to carry on working on it anyway). What does everybody think about this? Should I draw up a proposal of some kind? Chris
Mar 18 2011
parent reply Jacob Carlborg <doob me.com> writes:
On 2011-03-18 18:04, Chris Manning wrote:
 On 17/03/2011 22:49, Jason E. Aten wrote:
 Somewhat tongue in cheek, we could call it dabal.

 As in, "get on dabal!" :-)

If D gets accepted for Google Summer of Code, I think this would be a great idea for a project and I would be interested in implementing it as a student. Although, it does seem overly ambitious so maybe only some of this could be for the gsoc (and if I do this It'd be great to carry on working on it anyway). What does everybody think about this? Should I draw up a proposal of some kind? Chris

I've been thinking for quite some time to build a package management system for D, lets call it dpac as an example. This is the ideas that I have: Basically copy how RubyGems works. Use Ruby as a DSL for dpacsepc files which is used to create to create the dpac file. This is an example for how a file used to build a package could look like: name "Foo Bar" summary "This is the Foo Bar package" version "1.0.0" type :lib author "Jacob Carlborg" files ["lib.d"] # list of the files in the package build :make # other options could be :dsss :cmake and so on dversion 2 # D1 or D2 Build a dpac package out of the dpacspec file: dpac foobar.dpacspec Publish the package: $ dpac publish foobar Install the package: $ dpac install foobar A dpac package would just be a zip file (or some other type of archive) containing all the necessary files to build the package and a file with meta data. All packages would be manged on a basic RESTful web server. Using GET to download a package and POST to publish a package. I'm working on a build system for D that I was thinking about to integrate with the package management system. Then the build system could track the files needed to build the package, making the "files" attribute optional. I also has a tool called DVM, https://bitbucket.org/doob/dvm , used for installing and managing different versions of D compilers. I was thinking about integrating DVM with the package management system to be able to install different packages for different compilers. -- /Jacob Carlborg
Mar 19 2011
next sibling parent reply Chris Manning <cmanning999 gmail.com> writes:
On 19/03/2011 14:36, Jacob Carlborg wrote:
 On 2011-03-18 18:04, Chris Manning wrote:
 On 17/03/2011 22:49, Jason E. Aten wrote:
 Somewhat tongue in cheek, we could call it dabal.

 As in, "get on dabal!" :-)

If D gets accepted for Google Summer of Code, I think this would be a great idea for a project and I would be interested in implementing it as a student. Although, it does seem overly ambitious so maybe only some of this could be for the gsoc (and if I do this It'd be great to carry on working on it anyway). What does everybody think about this? Should I draw up a proposal of some kind? Chris

I've been thinking for quite some time to build a package management system for D, lets call it dpac as an example. This is the ideas that I have: Basically copy how RubyGems works. Use Ruby as a DSL for dpacsepc files which is used to create to create the dpac file. This is an example for how a file used to build a package could look like: name "Foo Bar" summary "This is the Foo Bar package" version "1.0.0" type :lib author "Jacob Carlborg" files ["lib.d"] # list of the files in the package build :make # other options could be :dsss :cmake and so on dversion 2 # D1 or D2 Build a dpac package out of the dpacspec file: dpac foobar.dpacspec Publish the package: $ dpac publish foobar Install the package: $ dpac install foobar A dpac package would just be a zip file (or some other type of archive) containing all the necessary files to build the package and a file with meta data. All packages would be manged on a basic RESTful web server. Using GET to download a package and POST to publish a package. I'm working on a build system for D that I was thinking about to integrate with the package management system. Then the build system could track the files needed to build the package, making the "files" attribute optional. I also has a tool called DVM, https://bitbucket.org/doob/dvm , used for installing and managing different versions of D compilers. I was thinking about integrating DVM with the package management system to be able to install different packages for different compilers.

or arch's AUR. This would mean that the sources could be stored anywhere and just the info to build a package would be stored in a centralised location. For publishing these build "scripts", again, AUR's system comes to mind, although perhaps something more controll(ed/able). Also, gentoo's sunrise overlay. Of course, this comes with the downside of the user having to compile the packages on their end. Chris
Mar 19 2011
parent Jacob Carlborg <doob me.com> writes:
On 2011-03-19 17:05, Chris Manning wrote:
 On 19/03/2011 14:36, Jacob Carlborg wrote:
 On 2011-03-18 18:04, Chris Manning wrote:
 On 17/03/2011 22:49, Jason E. Aten wrote:
 Somewhat tongue in cheek, we could call it dabal.

 As in, "get on dabal!" :-)

If D gets accepted for Google Summer of Code, I think this would be a great idea for a project and I would be interested in implementing it as a student. Although, it does seem overly ambitious so maybe only some of this could be for the gsoc (and if I do this It'd be great to carry on working on it anyway). What does everybody think about this? Should I draw up a proposal of some kind? Chris

I've been thinking for quite some time to build a package management system for D, lets call it dpac as an example. This is the ideas that I have: Basically copy how RubyGems works. Use Ruby as a DSL for dpacsepc files which is used to create to create the dpac file. This is an example for how a file used to build a package could look like: name "Foo Bar" summary "This is the Foo Bar package" version "1.0.0" type :lib author "Jacob Carlborg" files ["lib.d"] # list of the files in the package build :make # other options could be :dsss :cmake and so on dversion 2 # D1 or D2 Build a dpac package out of the dpacspec file: dpac foobar.dpacspec Publish the package: $ dpac publish foobar Install the package: $ dpac install foobar A dpac package would just be a zip file (or some other type of archive) containing all the necessary files to build the package and a file with meta data. All packages would be manged on a basic RESTful web server. Using GET to download a package and POST to publish a package. I'm working on a build system for D that I was thinking about to integrate with the package management system. Then the build system could track the files needed to build the package, making the "files" attribute optional. I also has a tool called DVM, https://bitbucket.org/doob/dvm , used for installing and managing different versions of D compilers. I was thinking about integrating DVM with the package management system to be able to install different packages for different compilers.

or arch's AUR. This would mean that the sources could be stored anywhere and just the info to build a package would be stored in a centralised location. For publishing these build "scripts", again, AUR's system comes to mind, although perhaps something more controll(ed/able). Also, gentoo's sunrise overlay. Of course, this comes with the downside of the user having to compile the packages on their end. Chris

Oh, I forgot a part. Just as with RubyGems you would be able to install a package from a repository (git, mercurial or something else) as long as it contains the file containing the package specification file (dpacspec). Of course you could manually downloading a package and installing it. -- /Jacob Carlborg
Mar 19 2011
prev sibling parent reply Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 19/03/2011 14:36, Jacob Carlborg wrote:
 On 2011-03-18 18:04, Chris Manning wrote:
 On 17/03/2011 22:49, Jason E. Aten wrote:
 Somewhat tongue in cheek, we could call it dabal.

 As in, "get on dabal!" :-)

If D gets accepted for Google Summer of Code, I think this would be a great idea for a project and I would be interested in implementing it as a student. Although, it does seem overly ambitious so maybe only some of this could be for the gsoc (and if I do this It'd be great to carry on working on it anyway). What does everybody think about this? Should I draw up a proposal of some kind? Chris

I've been thinking for quite some time to build a package management system for D, lets call it dpac as an example. This is the ideas that I have: Basically copy how RubyGems works. Use Ruby as a DSL for dpacsepc files which is used to create to create the dpac file. This is an example for how a file used to build a package could look like: name "Foo Bar" summary "This is the Foo Bar package" version "1.0.0" type :lib author "Jacob Carlborg" files ["lib.d"] # list of the files in the package build :make # other options could be :dsss :cmake and so on dversion 2 # D1 or D2

Copying Rubygems would be a bad idea, or at least a very incomplete one. And the example code above shows why, doesn't anyone else see it? Just look at the line: build :make # other options could be :dsss :cmake and so on Basicly, this would be a D package manager that requires another package manager to build D artifacts... kinda not very useful, right? If we skip the issue of building D artifacts (executables, dynamic libraries, static libraries, etc.) we skip one essential aspect of a D package/build management system (if not the main aspect, but that is debatable...). This is why I think just copying/cloning a PM from interpreted languages is not helping us much. -- Bruno Medeiros - Software Engineer
Apr 01 2011
parent reply Jacob Carlborg <doob me.com> writes:
On 2011-04-01 20:44, Bruno Medeiros wrote:
 On 19/03/2011 14:36, Jacob Carlborg wrote:
 On 2011-03-18 18:04, Chris Manning wrote:
 On 17/03/2011 22:49, Jason E. Aten wrote:
 Somewhat tongue in cheek, we could call it dabal.

 As in, "get on dabal!" :-)

If D gets accepted for Google Summer of Code, I think this would be a great idea for a project and I would be interested in implementing it as a student. Although, it does seem overly ambitious so maybe only some of this could be for the gsoc (and if I do this It'd be great to carry on working on it anyway). What does everybody think about this? Should I draw up a proposal of some kind? Chris

I've been thinking for quite some time to build a package management system for D, lets call it dpac as an example. This is the ideas that I have: Basically copy how RubyGems works. Use Ruby as a DSL for dpacsepc files which is used to create to create the dpac file. This is an example for how a file used to build a package could look like: name "Foo Bar" summary "This is the Foo Bar package" version "1.0.0" type :lib author "Jacob Carlborg" files ["lib.d"] # list of the files in the package build :make # other options could be :dsss :cmake and so on dversion 2 # D1 or D2

Copying Rubygems would be a bad idea, or at least a very incomplete one. And the example code above shows why, doesn't anyone else see it? Just look at the line: build :make # other options could be :dsss :cmake and so on Basicly, this would be a D package manager that requires another package manager to build D artifacts... kinda not very useful, right? If we skip the issue of building D artifacts (executables, dynamic libraries, static libraries, etc.) we skip one essential aspect of a D package/build management system (if not the main aspect, but that is debatable...). This is why I think just copying/cloning a PM from interpreted languages is not helping us much.

I don't think it's the package manger's job to build the application/library. The package manager operates on a level higher than a build tool. The build tool is working with files, tracking dependencies among files and building applications/libraries. The package manager is working with packages, tracking dependencies among packages. I was planning to creating my own build tool that would be integrated with the package manager. The package manager would use the build tool to track the files making up the package. But other build tool would be supported as well, not forcing anyone to use my build tool. -- /Jacob Carlborg
Apr 02 2011
parent Jacob Carlborg <doob me.com> writes:
On 2011-04-03 08:55, Russel Winder wrote:
 On Sat, 2011-04-02 at 21:35 +0200, Jacob Carlborg wrote:
 [ . . . ]
 I don't think it's the package manger's job to build the
 application/library. The package manager operates on a level higher than
 a build tool. The build tool is working with files, tracking
 dependencies among files and building applications/libraries. The
 package manager is working with packages, tracking dependencies among
 packages.

Haskell's Cabal has blurred the lines between package management and build in many people's minds, but it only succeeds when in isolation. As soon as there is a platform packaging system it is clear that build and packaging must be separate. Though obviously you can get into JVM/Maven, and indeed Go, situations where again the lines get blurred. For something such as D which should have some ability to work with C and C++ libraries, keeping the separation is likely a better route.
 I was planning to creating my own build tool that would be integrated
 with the package manager. The package manager would use the build tool
 to track the files making up the package. But other build tool would be
 supported as well, not forcing anyone to use my build tool.

Is there a need for a brand new build tool. Are CMake, SCons and Waf not sufficient to the task?

This might not be fair, because I haven't used Waf, SCons or CMake very much, but I think they look too complicated. I would like a simple tool, I like dsss/rebuild it's a simple tool. You just put the file containing the main function in a configuration file and it just works.
 Clearly there are no perfect tools, there is always room for evolution
 and new things.  However the history in the JVM-related milieu and the
 Go arena, indicate that the knee jerk reaction of "we have a new
 language, therefore we must write a new build framework" leads to a lot
 of isolationism, creation of ghettos, and a lot of wasted effort.  Go
 has at least 6 brand new build frameworks, all of which were deemed
 necessary, and none of which have gained any traction -- the core
 development team are believers that Make is all that is needed.  Rake
 has gained little traction outside the Ruby community. Leiningen has
 little traction outside the Clojure community.  SBT has little traction
 outside the Scala community.  Gradle is getting significant traction
 across the Java and Groovy communities as it solves problems inherent in
 the wildly successful Ant and Maven frameworks.

 SCons and Waf arose as dynamic language realizations of ADG-based
 systems to replace Make and Autotools respectively.  They have a clear
 reason for existence and are successful, indeed have traction.  CMake
 likewise.

 Note here though there is a yawning gap between build frameworks in the
 JVM-verse and the native-verse.  Tools from one are generally rubbish in
 the other, to the detriment of build.

 If there is to be a new build framework to compete against SCons, Waf,
 CMake, Gradle and Maven I suggest it must be designed from the outset to
 deal with the vastly different problems of C, C++, D, LaTeX, Python,
 Ruby, Java, Groovy, Scala, Clojure, Erlang, Haskell, Chapel, Fortress,
 X10, etc.

I don't want a tool to compete against SCons, Waf and others. I want a simple tool for building D software. I think this is one of the problems with the other build tools, that they should be able to handle almost every language out there. -- /Jacob Carlborg
Apr 03 2011
prev sibling next sibling parent spir <denis.spir gmail.com> writes:
On 03/18/2011 06:04 PM, Chris Manning wrote:
 On 17/03/2011 22:49, Jason E. Aten wrote:
 Somewhat tongue in cheek, we could call it dabal.

 As in, "get on dabal!" :-)

If D gets accepted for Google Summer of Code, I think this would be a great idea for a project and I would be interested in implementing it as a student. Although, it does seem overly ambitious so maybe only some of this could be for the gsoc (and if I do this It'd be great to carry on working on it anyway).

+++
 What does everybody think about this? Should I draw up a proposal of some kind?

Dunno how it's supposed to be done... Denis -- _________________ vita es estrany spir.wikidot.com
Mar 18 2011
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2011-03-17 23:44, Jason E. Aten wrote:
 On 3/17/11 4:00 PM, Tomek SowiƄski wrote:
 Yes, we need it badly.
 I think it's a good moment to start a discussion. First off, what
 exactly do we want from a package management system?


 On Thu, 17 Mar 2011 16:28:37 -0500, Andrei Alexandrescu wrote:
 Yah, would be great. It would be awesome if an expert in e.g. apt would
 join D and create the design of a package management system.

I would invite interested parties to review the cabal/cabal-install/ Hackage system documentation. It is described here. http://www.haskell.org/haskellwiki/How_to_write_a_Haskell_program Haskell has over 2000 contributed user libraries--largely because they have such a nice, easy to use, and well documented package system. Rather than expend much effort, in the tradition of lazy evaluation and getting 80/20 the way there, I would prefer to just do a clone of an already successful system such as cabal (or ?) and then take feedback to based on actual usage with D. Thoughts? Comments welcome. Jason

I would clone RubyGems. -- /Jacob Carlborg
Mar 19 2011
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, March 17, 2011 13:44:42 Jason E. Aten wrote:
 Please correct me if I'm wrong, but I observe that there doesn't appear
 to be a package management system / standard repository for D libraries.
 Or is there?
 
 I'm talking about something as easy to use as R's CRAN,
 
 install.packages("rforest")

or cpan for perl, ctan for latex, dpgk/apt for debian, cabal for Haskell/ Hackage, etc. If there's not a commonly utilized one currently, perhaps we could "borrow" cabal, with a trivial port. cabal is Haskell's package manager. Not only does having a standard package install system facilitate adoption, it greatly facilitates code sharing and library maturation.

D has nothing like that. Periodically someone will bring it up, but until someone actually sits down and implements something, it doesn't look like it's going to happen anytime soon. Like many things around here, it's only going to get done if someone who really wants it done has both the time and the know-how to do it (and then actually takes the time to do it). And that hasn't happened yet. - Jonathan M Davis
Mar 17 2011
prev sibling next sibling parent reply Lutger Blijdestijn <lutger.blijdestijn gmail.com> writes:
Jason E. Aten wrote:

 Please correct me if I'm wrong, but I observe that there doesn't appear
 to be a package management system / standard repository for D libraries.
 Or is there?
 
 I'm talking about something as easy to use as R's CRAN,
 install.packages("rforest")

or cpan for perl, ctan for latex, dpgk/apt for debian, cabal for Haskell/ Hackage, etc. If there's not a commonly utilized one currently, perhaps we could "borrow" cabal, with a trivial port. cabal is Haskell's package manager. Not only does having a standard package install system facilitate adoption, it greatly facilitates code sharing and library maturation.

There used to be one called dsss (d shared software system). It was widely used, I think some D1 libraries still use it but it hasn't been maintained for years.
Mar 17 2011
parent Jacob Carlborg <doob me.com> writes:
On 2011-03-17 22:47, Lutger Blijdestijn wrote:
 Jason E. Aten wrote:

 Please correct me if I'm wrong, but I observe that there doesn't appear
 to be a package management system / standard repository for D libraries.
 Or is there?

 I'm talking about something as easy to use as R's CRAN,
 install.packages("rforest")

or cpan for perl, ctan for latex, dpgk/apt for debian, cabal for Haskell/ Hackage, etc. If there's not a commonly utilized one currently, perhaps we could "borrow" cabal, with a trivial port. cabal is Haskell's package manager. Not only does having a standard package install system facilitate adoption, it greatly facilitates code sharing and library maturation.

There used to be one called dsss (d shared software system). It was widely used, I think some D1 libraries still use it but it hasn't been maintained for years.

It's still working fine for building and installing local libraries with D1, it's the net capabilities that doesn't work. -- /Jacob Carlborg
Mar 19 2011
prev sibling next sibling parent "Jason E. Aten" <j.e.aten gmail.com> writes:
 On 3/17/11 4:00 PM, Tomek SowiƄski wrote:
 Yes, we need it badly.
 I think it's a good moment to start a discussion. First off, what
 exactly do we want from a package management system?


 On Thu, 17 Mar 2011 16:28:37 -0500, Andrei Alexandrescu wrote:
 Yah, would be great. It would be awesome if an expert in e.g. apt would
 join D and create the design of a package management system.

I would invite interested parties to review the cabal/cabal-install/ Hackage system documentation. It is described here. http://www.haskell.org/haskellwiki/How_to_write_a_Haskell_program Haskell has over 2000 contributed user libraries--largely because they have such a nice, easy to use, and well documented package system. Rather than expend much effort, in the tradition of lazy evaluation and getting 80/20 the way there, I would prefer to just do a clone of an already successful system such as cabal (or ?) and then take feedback to based on actual usage with D. Thoughts? Comments welcome. Jason
Mar 17 2011
prev sibling next sibling parent "Jason E. Aten" <j.e.aten gmail.com> writes:
Somewhat tongue in cheek, we could call it dabal.

As in, "get on dabal!"  :-)
Mar 17 2011
prev sibling next sibling parent reply Russel Winder <russel russel.org.uk> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, 2011-03-17 at 20:44 +0000, Jason E. Aten wrote:
 Please correct me if I'm wrong, but I observe that there doesn't appear=

 to be a package management system / standard repository for D libraries. =

 Or is there?
=20
 I'm talking about something as easy to use as R's CRAN,
 install.packages("rforest")

or cpan for perl, ctan for latex, dpgk/apt for debian, cabal for Haskell/ Hackage, etc.

Note that every language-specific package manager conflicts directly with every operating system package manager. Thus RubyGems, CPAN, Cabal, Maven, Go, etc. conflicts with the package management of Debian, Fedora, SUSE, FreeBSD, MacPorts, etc. leading to pain. Pain leads to anger. Anger leads to hate. Hate leads to suffering.
 If there's not a commonly utilized one currently, perhaps we could=20
 "borrow" cabal, with a trivial port.  cabal is Haskell's package manager.
=20
 Not only does having a standard package install system facilitate=20
 adoption, it greatly facilitates code sharing and library maturation.

At the expense of easy system administration. I guess the only up side of language specific package management is that it enables people whose operating systems are not package structured to do things sensibly. Alternatively Windows users could switch to a sensible operating system ;-) Given that D has chosen to switch to Git for version control, doesn't this imply that package management transported over DVCS is the way forward. Go has certainly taken this route. It prioritizes Mercurial but supports Bazaar and Git as well. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel russel.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Mar 18 2011
next sibling parent reply Jesse Phillips <jessekphillips+D gmail.com> writes:
Russel Winder Wrote:

 At the expense of easy system administration.

Specifics? It allows for one packaging system across all operating systems. This means you don't need to figure out how to package your source in RPM, Deb, ipgk, arc, emerge, zip, or whatever else Linux has. Otherwise I would be for using the native packaging system, but it isn't easy for the contributor, the maintainer of the server, or the one building a system that works with all of them.
Mar 18 2011
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2011-03-18 22:20, Jason E. Aten wrote:
 On Fri, 18 Mar 2011 18:42:36 +0000, Russel Winder wrote:
 I still think basing a D packaging system on Git to be the best
 direction.

Basing package distribution on Git or hg could be a big win, and would help establish a customary case for revision control which is one of the things that make cabal work so well (they use darcs for everything). I find these revision control systems ver fast and very easy to use. The other thing that cabal standardizes is the make/build system. I've updated bud/build to compile under D2, with all the latest patches, but I'm far from convinced that it should be a make system of choice. I have limited experience here, but a "D aware" build system would seem to be highly preferable. What are people's experiences with the various options for build systems with D?

It's not very easy to make an incremental build system for D because of several reasons. Some are due to how the language works and some are due to how DMD works: * DMD doesn't output all data in all the object files - This can perhaps be solved by compiling with the -lib switch * When you change one D file you need to recompile ALL files that depend on the changed file. To compare with C/C++ which has source and header files you only need to recompile the source file if you change it * DMD doesn't keep the fully qualified module name when naming object files resulting in foo.bar will conflict with bar.bar. Issue 3541.
 To me, I like the design goals of Andreas Fredriksson's Tundra build
 system (he wants speed of incremental of builds prioritized over all
 else, which means utilizing multicores for builds as much as possible to
 get the quickest build), because fast builds are critical for game
 development, where D is very attractive.  Game projects compile tens of
 thousands of files. Tundra is GPL and it would be easy to extend to
 support D.

 http://voodoo-slide.blogspot.com/2010/08/tundra-my-build-system.html
 https://github.com/deplinenoise/tundra
 https://github.com/deplinenoise/tundra/downloads

-- /Jacob Carlborg
Mar 19 2011
prev sibling parent Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 18/03/2011 18:42, Russel Winder wrote:
 On Fri, 2011-03-18 at 10:41 -0400, Jesse Phillips wrote:
 [ . . . ]
 Otherwise I would be for using the native packaging system, but it
 isn't easy for the contributor, the maintainer of the server, or the
 one building a system that works with all of them.

The problem is that the system administrator wants to use just the OS packaging system to make things easy for them to maintain the system. The user wants to be able to install stuff easily and may not have sufficient permissions to actually use the OS packaging system. Sys admins don't want to have to learn N packages to deal with N languages, the use probably only works in a couple of languages and so doesn't care if they have to learn two different packagin systems so as to get stuff done.

I agree sys admins should not have to learn language specific package managers just because the user wants to use one. Rather what should happen is that language PMs should be able to be "installed" on any user-specified location, and installed multiple times. Just like a compiler should. They should not be designed to work only as an OS "singleton", so to speak.
 This is not an easy issue.  I just find the knee-jerk reaction of "we
 have this new language therefore we must have a brand new (build system|
 packaging system|shell|implementation of every comms protocol|new user
 interface library|operating system)" leads to too many distractions from
 getting stuff done using the good tools that are already available.

What I think makes sense for a D package manager is to integrate with existing *build* systems (make, Cmake, Ant, Maven, SCons, etc., etc., whatever might be appropriate). But definitely not with OS package managers. -- Bruno Medeiros - Software Engineer
Apr 01 2011
prev sibling next sibling parent Jonas Drewsen <jdrewsen nospam.com> writes:
On 18/03/11 09.52, Russel Winder wrote:
 On Thu, 2011-03-17 at 20:44 +0000, Jason E. Aten wrote:
 Please correct me if I'm wrong, but I observe that there doesn't appear
 to be a package management system / standard repository for D libraries.
 Or is there?

 I'm talking about something as easy to use as R's CRAN,
 install.packages("rforest")

or cpan for perl, ctan for latex, dpgk/apt for debian, cabal for Haskell/ Hackage, etc.

Note that every language-specific package manager conflicts directly with every operating system package manager. Thus RubyGems, CPAN, Cabal, Maven, Go, etc. conflicts with the package management of Debian, Fedora, SUSE, FreeBSD, MacPorts, etc. leading to pain. Pain leads to anger. Anger leads to hate. Hate leads to suffering.
 If there's not a commonly utilized one currently, perhaps we could
 "borrow" cabal, with a trivial port.  cabal is Haskell's package manager.

 Not only does having a standard package install system facilitate
 adoption, it greatly facilitates code sharing and library maturation.

At the expense of easy system administration. I guess the only up side of language specific package management is that it enables people whose operating systems are not package structured to do things sensibly. Alternatively Windows users could switch to a sensible operating system ;-)

The worst thing is that dependency tracking is lost when the sysadm uses another package system e.g CPAN. If the dependency problem could be solved in an elegant way I think a sysadm would be ok with it.
 Given that D has chosen to switch to Git for version control, doesn't
 this imply that package management transported over DVCS is the way
 forward.  Go has certainly taken this route.  It prioritizes Mercurial
 but supports Bazaar and Git as well.

I like how Go uses git but there is room for improvements. No reason the limit the protocols as long a it can be resonable supported. It is just a place to download from. Put http, ftp, samba, torrent, dropbox, facebook... in there as well :) DSSS was actually a nice solution. Maybe it could be brought up to date and improved to fit the needs? /Jonas
Mar 18 2011
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2011-03-18 09:52, Russel Winder wrote:
 On Thu, 2011-03-17 at 20:44 +0000, Jason E. Aten wrote:
 Please correct me if I'm wrong, but I observe that there doesn't appear
 to be a package management system / standard repository for D libraries.
 Or is there?

 I'm talking about something as easy to use as R's CRAN,
 install.packages("rforest")

or cpan for perl, ctan for latex, dpgk/apt for debian, cabal for Haskell/ Hackage, etc.

Note that every language-specific package manager conflicts directly with every operating system package manager. Thus RubyGems, CPAN, Cabal, Maven, Go, etc. conflicts with the package management of Debian, Fedora, SUSE, FreeBSD, MacPorts, etc. leading to pain. Pain leads to anger. Anger leads to hate. Hate leads to suffering.
 If there's not a commonly utilized one currently, perhaps we could
 "borrow" cabal, with a trivial port.  cabal is Haskell's package manager.

 Not only does having a standard package install system facilitate
 adoption, it greatly facilitates code sharing and library maturation.

At the expense of easy system administration. I guess the only up side of language specific package management is that it enables people whose operating systems are not package structured to do things sensibly. Alternatively Windows users could switch to a sensible operating system ;-)

Another advantage that as least RubyGems has, in the combination with RVM, is that you can have different gems/packages installed for different Ruby compilers.
 Given that D has chosen to switch to Git for version control, doesn't
 this imply that package management transported over DVCS is the way
 forward.  Go has certainly taken this route.  It prioritizes Mercurial
 but supports Bazaar and Git as well.

-- /Jacob Carlborg
Mar 19 2011
prev sibling next sibling parent Lutger Blijdestijn <lutger.blijdestijn gmail.com> writes:
Russel Winder wrote:

 On Thu, 2011-03-17 at 20:44 +0000, Jason E. Aten wrote:
 Please correct me if I'm wrong, but I observe that there doesn't appear
 to be a package management system / standard repository for D libraries.
 Or is there?
 
 I'm talking about something as easy to use as R's CRAN,
 install.packages("rforest")

or cpan for perl, ctan for latex, dpgk/apt for debian, cabal for Haskell/ Hackage, etc.

Note that every language-specific package manager conflicts directly with every operating system package manager. Thus RubyGems, CPAN, Cabal, Maven, Go, etc. conflicts with the package management of Debian, Fedora, SUSE, FreeBSD, MacPorts, etc. leading to pain. Pain leads to anger. Anger leads to hate. Hate leads to suffering.
 If there's not a commonly utilized one currently, perhaps we could
 "borrow" cabal, with a trivial port.  cabal is Haskell's package manager.
 
 Not only does having a standard package install system facilitate
 adoption, it greatly facilitates code sharing and library maturation.

At the expense of easy system administration.

Not necessarily, fedora has rpm packages of gems for example.
 I guess the only up side of language specific package management is that
 it enables people whose operating systems are not package structured to
 do things sensibly.  Alternatively Windows users could switch to a
 sensible operating system ;-)
 

It's also often easier to package libraries with system specifically designed to do so for a particular language. That, combined with a common repository, usually results in a much wider selection of apis than a native distribution offers.
Mar 19 2011
prev sibling parent Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
Hi Russel. (BTW, I was the guy a few weeks ago at the London Java 
Community Meetup pub meet that Barry Cranford mentioned in an SMS. Sorry 
you couldn't make it, maybe next time. :p )

On 18/03/2011 08:52, Russel Winder wrote:
 Note that every language-specific package manager conflicts directly
 with every operating system package manager.  Thus RubyGems, CPAN,
 Cabal, Maven, Go, etc. conflicts with the package management of Debian,
 Fedora, SUSE, FreeBSD, MacPorts, etc.

Which is the way it should be. (Note that "conflicts" is likely not the best word, rather "ignores" or "does not make use or interact with" is a better term.) I'm well convinced of this because these two - OS and language package manager - /usually/ serve very distinct purposes. The OS PMs are usually focused on installing applications, not building them (yes, there are exceptions, like Gentoo). In particular the OS PMs most likely won't support functionality you would want from a language PM. (say for example that you would want to specify that your library has a dependency on any other library that exports a D package named org.foo.bar?) But we don't even need to go there, just the fact that OS PMs are not cross-platform is reason enough for them to be unsuitable to be used (on their own) as language PM. There needs to be a language PM abstracted from OS/platform. Perhaps some additional (and optional) integration with the OS PM could be available in the language PM. But the language PM would still be "on top". And even so I'm not convinced this integrations makes much sense. Maybe for dynamic and interpreted languages like Ruby and Python, where built application and sources are often the same... here perhaps the term "conflict" as mentioned above is more appropriate, the distinction between end-user and developer-user is less clear. But none of this applies to D, being a compiled language. -- Bruno Medeiros - Software Engineer
Apr 01 2011
prev sibling next sibling parent Ulrik Mikaelsson <ulrik.mikaelsson gmail.com> writes:
2011/3/18 Jesse Phillips <jessekphillips+D gmail.com>:
 Russel Winder Wrote:
 At the expense of easy system administration.


For the developer, yes. For the user, it just means that you have to learn N different packaging systems, which not uncommonly cause conflicts, for instance in language-bindings conflicting with "native" libraries. I know of at least one company that were quite serious about migrating their webapps from Java to Ruby/rails, but after a while cancelled due to just those packaging issues with gems creating weird conflicts and silent errors when bindings were complied slightly differently from the native C-lib. ---- For the record, their apps were designed for deployment on Ubuntu Server, which at the time had native support for almost all Java-related packages, but less wide support for Ruby. The situation have changed a little since then, and quite a lot of Ruby-packages are now in native Ubuntu.
Mar 18 2011
prev sibling next sibling parent Russel Winder <russel russel.org.uk> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Fri, 2011-03-18 at 10:41 -0400, Jesse Phillips wrote:
[ . . . ]
 Otherwise I would be for using the native packaging system, but it
 isn't easy for the contributor, the maintainer of the server, or the
 one building a system that works with all of them.

The problem is that the system administrator wants to use just the OS packaging system to make things easy for them to maintain the system. The user wants to be able to install stuff easily and may not have sufficient permissions to actually use the OS packaging system. Sys admins don't want to have to learn N packages to deal with N languages, the use probably only works in a couple of languages and so doesn't care if they have to learn two different packagin systems so as to get stuff done. This is not an easy issue. I just find the knee-jerk reaction of "we have this new language therefore we must have a brand new (build system| packaging system|shell|implementation of every comms protocol|new user interface library|operating system)" leads to too many distractions from getting stuff done using the good tools that are already available. I still think basing a D packaging system on Git to be the best direction. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel russel.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Mar 18 2011
prev sibling next sibling parent "Jason E. Aten" <j.e.aten gmail.com> writes:
On Fri, 18 Mar 2011 18:42:36 +0000, Russel Winder wrote:
 I still think basing a D packaging system on Git to be the best
 direction.

Basing package distribution on Git or hg could be a big win, and would help establish a customary case for revision control which is one of the things that make cabal work so well (they use darcs for everything). I find these revision control systems ver fast and very easy to use. The other thing that cabal standardizes is the make/build system. I've updated bud/build to compile under D2, with all the latest patches, but I'm far from convinced that it should be a make system of choice. I have limited experience here, but a "D aware" build system would seem to be highly preferable. What are people's experiences with the various options for build systems with D? To me, I like the design goals of Andreas Fredriksson's Tundra build system (he wants speed of incremental of builds prioritized over all else, which means utilizing multicores for builds as much as possible to get the quickest build), because fast builds are critical for game development, where D is very attractive. Game projects compile tens of thousands of files. Tundra is GPL and it would be easy to extend to support D. http://voodoo-slide.blogspot.com/2010/08/tundra-my-build-system.html https://github.com/deplinenoise/tundra https://github.com/deplinenoise/tundra/downloads
Mar 18 2011
prev sibling next sibling parent "Jason E. Aten" <j.e.aten gmail.com> writes:
 On Sat, 19 Mar 2011 12:19:58 +0100, Jacob Carlborg wrote:
 What are people's experiences with the various options for build
 systems with D?

It's not very easy to make an incremental build system for D because of several reasons. Some are due to how the language works and some are due to how DMD works: * DMD doesn't output all data in all the object files - This can perhaps be solved by compiling with the -lib switch * When you change one D file you need to recompile ALL files that depend on the changed file. To compare with C/C++ which has source and header files you only need to recompile the source file if you change it * DMD doesn't keep the fully qualified module name when naming object files resulting in foo.bar will conflict with bar.bar. Issue 3541.

[The above is from the packaging system discussion in the "a cabal for D?" thread; here I am branching this to a new topic because I'd like anyone interested in incremental build processes to notice and contribute if they have input.] That is an interesting observation, Jacob. Thank you for pointing that out. Is there anything else (open question anyone) that would prevent D projects from doing incremental builds? Lack of support for incremental builds is a show stopper. Or in this case, the show would never get funded to begin with. Or to ask it another way, what would it take to get incremental builds?
Mar 19 2011
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
Even AutoHotkey has a system of finding and installing libraries. It
can run example code without installing (well, it is a scripting
language), show the source, documentation and some other things.

Snapshot: http://i.imgur.com/Uv1Gr.jpg
Mar 19 2011
prev sibling next sibling parent reply Jonas Drewsen <jdrewsen nospam.com> writes:
On 17/03/11 21.44, Jason E. Aten wrote:
 Please correct me if I'm wrong, but I observe that there doesn't appear
 to be a package management system / standard repository for D libraries.
 Or is there?

 I'm talking about something as easy to use as R's CRAN,
 install.packages("rforest")

or cpan for perl, ctan for latex, dpgk/apt for debian, cabal for Haskell/ Hackage, etc. If there's not a commonly utilized one currently, perhaps we could "borrow" cabal, with a trivial port. cabal is Haskell's package manager. Not only does having a standard package install system facilitate adoption, it greatly facilitates code sharing and library maturation.

It seems quite a few people are interested in such a tool. I suggest that those who are interested in the subject form some kind of SIG and creates a page on http://prowiki.org/wiki4d with ideas, insights, requirements that the rest of the community can follow. I think discussions still need to be done in this newsgroup though. This means that someone will actually need to implement something or modify an existing solution at some point for it to make sense. But initially we need to settle on something that "everybody" would be happy with and would support as the standard tool if possible. I'm willing to throw in some time implementing it when the curl stuff that I'm currently working on is done. Hopefully others can put some work in this tool as well to get things rolling. Anyone think this is a good idea? Should I create a SIG page? /Jonas
Mar 22 2011
parent Jacob Carlborg <doob me.com> writes:
On 2011-03-22 12:03, Jonas Drewsen wrote:
 On 17/03/11 21.44, Jason E. Aten wrote:
 Please correct me if I'm wrong, but I observe that there doesn't appear
 to be a package management system / standard repository for D libraries.
 Or is there?

 I'm talking about something as easy to use as R's CRAN,
 install.packages("rforest")

or cpan for perl, ctan for latex, dpgk/apt for debian, cabal for Haskell/ Hackage, etc. If there's not a commonly utilized one currently, perhaps we could "borrow" cabal, with a trivial port. cabal is Haskell's package manager. Not only does having a standard package install system facilitate adoption, it greatly facilitates code sharing and library maturation.

It seems quite a few people are interested in such a tool. I suggest that those who are interested in the subject form some kind of SIG and creates a page on http://prowiki.org/wiki4d with ideas, insights, requirements that the rest of the community can follow. I think discussions still need to be done in this newsgroup though. This means that someone will actually need to implement something or modify an existing solution at some point for it to make sense. But initially we need to settle on something that "everybody" would be happy with and would support as the standard tool if possible. I'm willing to throw in some time implementing it when the curl stuff that I'm currently working on is done. Hopefully others can put some work in this tool as well to get things rolling. Anyone think this is a good idea? Should I create a SIG page? /Jonas

I think it's a good idea. I'm planning to start on a tool like this when I finished the std.net.isemail module. -- /Jacob Carlborg
Mar 22 2011
prev sibling parent Russel Winder <russel russel.org.uk> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Sat, 2011-04-02 at 21:35 +0200, Jacob Carlborg wrote:
[ . . . ]
 I don't think it's the package manger's job to build the=20
 application/library. The package manager operates on a level higher than=

 a build tool. The build tool is working with files, tracking=20
 dependencies among files and building applications/libraries. The=20
 package manager is working with packages, tracking dependencies among=20
 packages.

Haskell's Cabal has blurred the lines between package management and build in many people's minds, but it only succeeds when in isolation. As soon as there is a platform packaging system it is clear that build and packaging must be separate. Though obviously you can get into JVM/Maven, and indeed Go, situations where again the lines get blurred. For something such as D which should have some ability to work with C and C++ libraries, keeping the separation is likely a better route.
 I was planning to creating my own build tool that would be integrated=20
 with the package manager. The package manager would use the build tool=

 to track the files making up the package. But other build tool would be=

 supported as well, not forcing anyone to use my build tool.

Is there a need for a brand new build tool. Are CMake, SCons and Waf not sufficient to the task? Clearly there are no perfect tools, there is always room for evolution and new things. However the history in the JVM-related milieu and the Go arena, indicate that the knee jerk reaction of "we have a new language, therefore we must write a new build framework" leads to a lot of isolationism, creation of ghettos, and a lot of wasted effort. Go has at least 6 brand new build frameworks, all of which were deemed necessary, and none of which have gained any traction -- the core development team are believers that Make is all that is needed. Rake has gained little traction outside the Ruby community. Leiningen has little traction outside the Clojure community. SBT has little traction outside the Scala community. Gradle is getting significant traction across the Java and Groovy communities as it solves problems inherent in the wildly successful Ant and Maven frameworks. SCons and Waf arose as dynamic language realizations of ADG-based systems to replace Make and Autotools respectively. They have a clear reason for existence and are successful, indeed have traction. CMake likewise. Note here though there is a yawning gap between build frameworks in the JVM-verse and the native-verse. Tools from one are generally rubbish in the other, to the detriment of build. If there is to be a new build framework to compete against SCons, Waf, CMake, Gradle and Maven I suggest it must be designed from the outset to deal with the vastly different problems of C, C++, D, LaTeX, Python, Ruby, Java, Groovy, Scala, Clojure, Erlang, Haskell, Chapel, Fortress, X10, etc. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel russel.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Apr 02 2011