www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Make dub part of the standard dmd distribution

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Let's make this part of 2.068:

https://issues.dlang.org/show_bug.cgi?id=14636

It's preapproved. Who would want to work on it?


Thanks,

Andrei
May 31 2015
next sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 1 June 2015 at 09:01, Andrei Alexandrescu via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 Let's make this part of 2.068:

 https://issues.dlang.org/show_bug.cgi?id=14636

 It's preapproved. Who would want to work on it?
Please declare a standard unix location for D 'includes'. Nobody agrees where in the filesystem D files should be. I use /usr/include/d2/ for my stuff (I saw it precedented a few times before, but it doesn't seem that great), but I want a standard place that stuff bundled by linux package managers can agree on. As for dub, I'd use it if it worked like a package manager; dub get libcurl-d libqt-d zlib-d libsdl2-d etc I have no use for it as a build system, and therefore it's expression of dependencies is no use to me. I just want something that works the same way as '-dev' packages already work perfectly well in linux, that is, they fetch headers and libs, and put them in a standard location that all the tooling can find.
May 31 2015
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/31/15 8:48 PM, Manu via Digitalmars-d wrote:
 As for dub, I'd use it if it worked like a package manager; dub get
 libcurl-d libqt-d zlib-d libsdl2-d etc
 I have no use for it as a build system, and therefore it's expression
 of dependencies is no use to me. I just want something that works the
 same way as '-dev' packages already work perfectly well in linux, that
 is, they fetch headers and libs, and put them in a standard location
 that all the tooling can find.
I thought it does that. If dub doesn't allow me to type one command to download and install all I need about a package, we need to add that pronto. I consider it a dealbreaker. Andrei
May 31 2015
parent reply "Brad Anderson" <eco gnuk.net> writes:
On Monday, 1 June 2015 at 04:36:06 UTC, Andrei Alexandrescu wrote:
 On 5/31/15 8:48 PM, Manu via Digitalmars-d wrote:
 As for dub, I'd use it if it worked like a package manager; 
 dub get
 libcurl-d libqt-d zlib-d libsdl2-d etc
 I have no use for it as a build system, and therefore it's 
 expression
 of dependencies is no use to me. I just want something that 
 works the
 same way as '-dev' packages already work perfectly well in 
 linux, that
 is, they fetch headers and libs, and put them in a standard 
 location
 that all the tooling can find.
I thought it does that. If dub doesn't allow me to type one command to download and install all I need about a package, we need to add that pronto. I consider it a dealbreaker. Andrei
dub fetch does this already (though probably not quite what you are thinking of). You'd need to specify the paths manually because if it installed them to the global compiler paths we'd have dependency hell (what if 5 projects I have need 3 different versions of a library?). Also, you'd need root permissions. That's not really how you use dub though. dub simply isn't a good fit for people who want it to be a system package manager. Its goals are different. If people want that they should work on getting libraries added to their preferred system's package registries. With dub you specify the dependencies in the dub config file, not in some obscure section of an INSTALL file as a command the users need to run. You can checkout a project using dub and with a single command have dub download and build all the dependencies (and their dependencies) and then build your project against them. dub is about making it easy for 99% of users. If you need your own build system then using dub just to download packages is overkill. Use git submodules or add something to do a download of your dependencies from github as part of your custom build system.
May 31 2015
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/31/15 10:05 PM, Brad Anderson wrote:
 dub fetch does this already (though probably not quite what you are
 thinking of). You'd need to specify the paths manually because if it
 installed them to the global compiler paths we'd have dependency hell
 (what if 5 projects I have need 3 different versions of a library?).
 Also, you'd need root permissions.
Usability is key here. Installing stuff should really be as simple as pasting a command. That's how Flash won over Java applets etc. etc. I installed a Python program the other day. I think it's rather obscure - it's called "lcinvestor". I got from knowing nothing about how to install Python stuff to having the application running in minutes. The installation page at https://pypi.python.org/pypi/lcinvestor mentions at a point: ==== The easiest way to install is with pip: sudo pip install lcinvestor ==== So I ran that. It didn't find pip. So I googled for it (first hit if you google for "pip"). Essentially I downloaded the script get-pip.py and then ran: python get-pip.py That worked nicely. Then I tried running pip again, it worked, and boom, I was up and running. But wait, there's more. I then wanted to install lcinvestor on a machine I didn't have sudo access to. Sure enough, searching for things like ``pip without sudo'' took me to http://kazhack.org/?post/2014/12/12/pip-gem-install-without-sudo where I figured all I had to do: pip install --local lcinvestor Boom, I was set - the installation target was ~/.local/bin/. Sure enough that was in PATH and all so the program just worked. That's the benchmark, folks. We need to get as good or better.
 That's not really how you use dub though. dub simply isn't a good fit
 for people who want it to be a system package manager. Its goals are
 different. If people want that they should work on getting libraries
 added to their preferred system's package registries.
Then we need a different program than dub. "dip" :o).
 With dub you specify the dependencies in the dub config file, not in
 some obscure section of an INSTALL file as a command the users need to
 run. You can checkout a project using dub and with a single command have
 dub download and build all the dependencies (and their dependencies) and
 then build your project against them.
That seems reasonable.
 dub is about making it easy for 99% of users. If you need your own build
 system then using dub just to download packages is overkill. Use git
 submodules or add something to do a download of your dependencies from
 github as part of your custom build system.
That seems reasonable, too. So are we there yet? Andrei
May 31 2015
parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 06/01/2015 01:21 AM, Andrei Alexandrescu wrote:
 I installed a Python program the other day. I think it's rather obscure
 - it's called "lcinvestor". I got from knowing nothing about how to
 install Python stuff to having the application running in minutes.
That doesn't even remotely match my experience trying to use/install python-based software/libs: https://semitwist.com/articles/article/view/why-i-hate-python-or-any-dynamic-language-really
May 31 2015
prev sibling next sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 1 June 2015 at 15:05, Brad Anderson via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Monday, 1 June 2015 at 04:36:06 UTC, Andrei Alexandrescu wrote:
 On 5/31/15 8:48 PM, Manu via Digitalmars-d wrote:
 As for dub, I'd use it if it worked like a package manager; dub get
 libcurl-d libqt-d zlib-d libsdl2-d etc
 I have no use for it as a build system, and therefore it's expression
 of dependencies is no use to me. I just want something that works the
 same way as '-dev' packages already work perfectly well in linux, that
 is, they fetch headers and libs, and put them in a standard location
 that all the tooling can find.
I thought it does that. If dub doesn't allow me to type one command to download and install all I need about a package, we need to add that pronto. I consider it a dealbreaker. Andrei
dub fetch does this already (though probably not quite what you are thinking of). You'd need to specify the paths manually because if it installed them to the global compiler paths we'd have dependency hell (what if 5 projects I have need 3 different versions of a library?). Also, you'd need root permissions.
Yeah, but regardless, that's what I want. I don't have version hell with C libs distributed this way...? Is this a problem that people are specifically trying to avoid?
 That's not really how you use dub though. dub simply isn't a good fit for
 people who want it to be a system package manager. Its goals are different.
 If people want that they should work on getting libraries added to their
 preferred system's package registries.
Right, so, someone decide a path, we'll write it on dlang.org, and then everyone will agree and fall in line :)
 With dub you specify the dependencies in the dub config file, not in some
 obscure section of an INSTALL file as a command the users need to run. You
 can checkout a project using dub and with a single command have dub download
 and build all the dependencies (and their dependencies) and then build your
 project against them.
I get it, it sounds great... if your app suits the model. I have no D-only projects, all my programs combine many languages and ecosystems. There are also existing build systems that are well established that I prefer to use, integrate with IDE's, etc. I don't mind if people use dub, but I just want a way to fetch libs that the compilers will then find automatically.
 dub is about making it easy for 99% of users. If you need your own build
 system then using dub just to download packages is overkill. Use git
 submodules or add something to do a download of your dependencies from
 github as part of your custom build system.
Point is, I don't have to do this with C. I just install the dev package, once, and I'm done. Package manager distributes updates automatically, everything it exactly how I want it. It's just not a wheel I have any interest in reinventing. I'm not trying to heckle. I just want someone to declare the word on where in the filesystem we put .d files, parallel to c includes. I think the single most important thing is for bindings against C libs that are installed by common -dev packages, it would be easiest if the bindings were fetched and installed the same way as the libs they refer to.
May 31 2015
next sibling parent Rikki Cattermole <alphaglosined gmail.com> writes:
On 1/06/2015 5:57 p.m., Manu via Digitalmars-d wrote:
 On 1 June 2015 at 15:05, Brad Anderson via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On Monday, 1 June 2015 at 04:36:06 UTC, Andrei Alexandrescu wrote:
 On 5/31/15 8:48 PM, Manu via Digitalmars-d wrote:
 As for dub, I'd use it if it worked like a package manager; dub get
 libcurl-d libqt-d zlib-d libsdl2-d etc
 I have no use for it as a build system, and therefore it's expression
 of dependencies is no use to me. I just want something that works the
 same way as '-dev' packages already work perfectly well in linux, that
 is, they fetch headers and libs, and put them in a standard location
 that all the tooling can find.
I thought it does that. If dub doesn't allow me to type one command to download and install all I need about a package, we need to add that pronto. I consider it a dealbreaker. Andrei
dub fetch does this already (though probably not quite what you are thinking of). You'd need to specify the paths manually because if it installed them to the global compiler paths we'd have dependency hell (what if 5 projects I have need 3 different versions of a library?). Also, you'd need root permissions.
Yeah, but regardless, that's what I want. I don't have version hell with C libs distributed this way...? Is this a problem that people are specifically trying to avoid?
 That's not really how you use dub though. dub simply isn't a good fit for
 people who want it to be a system package manager. Its goals are different.
 If people want that they should work on getting libraries added to their
 preferred system's package registries.
Right, so, someone decide a path, we'll write it on dlang.org, and then everyone will agree and fall in line :)
 With dub you specify the dependencies in the dub config file, not in some
 obscure section of an INSTALL file as a command the users need to run. You
 can checkout a project using dub and with a single command have dub download
 and build all the dependencies (and their dependencies) and then build your
 project against them.
I get it, it sounds great... if your app suits the model. I have no D-only projects, all my programs combine many languages and ecosystems. There are also existing build systems that are well established that I prefer to use, integrate with IDE's, etc. I don't mind if people use dub, but I just want a way to fetch libs that the compilers will then find automatically.
 dub is about making it easy for 99% of users. If you need your own build
 system then using dub just to download packages is overkill. Use git
 submodules or add something to do a download of your dependencies from
 github as part of your custom build system.
Point is, I don't have to do this with C. I just install the dev package, once, and I'm done. Package manager distributes updates automatically, everything it exactly how I want it. It's just not a wheel I have any interest in reinventing. I'm not trying to heckle. I just want someone to declare the word on where in the filesystem we put .d files, parallel to c includes. I think the single most important thing is for bindings against C libs that are installed by common -dev packages, it would be easiest if the bindings were fetched and installed the same way as the libs they refer to.
Perhaps a new dub command (--locate)? $ dub build <package> --version=<version> --arch=<arch> --compiler=<compiler> --locate Base: ~/.dub/<package>-<version>/ Binary: ~/.dub/<package>-<version>/output.lib ImportPath: ~/.dub/<package>-<version>/source Pass Well you get the general gist. Fetches and builds if necessary for specific arch ext. Really it just minimizes output and only output location info.
May 31 2015
prev sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 06/01/2015 01:57 AM, Manu via Digitalmars-d wrote:
 On 1 June 2015 at 15:05, Brad Anderson via Digitalmars-d
 That's not really how you use dub though. dub simply isn't a good fit for
 people who want it to be a system package manager. Its goals are different.
 If people want that they should work on getting libraries added to their
 preferred system's package registries.
Right, so, someone decide a path, we'll write it on dlang.org, and then everyone will agree and fall in line :)
Not sure how serious/joking you are about that, but when has declaring a standard whatever like that ever worked for anything ever? ;) Linux can't even agree with Linux on where things should go, apparently that's why C on linux has pkg-config.
May 31 2015
next sibling parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 1 Jun 2015 08:45, "Nick Sabalausky via Digitalmars-d" <
digitalmars-d puremagic.com> wrote:
 On 06/01/2015 01:57 AM, Manu via Digitalmars-d wrote:
 On 1 June 2015 at 15:05, Brad Anderson via Digitalmars-d
 That's not really how you use dub though. dub simply isn't a good fit
for
 people who want it to be a system package manager. Its goals are
different.
 If people want that they should work on getting libraries added to their
 preferred system's package registries.
Right, so, someone decide a path, we'll write it on dlang.org, and then everyone will agree and fall in line :)
Not sure how serious/joking you are about that, but when has declaring a
standard whatever like that ever worked for anything ever? ;) Linux can't even agree with Linux on where things should go, apparently that's why C on linux has pkg-config.

Leave it to the distribution is the safe option in my experience.  To have
something along the lines of what Manu wants, I guess we need something
like virtualenv to allow building in a clean/standard environment.
May 31 2015
parent reply "Adam D. Ruppe" <destructionator gmail.com> writes:
Writing to something that requires root is just doing it wrong 
IMO, unless the user specifically opts in to it.

/usr/anything is thus wrong, it should all be done locally. My 
preference is actually right in the current directory, do it on a 
per-project level. But if you must go higher, the user's home 
directory is also acceptable.
Jun 01 2015
parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Monday, 1 June 2015 at 13:03:28 UTC, Adam D. Ruppe wrote:
 Writing to something that requires root is just doing it wrong 
 IMO, unless the user specifically opts in to it.

 /usr/anything is thus wrong, it should all be done locally. My 
 preference is actually right in the current directory, do it on 
 a per-project level. But if you must go higher, the user's home 
 directory is also acceptable.
I believe Manu was referring to (system) package managers. Those that are run by users (like rubygems/bundle, pip, dub) must of course never write outside the users' home directory.
Jun 01 2015
prev sibling next sibling parent Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 1 June 2015 at 16:54, Iain Buclaw via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 1 Jun 2015 08:45, "Nick Sabalausky via Digitalmars-d"
 <digitalmars-d puremagic.com> wrote:
 On 06/01/2015 01:57 AM, Manu via Digitalmars-d wrote:
 On 1 June 2015 at 15:05, Brad Anderson via Digitalmars-d
 That's not really how you use dub though. dub simply isn't a good fit
 for

 people who want it to be a system package manager. Its goals are
 different.
 If people want that they should work on getting libraries added to their
 preferred system's package registries.
Right, so, someone decide a path, we'll write it on dlang.org, and then everyone will agree and fall in line :)
Not sure how serious/joking you are about that, but when has declaring a standard whatever like that ever worked for anything ever? ;) Linux can't even agree with Linux on where things should go, apparently that's why C on linux has pkg-config.
Leave it to the distribution is the safe option in my experience. To have something along the lines of what Manu wants, I guess we need something like virtualenv to allow building in a clean/standard environment.
Yeah, I think I can see 2 parallel problems here. 1. There is a lib installed from a -dev package managed by the distribution... I just want the complementary .d files. (this is what I actually care about) 2. There is some open-source D code which isn't distributed as a binary, it's just a git repo and you fetch it and build it locally. (I find that I rarely need this, so I don't have much opinion on this case) For case 1, my preference would be a distro managed package alongside the lib itself, and installed into a standard location. If dub could pull the bindings I want and put them in some common location, fine. For case 2... I dunno. What if you offer a lib that falls into case 2; source is available, user can fetch and build against it locally, but it contains C code too? dub isn't exactly a build system which can express a complex build environment. I can't create a dub package for my engine, which might be of interest to D gamedevs.
Jun 01 2015
prev sibling parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 1 Jun 2015 09:09, "Manu via Digitalmars-d" <digitalmars-d puremagic.com>
wrote:
 On 1 June 2015 at 16:54, Iain Buclaw via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On 1 Jun 2015 08:45, "Nick Sabalausky via Digitalmars-d"
 <digitalmars-d puremagic.com> wrote:
 On 06/01/2015 01:57 AM, Manu via Digitalmars-d wrote:
 On 1 June 2015 at 15:05, Brad Anderson via Digitalmars-d
 That's not really how you use dub though. dub simply isn't a good fit
 for

 people who want it to be a system package manager. Its goals are
 different.
 If people want that they should work on getting libraries added to
their
 preferred system's package registries.
Right, so, someone decide a path, we'll write it on dlang.org, and then everyone will agree and fall in line :)
Not sure how serious/joking you are about that, but when has declaring
a
 standard whatever like that ever worked for anything ever? ;) Linux
can't
 even agree with Linux on where things should go, apparently that's why
C on
 linux has pkg-config.
Leave it to the distribution is the safe option in my experience. To
have
 something along the lines of what Manu wants, I guess we need something
like
 virtualenv to allow building in a clean/standard environment.
Yeah, I think I can see 2 parallel problems here. 1. There is a lib installed from a -dev package managed by the distribution... I just want the complementary .d files. (this is what I actually care about) 2. There is some open-source D code which isn't distributed as a binary, it's just a git repo and you fetch it and build it locally. (I find that I rarely need this, so I don't have much opinion on this case) For case 1, my preference would be a distro managed package alongside the lib itself, and installed into a standard location. If dub could pull the bindings I want and put them in some common location, fine. For case 2... I dunno. What if you offer a lib that falls into case 2; source is available, user can fetch and build against it locally, but it contains C code too? dub isn't exactly a build system which can express a complex build environment. I can't create a dub package for my engine, which might be of interest to D gamedevs.
In one sense this can be solved at the distribution level. If dub was provided through your package manager, the package maintainers can ensure that dub was configured to understand where all system sources are located (or will be located). Can all of dubs default settings be dumped to json and be overriden?
Jun 01 2015
next sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Monday, 1 June 2015 at 07:19:28 UTC, Iain Buclaw wrote:
 In one sense this can be solved at the distribution level.  If 
 dub was
 provided through your package manager, the package maintainers 
 can ensure
 that dub was configured to understand where all system sources 
 are located
 (or will be located).

 Can all of dubs default settings be dumped to json and be 
 overriden?
The way I do it right now is simply adding -I/usr/include/dlang -I/usr/include/dlang/<compiler> to dmd.conf / ldc.conf / patch GDC in a similar way. And surely if dub would ever attempt to write anything in /usr on its own I will simply throw it away from Arch packages.
Jun 01 2015
parent reply "Wyatt" <wyatt.epp gmail.com> writes:
On Monday, 1 June 2015 at 11:14:43 UTC, Dicebot wrote:
 The way I do it right now is simply adding -I/usr/include/dlang 
 -I/usr/include/dlang/<compiler> to dmd.conf / ldc.conf / patch 
 GDC in a similar way.

 And surely if dub would ever attempt to write anything in /usr 
 on its own I will simply throw it away from Arch packages.
This whole conversation reminds me a lot of the perl ecosystem. How does Arch deal with CPAN? -Wyatt
Jun 01 2015
parent reply "Dicebot" <public dicebot.lv> writes:
On Monday, 1 June 2015 at 15:25:13 UTC, Wyatt wrote:
 On Monday, 1 June 2015 at 11:14:43 UTC, Dicebot wrote:
 The way I do it right now is simply adding 
 -I/usr/include/dlang -I/usr/include/dlang/<compiler> to 
 dmd.conf / ldc.conf / patch GDC in a similar way.

 And surely if dub would ever attempt to write anything in /usr 
 on its own I will simply throw it away from Arch packages.
This whole conversation reminds me a lot of the perl ecosystem. How does Arch deal with CPAN? -Wyatt
Same as pip and gems and all similar kind of crap. Bad. That is why I don't use them. Though it is hard to invent a package management for interpreted language that is not insane. It is much better with native ones because runtime dependencies are separated from development dependencies.
Jun 01 2015
parent reply Jacob Carlborg <doob me.com> writes:
On 2015-06-01 18:56, Dicebot wrote:

 Though it is hard to invent a package management for interpreted
 language that is not insane. It is much better with native ones because
 runtime dependencies are separated from development dependencies.
Rubygems supports development dependencies. -- /Jacob Carlborg
Jun 02 2015
parent reply "Dicebot" <public dicebot.lv> writes:
On Tuesday, 2 June 2015 at 07:02:50 UTC, Jacob Carlborg wrote:
 On 2015-06-01 18:56, Dicebot wrote:

 Though it is hard to invent a package management for 
 interpreted
 language that is not insane. It is much better with native 
 ones because
 runtime dependencies are separated from development 
 dependencies.
Rubygems supports development dependencies.
You must have though of different "development dependencies". I don't mean tools and stuff like that but all sorts of small static libraries (in native compilation world). With scripting language this unavoidably propagates on the end user unless you merge all deps into your sources inline for distribution (which no one does).
Jun 02 2015
parent Jacob Carlborg <doob me.com> writes:
On 2015-06-02 20:54, Dicebot wrote:

 You must have though of different "development dependencies". I don't
 mean tools and stuff like that but all sorts of small static libraries
 (in native compilation world). With scripting language this unavoidably
 propagates on the end user unless you merge all deps into your sources
 inline for distribution (which no one does).
Oh, you mean like that. -- /Jacob Carlborg
Jun 02 2015
prev sibling parent =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com> writes:
Am 01.06.2015 um 09:19 schrieb Iain Buclaw via Digitalmars-d:
 On 1 Jun 2015 09:09, "Manu via Digitalmars-d"
 <digitalmars-d puremagic.com <mailto:digitalmars-d puremagic.com>> wrote:
  >
  > On 1 June 2015 at 16:54, Iain Buclaw via Digitalmars-d
  > <digitalmars-d puremagic.com <mailto:digitalmars-d puremagic.com>> wrote:
  > >
  > > On 1 Jun 2015 08:45, "Nick Sabalausky via Digitalmars-d"
  > > <digitalmars-d puremagic.com <mailto:digitalmars-d puremagic.com>>
 wrote:
  > >>
  > >> On 06/01/2015 01:57 AM, Manu via Digitalmars-d wrote:
  > >>>
  > >>> On 1 June 2015 at 15:05, Brad Anderson via Digitalmars-d
  > >>>>
  > >>>> That's not really how you use dub though. dub simply isn't a
 good fit
  > >>>> for
  > >>>>
  > >>>> people who want it to be a system package manager. Its goals are
  > >>>> different.
  > >>>> If people want that they should work on getting libraries added
 to their
  > >>>> preferred system's package registries.
  > >>>
  > >>>
  > >>> Right, so, someone decide a path, we'll write it on dlang.org
 <http://dlang.org>, and
  > >>> then everyone will agree and fall in line :)
  > >>>
  > >>
  > >> Not sure how serious/joking you are about that, but when has
 declaring a
  > >> standard whatever like that ever worked for anything ever? ;)
 Linux can't
  > >> even agree with Linux on where things should go, apparently that's
 why C on
  > >> linux has pkg-config.
  > >>
  > >
  > > Leave it to the distribution is the safe option in my experience.
 To have
  > > something along the lines of what Manu wants, I guess we need
 something like
  > > virtualenv to allow building in a clean/standard environment.
  >
  > Yeah, I think I can see 2 parallel problems here.
  > 1. There is a lib installed from a -dev package managed by the
  > distribution... I just want the complementary .d files. (this is what
  > I actually care about)
  > 2. There is some open-source D code which isn't distributed as a
  > binary, it's just a git repo and you fetch it and build it locally. (I
  > find that I rarely need this, so I don't have much opinion on this
  > case)
  >
  > For case 1, my preference would be a distro managed package alongside
  > the lib itself, and installed into a standard location. If dub could
  > pull the bindings I want and put them in some common location, fine.
  > For case 2... I dunno. What if you offer a lib that falls into case 2;
  > source is available, user can fetch and build against it locally, but
  > it contains C code too? dub isn't exactly a build system which can
  > express a complex build environment.
  > I can't create a dub package for my engine, which might be of interest
  > to D gamedevs.

 In one sense this can be solved at the distribution level.  If dub was
 provided through your package manager, the package maintainers can
 ensure that dub was configured to understand where all system sources
 are located (or will be located).

 Can all of dubs default settings be dumped to json and be overriden?
It would be possible to put a ".dub/settings.json" with custom settings into the distribution package (hmm, /etc/dub/settings.json or similar should probably supported in addition to that...).
Jun 04 2015
prev sibling next sibling parent Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 1 Jun 2015 07:57, "Manu via Digitalmars-d" <digitalmars-d puremagic.com>
wrote:
 On 1 June 2015 at 15:05, Brad Anderson via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On Monday, 1 June 2015 at 04:36:06 UTC, Andrei Alexandrescu wrote:
 On 5/31/15 8:48 PM, Manu via Digitalmars-d wrote:
 As for dub, I'd use it if it worked like a package manager; dub get
 libcurl-d libqt-d zlib-d libsdl2-d etc
 I have no use for it as a build system, and therefore it's expression
 of dependencies is no use to me. I just want something that works the
 same way as '-dev' packages already work perfectly well in linux, that
 is, they fetch headers and libs, and put them in a standard location
 that all the tooling can find.
I thought it does that. If dub doesn't allow me to type one command to download and install
all I
 need about a package, we need to add that pronto. I consider it a
 dealbreaker.


 Andrei
dub fetch does this already (though probably not quite what you are
thinking
 of). You'd need to specify the paths manually because if it installed
them
 to the global compiler paths we'd have dependency hell (what if 5
projects I
 have need 3 different versions of a library?). Also, you'd need root
 permissions.
Yeah, but regardless, that's what I want. I don't have version hell with C libs distributed this way...? Is this a problem that people are specifically trying to avoid?
 That's not really how you use dub though. dub simply isn't a good fit
for
 people who want it to be a system package manager. Its goals are
different.
 If people want that they should work on getting libraries added to their
 preferred system's package registries.
Right, so, someone decide a path, we'll write it on dlang.org, and then everyone will agree and fall in line :)
 With dub you specify the dependencies in the dub config file, not in
some
 obscure section of an INSTALL file as a command the users need to run.
You
 can checkout a project using dub and with a single command have dub
download
 and build all the dependencies (and their dependencies) and then build
your
 project against them.
I get it, it sounds great... if your app suits the model. I have no D-only projects, all my programs combine many languages and ecosystems. There are also existing build systems that are well established that I prefer to use, integrate with IDE's, etc. I don't mind if people use dub, but I just want a way to fetch libs that the compilers will then find automatically.
Just to be clear, libs are source libraries, right?
 dub is about making it easy for 99% of users. If you need your own build
 system then using dub just to download packages is overkill. Use git
 submodules or add something to do a download of your dependencies from
 github as part of your custom build system.
Point is, I don't have to do this with C. I just install the dev package, once, and I'm done. Package manager distributes updates automatically, everything it exactly how I want it. It's just not a wheel I have any interest in reinventing.
This is a feature of your distribution, and not the language itself. I'm having talks with the Debian toolchain maintainer, we want to start shipping D programs and libraries with Debian/Ubuntu. Binary libraries are going to be the most interesting problem here because dmd and ldc will be shut out from using them. This is a semi call-out to the ldc devs, we should really align our ABIs together.
Jun 01 2015
prev sibling next sibling parent Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 1 June 2015 at 17:09, Iain Buclaw via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 1 Jun 2015 07:57, "Manu via Digitalmars-d" <digitalmars-d puremagic.com>
 wrote:
 On 1 June 2015 at 15:05, Brad Anderson via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On Monday, 1 June 2015 at 04:36:06 UTC, Andrei Alexandrescu wrote:
 On 5/31/15 8:48 PM, Manu via Digitalmars-d wrote:
 As for dub, I'd use it if it worked like a package manager; dub get
 libcurl-d libqt-d zlib-d libsdl2-d etc
 I have no use for it as a build system, and therefore it's expression
 of dependencies is no use to me. I just want something that works the
 same way as '-dev' packages already work perfectly well in linux, that
 is, they fetch headers and libs, and put them in a standard location
 that all the tooling can find.
I thought it does that. If dub doesn't allow me to type one command to download and install all I need about a package, we need to add that pronto. I consider it a dealbreaker. Andrei
dub fetch does this already (though probably not quite what you are thinking of). You'd need to specify the paths manually because if it installed them to the global compiler paths we'd have dependency hell (what if 5 projects I have need 3 different versions of a library?). Also, you'd need root permissions.
Yeah, but regardless, that's what I want. I don't have version hell with C libs distributed this way...? Is this a problem that people are specifically trying to avoid?
 That's not really how you use dub though. dub simply isn't a good fit
 for
 people who want it to be a system package manager. Its goals are
 different.
 If people want that they should work on getting libraries added to their
 preferred system's package registries.
Right, so, someone decide a path, we'll write it on dlang.org, and then everyone will agree and fall in line :)
 With dub you specify the dependencies in the dub config file, not in
 some
 obscure section of an INSTALL file as a command the users need to run.
 You
 can checkout a project using dub and with a single command have dub
 download
 and build all the dependencies (and their dependencies) and then build
 your
 project against them.
I get it, it sounds great... if your app suits the model. I have no D-only projects, all my programs combine many languages and ecosystems. There are also existing build systems that are well established that I prefer to use, integrate with IDE's, etc. I don't mind if people use dub, but I just want a way to fetch libs that the compilers will then find automatically.
Just to be clear, libs are source libraries, right?
Well, it should support libs that aren't JUST source. Binary libs needs some .d files to declare the API and all that, so at least some source would always present. A lib on its own us no use :) But any solution needs to support closed-source libraries too. Not everything can be conveniently distributed as full-source, and many libs are bindings against binary C libs which would probably be sourced externally.
 dub is about making it easy for 99% of users. If you need your own build
 system then using dub just to download packages is overkill. Use git
 submodules or add something to do a download of your dependencies from
 github as part of your custom build system.
Point is, I don't have to do this with C. I just install the dev package, once, and I'm done. Package manager distributes updates automatically, everything it exactly how I want it. It's just not a wheel I have any interest in reinventing.
This is a feature of your distribution, and not the language itself. I'm having talks with the Debian toolchain maintainer, we want to start shipping D programs and libraries with Debian/Ubuntu.
Do want! :)
 Binary libraries are going to be the most interesting problem here because
 dmd and ldc will be shut out from using them.
? Why is?
 This is a semi call-out to the ldc devs, we should really align our ABIs
 together.
Oh, are the LDC/GDC ABI's not consistent on linux? Surely many/most libs are just references to C '-dev' packages as dependencies?
Jun 01 2015
prev sibling next sibling parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 1 June 2015 at 09:37, Manu via Digitalmars-d <digitalmars-d puremagic.com
 wrote:
 On 1 June 2015 at 17:09, Iain Buclaw via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On 1 Jun 2015 07:57, "Manu via Digitalmars-d" <
digitalmars-d puremagic.com>
 wrote:
 On 1 June 2015 at 15:05, Brad Anderson via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On Monday, 1 June 2015 at 04:36:06 UTC, Andrei Alexandrescu wrote:
 On 5/31/15 8:48 PM, Manu via Digitalmars-d wrote:
 As for dub, I'd use it if it worked like a package manager; dub get
 libcurl-d libqt-d zlib-d libsdl2-d etc
 I have no use for it as a build system, and therefore it's
expression
 of dependencies is no use to me. I just want something that works
the
 same way as '-dev' packages already work perfectly well in linux,
that
 is, they fetch headers and libs, and put them in a standard location
 that all the tooling can find.
I thought it does that. If dub doesn't allow me to type one command to download and install
all
 I
 need about a package, we need to add that pronto. I consider it a
 dealbreaker.


 Andrei
dub fetch does this already (though probably not quite what you are thinking of). You'd need to specify the paths manually because if it installed them to the global compiler paths we'd have dependency hell (what if 5 projects I have need 3 different versions of a library?). Also, you'd need root permissions.
Yeah, but regardless, that's what I want. I don't have version hell with C libs distributed this way...? Is this a problem that people are specifically trying to avoid?
 That's not really how you use dub though. dub simply isn't a good fit
 for
 people who want it to be a system package manager. Its goals are
 different.
 If people want that they should work on getting libraries added to
their
 preferred system's package registries.
Right, so, someone decide a path, we'll write it on dlang.org, and then everyone will agree and fall in line :)
 With dub you specify the dependencies in the dub config file, not in
 some
 obscure section of an INSTALL file as a command the users need to run.
 You
 can checkout a project using dub and with a single command have dub
 download
 and build all the dependencies (and their dependencies) and then build
 your
 project against them.
I get it, it sounds great... if your app suits the model. I have no D-only projects, all my programs combine many languages and ecosystems. There are also existing build systems that are well established that I prefer to use, integrate with IDE's, etc. I don't mind if people use dub, but I just want a way to fetch libs that the compilers will then find automatically.
Just to be clear, libs are source libraries, right?
Well, it should support libs that aren't JUST source. Binary libs needs some .d files to declare the API and all that, so at least some source would always present. A lib on its own us no use :) But any solution needs to support closed-source libraries too. Not everything can be conveniently distributed as full-source, and many libs are bindings against binary C libs which would probably be sourced externally.
 dub is about making it easy for 99% of users. If you need your own
build
 system then using dub just to download packages is overkill. Use git
 submodules or add something to do a download of your dependencies from
 github as part of your custom build system.
Point is, I don't have to do this with C. I just install the dev package, once, and I'm done. Package manager distributes updates automatically, everything it exactly how I want it. It's just not a wheel I have any interest in reinventing.
This is a feature of your distribution, and not the language itself. I'm having talks with the Debian toolchain maintainer, we want to start
shipping
 D programs and libraries with Debian/Ubuntu.
Do want! :)
 Binary libraries are going to be the most interesting problem here
because
 dmd and ldc will be shut out from using them.
? Why is?
 This is a semi call-out to the ldc devs, we should really align our ABIs
 together.
Oh, are the LDC/GDC ABI's not consistent on linux? Surely many/most libs are just references to C '-dev' packages as dependencies?
Our druntimes need to be aligned up. The first thing you'd notice is that each may give undefined references to things the other compiler does not include in it's runtime.
Jun 01 2015
parent "Dicebot" <public dicebot.lv> writes:
On Monday, 1 June 2015 at 07:42:02 UTC, Iain Buclaw wrote:
 Our druntimes need to be aligned up.  The first thing you'd 
 notice is that
 each may give undefined references to things the other compiler 
 does not
 include in it's runtime.
Unifying GDC/LDC ABI would simplify packager life a lot.
Jun 01 2015
prev sibling next sibling parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Mon, 01 Jun 2015 05:05:20 +0000, Brad Anderson wrote:

 dub is about making it easy for 99% of users. If you need your own build
 system then using dub just to download packages is overkill. Use git
 submodules or add something to do a download of your dependencies from
 github as part of your custom build system.
now you killed dub for "normal" GNU/Linux programmer. dub as a build tool sux. no need to discuss that, it simply sux. and now you killed the only sensible way to use it: as package manager. that's why, for example, i never considered adding anything to dub=20 repository: that repository is completely useless for me. and other=20 people who aren't use dub to build their projects will ignore dub=20 repository too.=
Jun 01 2015
next sibling parent reply "extrawurst" <stephan extrawurst.org> writes:
On Monday, 1 June 2015 at 10:24:46 UTC, ketmar wrote:
 On Mon, 01 Jun 2015 05:05:20 +0000, Brad Anderson wrote:

 dub is about making it easy for 99% of users. If you need your 
 own build
 system then using dub just to download packages is overkill. 
 Use git
 submodules or add something to do a download of your 
 dependencies from
 github as part of your custom build system.
now you killed dub for "normal" GNU/Linux programmer. dub as a build tool sux. no need to discuss that, it simply sux.
why do you think it sucks ?
 and now you killed the only sensible way to use it: as package 
 manager.

 that's why, for example, i never considered adding anything to 
 dub
 repository: that repository is completely useless for me. and 
 other
 people who aren't use dub to build their projects will ignore 
 dub
 repository too.
Jun 01 2015
next sibling parent reply "Atila Neves" <atila.neves gmail.com> writes:
 dub as a build tool sux. no need to discuss that, it simply 
 sux.
why do you think it sucks ?
dub is a good package manager, but it's not a good build system. For small projects that just want to list dependencies and have it build, it's sufficient. Any larger project will have needs that won't be easily solved by using dub. Which is why I wrote my own. Atila
Jun 01 2015
parent reply "extrawurst" <stephan extrawurst.org> writes:
On Monday, 1 June 2015 at 12:24:18 UTC, Atila Neves wrote:
 dub as a build tool sux. no need to discuss that, it simply 
 sux.
why do you think it sucks ?
dub is a good package manager, but it's not a good build system. For small projects that just want to list dependencies and have it build, it's sufficient. Any larger project will have needs that won't be easily solved by using dub. Which is why I wrote my own. Atila
Hi Atila, you are talking about cross language builds ? Other than that limitation, what do you miss ? I have a couple of rather sizable projects running with dub without problems and one could argue vibe.d is a pretty big project too which works fine. The D community already suffered enough from not uniting workforces behind at least SOME standards (hint at tango), a plethora of options for newcomers to choose from for basics like building is also daunting IMHO. Stephan
Jun 01 2015
next sibling parent "Atila Neves" <atila.neves gmail.com> writes:
On Monday, 1 June 2015 at 12:34:47 UTC, extrawurst wrote:
 dub is a good package manager, but it's not a good build 
 system. For small projects that just want to list dependencies 
 and have it build, it's sufficient. Any larger project will 
 have needs that won't be easily solved by using dub. Which is 
 why I wrote my own.

 Atila
Hi Atila, you are talking about cross language builds ?
That too.
 Other than that limitation, what do you miss ?
Everything I could do (and have) in cmake. Here are two really simple examples: 1) Automatically run a script to generate the file containing "main" before doing the build, with proper dependency tracking 2) Concurrently building release and unittest builds I've worked on complicated build systems before, I know the kind of thing I'd need.
 I have a couple of rather sizable projects running with dub 
 without problems and one could argue vibe.d is a pretty big 
 project too which works fine.
I wouldn't consider vibe.d to be a large project.
 The D community already suffered enough from not uniting 
 workforces behind at least SOME standards (hint at tango), a 
 plethora of options for newcomers to choose from for basics 
 like building is also daunting IMHO.
I fully support dub being the default package manager and for most people/projects it is a good enough build system. That's why my build system builds on dub instead of wanting to replace it. It leaves the package dependency part to dub and focusses on build descriptions that can leverage dub's existing knowledge of the build. Atila
Jun 01 2015
prev sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 06/01/2015 08:34 AM, extrawurst wrote:
 On Monday, 1 June 2015 at 12:24:18 UTC, Atila Neves wrote:
 dub as a build tool sux. no need to discuss that, it simply sux.
why do you think it sucks ?
dub is a good package manager, but it's not a good build system. For small projects that just want to list dependencies and have it build, it's sufficient. Any larger project will have needs that won't be easily solved by using dub. Which is why I wrote my own. Atila
Hi Atila, you are talking about cross language builds ? Other than that limitation, what do you miss ?
Unlike rdmd, dub always tries to compile all sources regardless of whether they're actually imported. This has been a constant source of problems for me, including breakage of conditional importing under various circumstances.
Jun 01 2015
parent reply Jacob Carlborg <doob me.com> writes:
On 2015-06-01 20:39, Nick Sabalausky wrote:

 Unlike rdmd, dub always tries to compile all sources regardless of
 whether they're actually imported. This has been a constant source of
 problems for me, including breakage of conditional importing under
 various circumstances.
Isn't that the only way to compile it if the project is a library? without having to use workarounds like create a new module which imports all other modules. -- /Jacob Carlborg
Jun 01 2015
parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 06/02/2015 02:53 AM, Jacob Carlborg wrote:
 On 2015-06-01 20:39, Nick Sabalausky wrote:

 Unlike rdmd, dub always tries to compile all sources regardless of
 whether they're actually imported. This has been a constant source of
 problems for me, including breakage of conditional importing under
 various circumstances.
Isn't that the only way to compile it if the project is a library? without having to use workarounds like create a new module which imports all other modules.
My libs always have a "package.d" or equivalent anyway, so I don't find it to be any sort of "workaround". Even without that, rdmd-style still works if the lib is being used as a source lib rather than a pre-compiled static (or dynamic) lib. But yea, if you're doing a precompiled lib and you don't want any sort of "root" module (don't see it as all that big of a deal though), then yea I guess you'd have to collect the list of sources dub-style instead of rdmd-style.
Jun 02 2015
prev sibling parent ketmar <ketmar ketmar.no-ip.org> writes:
On Mon, 01 Jun 2015 11:59:12 +0000, extrawurst wrote:

 dub as a build tool sux. no need to discuss that, it simply sux.
=20 why do you think it sucks ?
'cause it's not more than a simple script in it's core. it can't do good=20 conditional builds, it can't track dependencies for generated files or=20 non-D code, and so on. it's "rdmd with some bells and some features=20 stripped". this is fine for simple pure D projects, but for complex projects one=20 have to use supplement build tool. and if we have *two* build tools in=20 use, it's logical to remove one that is more tied to the language. and=20 it's dub. so while it's ok to use dub to build simple things, it's not a good build=20 system. my k8jam, while can't do "consolidated" (i.e. non-separate ;-)=20 builds, can build D code with C/C++ libraries (which are built from=20 source too), generate files, track dependencies, do 'configure' work,=20 allows "dlang.require" operators with dependency tracking between=20 libraries and so on. it can't download libraries, though -- i choose to=20 not implement that, hoping that dub can fulfill that role.=
Jun 02 2015
prev sibling parent reply "Adam D. Ruppe" <destructionator gmail.com> writes:
On Monday, 1 June 2015 at 10:24:46 UTC, ketmar wrote:
 and now you killed the only sensible way to use it: as package 
 manager.
I don't care for it as a package manager either because it is too complex. As you know, I like to do my things as individual files. I don't go through the pain of making a new github repo with a gigantic folder layout for every little thing. One of the things I really like about D is how modules are encapsulated into individual files. The code is there, the documentation is there, the tests are there. No separate headers or anything else. The only package management scheme I'd even consider using is one that has a per-file option... and even then, I probably wouldn't bother because individual files are so easy to manage anyway that downloading a new program to use them doesn't exceed the "existing option is good enough for me" threshold. But even if I don't use it myself, it would be kinda nice if the repository could handle my files just so I can toss it up there and not have to answer questions about that anymore. I'd be willing to add them all individually to the repo, even listing the dependencies manually if I had to (tip: make this a web form that I can just submit to add stuff to it, no json upload).... but I'm not willing to separate my little repo into 25 other repos and separate my folder into 125 other folders to be added to it.
Jun 01 2015
next sibling parent reply Rikki Cattermole <alphaglosined gmail.com> writes:
On 2/06/2015 1:11 a.m., Adam D. Ruppe wrote:
 On Monday, 1 June 2015 at 10:24:46 UTC, ketmar wrote:
 and now you killed the only sensible way to use it: as package manager.
I don't care for it as a package manager either because it is too complex. As you know, I like to do my things as individual files. I don't go through the pain of making a new github repo with a gigantic folder layout for every little thing. One of the things I really like about D is how modules are encapsulated into individual files. The code is there, the documentation is there, the tests are there. No separate headers or anything else. The only package management scheme I'd even consider using is one that has a per-file option... and even then, I probably wouldn't bother because individual files are so easy to manage anyway that downloading a new program to use them doesn't exceed the "existing option is good enough for me" threshold. But even if I don't use it myself, it would be kinda nice if the repository could handle my files just so I can toss it up there and not have to answer questions about that anymore. I'd be willing to add them all individually to the repo, even listing the dependencies manually if I had to (tip: make this a web form that I can just submit to add stuff to it, no json upload).... but I'm not willing to separate my little repo into 25 other repos and separate my folder into 125 other folders to be added to it.
Something I've thought about but haven't suggested up to now is making dub-repo to support personal projects. Basically one repo per person preconfigured if e.g. Github/Bitbucket is specified. github.com/rikkimax/dubcode Just as an idea for what I'm meaning url wise. So then when specifying a dependency in dub, its just a matter of saying <person>:<subproject> It would also help with people having there own util libraries all listed. Of course then you'd only have to dump your code in there and have a dub file with each file being its own sub package. Seems simple to implement, but idk if it is the answer.
Jun 01 2015
next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
I think the way I'd do it is you still make projects in the repo 
that are referenced like normally, just it refers to a file in a 
specific repo instead of a whole repo.

Maybe using the subpackage feature it can get close enough. I'll 
try using an empty source folder then do the others as a single 
additional file.

But actually, the multiple listing on the repo is the most 
important part to me, these are really individual things, they 
shouldn't be listed as "miscellaneous from me" on the listing...
Jun 01 2015
prev sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 06/01/2015 09:22 AM, Rikki Cattermole wrote:
 Something I've thought about but haven't suggested up to now is making
 dub-repo to support personal projects.
 Basically one repo per person preconfigured if e.g. Github/Bitbucket is
 specified.

 github.com/rikkimax/dubcode

 Just as an idea for what I'm meaning url wise.

 So then when specifying a dependency in dub, its just a matter of saying
 <person>:<subproject>
Already works. $ git clone https://github.com/rikkimax/dubcode.git path-to-dubcode $ dub add-local path-to-dubcode Done.
Jun 01 2015
parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 06/01/2015 02:51 PM, Nick Sabalausky wrote:
 On 06/01/2015 09:22 AM, Rikki Cattermole wrote:
 Something I've thought about but haven't suggested up to now is making
 dub-repo to support personal projects.
 Basically one repo per person preconfigured if e.g. Github/Bitbucket is
 specified.

 github.com/rikkimax/dubcode

 Just as an idea for what I'm meaning url wise.

 So then when specifying a dependency in dub, its just a matter of saying
 <person>:<subproject>
Already works. $ git clone https://github.com/rikkimax/dubcode.git path-to-dubcode $ dub add-local path-to-dubcode Done.
Ehh, to clarify a few things: $ cd my-project $ cat dub.json { "name": "my-project", "dependencies": { "not-in-dub-repo": "~>1.1.0" } } $ git clone https://github.com/user/not-in-dub-repo.git path/to/not-in-dub-repo $ cat path/to/not-in-dub-repo/dub.json { "name": "not-in-dub-repo" } $ dub add-local path/to/not-in-dub-repo
Jun 01 2015
prev sibling next sibling parent reply "ponce" <contact gam3sfrommars.fr> writes:
On Monday, 1 June 2015 at 13:11:43 UTC, Adam D. Ruppe wrote:
 The only package management scheme I'd even consider using is 
 one that has a per-file option... and even then, I probably 
 wouldn't bother because individual files are so easy to manage 
 anyway that downloading a new program to use them doesn't 
 exceed the "existing option is good enough for me" threshold.
What happens when you find a bug in your library? Then every user has new work to do to merge the changes. With DUB you don't do a thing for patch versions in user code. _This provide a way to warn your users and make them upgrade_. This is the main interest of DUB. You could use git submodule for that, but not everyone use git, and not everyone want to search commit logs for the right version.
Jun 01 2015
parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Monday, 1 June 2015 at 14:09:32 UTC, ponce wrote:
 What happens when you find a bug in your library?
Users still have to test bug fixes because what I consider a bug they might consider a feature. I dread software updates not because downloading a file or applying a patch is hard, but because I fear unrelated and/or silent breakage.
Jun 01 2015
prev sibling parent ketmar <ketmar ketmar.no-ip.org> writes:
On Mon, 01 Jun 2015 13:11:42 +0000, Adam D. Ruppe wrote:

 One of the things I really like about D is how modules are encapsulated
 into individual files. The code is there, the documentation is there,
 the tests are there. No separate headers or anything else.
i like this too.
 But even if I don't use it myself, it would be kinda nice if the
 repository could handle my files just so I can toss it up there and not
 have to answer questions about that anymore. I'd be willing to add them
 all individually to the repo, even listing the dependencies manually if
 I had to (tip: make this a web form that I can just submit to add stuff
 to it, no json upload).... but I'm not willing to separate my little
 repo into 25 other repos and separate my folder into 125 other folders
 to be added to it.
yep. i don't feel like splitting my IV repo to alot of individual=20 repositories. quite the contrary, i'm moving modules from separate=20 repositories to IV when they are ready. some of IV things are perfectly=20 usable with vanilla, so they can go to code.dlang.org, but... but there=20 is no way to do that now.=
Jun 02 2015
prev sibling parent "Morbid.Obesity" <Morbid.Obesity mail.com> writes:
On Monday, 1 June 2015 at 05:05:21 UTC, Brad Anderson wrote:
 On Monday, 1 June 2015 at 04:36:06 UTC, Andrei Alexandrescu 
 wrote:
 On 5/31/15 8:48 PM, Manu via Digitalmars-d wrote:
 As for dub, I'd use it if it worked like a package manager; 
 dub get
 libcurl-d libqt-d zlib-d libsdl2-d etc
 I have no use for it as a build system, and therefore it's 
 expression
 of dependencies is no use to me. I just want something that 
 works the
 same way as '-dev' packages already work perfectly well in 
 linux, that
 is, they fetch headers and libs, and put them in a standard 
 location
 that all the tooling can find.
I thought it does that. If dub doesn't allow me to type one command to download and install all I need about a package, we need to add that pronto. I consider it a dealbreaker. Andrei
dub fetch does this already (though probably not quite what you are thinking of). You'd need to specify the paths manually because if it installed them to the global compiler paths we'd have dependency hell (what if 5 projects I have need 3 different versions of a library?). Also, you'd need root permissions.
I wonder if essentially hashing a library(ultimately each file) could be used to avoid dll hell. Essentially hash the .d files and then different versions will have different hashes. When you compile your project and it is successful, the compiler/linker can write a configuration file that includes the hashes of the various files used. This then could be used to avoid dll, if one has a proper way to "check" for versioning(online database) then one can find the proper versions. Obviously small changes to any d file, even if it is non-breaking will change the hash. This is not a bad thing but since there will be no correlation between hashes and versions one must have a database of versions and their hashes. The good thing is that if, say, a project doesn't compile after updating a "library" to the "wrong" versions, the configuration file won't be modified since the compiler won't be able to compile the code. One can then easily use the (old) config file, if it exists from a previous (working) build, to deal with the versioning issue. (e.g., hash is different for mylib.d, means wrong version. Get correct version) In fact, maybe the way to go is for the config file essentially to keep a history of several last working builds, e.g., 05/31/15 - mylib.d[3FAB9999ABCCC34C3C] - Compiled OK - myotherlib.d[4C934DFFFFFFF34C3C] - Compiled OK - myproject.d [FFABC33CCA4367C4EE] - Compiled OK - Project [3493493428439294292] - Compiled OK 06/01/15 - mylib.d[3FAB9999ABCCC34C3C] - Compiled OK - myotherlib.d[4C954D5F3F2F434EEC] - Compiled OK - myproject.d [FFABC33CCA4367C4EE] - Failed - Project [3493493428439294292] - Failed In the above case, the myotherlib.d's hash changed. the project did not compile. Hence we have a versioning issue. It is easy to see what the problem is. One then simply needs an online datable to match the hashes to the versions or simply search all "versions" and hash them. Also, one benefits from simply hashing any file to get it's "version". If you had 10 myotherlib.d files from the past but don't know which is which(different copies scattered across different backups, etc), simply hashing them and finding the correct one(in this case, 4C934DFFFFFFF34C3C) can be used to get the project working again. Since most of this stuff can be automated, it seems like it would be a better way to deal with versioning than the standard methods which relies on users correctly defining the versions and maintain them.
Jun 01 2015
prev sibling next sibling parent reply "weaselcat" <weaselcat gmail.com> writes:
On Monday, 1 June 2015 at 03:48:31 UTC, Manu wrote:
 On 1 June 2015 at 09:01, Andrei Alexandrescu via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 Let's make this part of 2.068:

 https://issues.dlang.org/show_bug.cgi?id=14636

 It's preapproved. Who would want to work on it?
Please declare a standard unix location for D 'includes'. Nobody agrees where in the filesystem D files should be. I use /usr/include/d2/ for my stuff (I saw it precedented a few times before, but it doesn't seem that great), but I want a standard place that stuff bundled by linux package managers can agree on. As for dub, I'd use it if it worked like a package manager; dub get libcurl-d libqt-d zlib-d libsdl2-d etc I have no use for it as a build system, and therefore it's expression of dependencies is no use to me. I just want something that works the same way as '-dev' packages already work perfectly well in linux, that is, they fetch headers and libs, and put them in a standard location that all the tooling can find.
run dub fetch --help
May 31 2015
parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 1 June 2015 at 14:57, weaselcat via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Monday, 1 June 2015 at 03:48:31 UTC, Manu wrote:
 On 1 June 2015 at 09:01, Andrei Alexandrescu via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 Let's make this part of 2.068:

 https://issues.dlang.org/show_bug.cgi?id=14636

 It's preapproved. Who would want to work on it?
Please declare a standard unix location for D 'includes'. Nobody agrees where in the filesystem D files should be. I use /usr/include/d2/ for my stuff (I saw it precedented a few times before, but it doesn't seem that great), but I want a standard place that stuff bundled by linux package managers can agree on. As for dub, I'd use it if it worked like a package manager; dub get libcurl-d libqt-d zlib-d libsdl2-d etc I have no use for it as a build system, and therefore it's expression of dependencies is no use to me. I just want something that works the same way as '-dev' packages already work perfectly well in linux, that is, they fetch headers and libs, and put them in a standard location that all the tooling can find.
run dub fetch --help
Interesting. I'm amazed this never came up before in discussion... I've talked about this so many times. So, DMD/LDC/GDC know where to look to find these packages? What happens if the package includes a binary lib? That that, I still want someone to declare an official path for D 'includes' in the *nix filesystem, so D lib packages have somewhere to install...
May 31 2015
next sibling parent "weaselcat" <weaselcat gmail.com> writes:
On Monday, 1 June 2015 at 05:27:00 UTC, Manu wrote:
 On 1 June 2015 at 14:57, weaselcat via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On Monday, 1 June 2015 at 03:48:31 UTC, Manu wrote:
 On 1 June 2015 at 09:01, Andrei Alexandrescu via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 Let's make this part of 2.068:

 https://issues.dlang.org/show_bug.cgi?id=14636

 It's preapproved. Who would want to work on it?
Please declare a standard unix location for D 'includes'. Nobody agrees where in the filesystem D files should be. I use /usr/include/d2/ for my stuff (I saw it precedented a few times before, but it doesn't seem that great), but I want a standard place that stuff bundled by linux package managers can agree on. As for dub, I'd use it if it worked like a package manager; dub get libcurl-d libqt-d zlib-d libsdl2-d etc I have no use for it as a build system, and therefore it's expression of dependencies is no use to me. I just want something that works the same way as '-dev' packages already work perfectly well in linux, that is, they fetch headers and libs, and put them in a standard location that all the tooling can find.
run dub fetch --help
Interesting. I'm amazed this never came up before in discussion... I've talked about this so many times. So, DMD/LDC/GDC know where to look to find these packages? What happens if the package includes a binary lib?
dub uses git to manage packages, it keeps the list of D packages on http://code.dlang.org/ the help is a bit unintuitive, it just gives a brief overview with dub --help, you have to issue a subcommand to get the help about it.
 That that, I still want someone to declare an official path for 
 D
 'includes' in the *nix filesystem, so D lib packages have 
 somewhere to
 install...
the problem is that XDG didn't really define a standard for user-level libraries and binaries, so it's a huge mess. Dub is actually violating part of the XDG standard as is because it defaults to ~/.dub http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html
May 31 2015
prev sibling next sibling parent "Atila Neves" <atila.neves gmail.com> writes:
On Monday, 1 June 2015 at 05:27:00 UTC, Manu wrote:
 On 1 June 2015 at 14:57, weaselcat via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On Monday, 1 June 2015 at 03:48:31 UTC, Manu wrote:
 On 1 June 2015 at 09:01, Andrei Alexandrescu via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 Let's make this part of 2.068:

 https://issues.dlang.org/show_bug.cgi?id=14636

 It's preapproved. Who would want to work on it?
Please declare a standard unix location for D 'includes'. Nobody agrees where in the filesystem D files should be. I use /usr/include/d2/ for my stuff (I saw it precedented a few times before, but it doesn't seem that great), but I want a standard place that stuff bundled by linux package managers can agree on. As for dub, I'd use it if it worked like a package manager; dub get libcurl-d libqt-d zlib-d libsdl2-d etc I have no use for it as a build system, and therefore it's expression of dependencies is no use to me. I just want something that works the same way as '-dev' packages already work perfectly well in linux, that is, they fetch headers and libs, and put them in a standard location that all the tooling can find.
run dub fetch --help
Interesting. I'm amazed this never came up before in discussion... I've talked about this so many times. So, DMD/LDC/GDC know where to look to find these packages? What happens if the package includes a binary lib?
No. But dub knows where the packages are, and the usual way to use it is to use dub to compile. If not, there's always `dub describe`, which will tell you exactly where the dependencies are. Atila
 That that, I still want someone to declare an official path for 
 D
 'includes' in the *nix filesystem, so D lib packages have 
 somewhere to
 install...
May 31 2015
prev sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Mon, 1 Jun 2015 15:26:51 +1000
schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>:

 So, DMD/LDC/GDC know where to look to find these packages?
IIRC no.
 What
 happens if the package includes a binary lib?
 
 That that, I still want someone to declare an official path for D
 'includes' in the *nix filesystem, so D lib packages have somewhere to
 install...
GDC looks in /usr/include/d2. Dicebot changed that to /usr/include/d/gdc for archlinux and there are some reasons why this is (for now) necessary: Sharing D sources/headers between compilers is possible, but there are some headers (druntime,phobos) which can be compiler specific. As long as these are not in a common import path it's not a problem though. The main problem is we don't have ABI compatibility. This means we can't share the libraries between compilers. So we could make 'import foo' work but linking with -lfoo without manually adjusting linker paths is not possible. There are some solutions, all have drawbacks: * Have compiler specific directories for libraries (/usr/lib/gdc/libvibed.a, /usr/lib/dmd/libvibed.a). Only works for static libraries. Can only have one version of a library installed * have per-library directories. I guess dub uses this. Can have multiple versions of the same library. Drawback: compiler can't know the correct library path.[1] * Shared libraries should be installed in a common location (/usr/lib) anyway. This is only possible if we have ABI compatibility[2]. Library versioning is limited to the standard C/C++ like versioning. I guess the main reason why we can't have a C/C++ like out of the box experience is ABI compatibility. There'll always be some quirks as long as we don't fix this. [1] you could actually build the directory structure logic into the compilers. Do something like dmd --library=vibe.d:0.7.23 where this argument sets up correct import paths and linking. I think .NET uses something similar (GAC). With ABI compatible compilers we might even make this work for shared libraries. Could be implemented as a small tool calling the compiler. You could also do something like this: Create a package format: .dlib (simply a renamed .tar.xz) Contents: - PACKAGEINFO //meta information (version) - include/* //include files - doc/* //documentation in standard format for IDEs - lib/dmd/libfoo.a - lib/gdc/libfoo.a Write a small tool xdmd. Implement: //simply move to /usr/share/dlib/libfoo/version/ //or ~/.share/dlib/libfoo/version/ xdmd --install libfoo.dlib Note: when installing using a systems package manager, packages could be installed in the same way: pacman -S libfoo could call xdmd --install. //extract /usr/share/dlib/libfoo/0.9.3/libfoo.dlib //to temporary directory. Pass link and include flags and //rest of arguments to dmd xdmd --library=foo:0.9.3 test.d //directly use local package xdmd --library=libfoo-0.9.3.dlib test.d This is essentially are very slimmed down dub. Would be a nice weekend exercise to write such a tool. [2] or you use one compiler exclusively.
Jun 01 2015
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2015-06-01 20:03, Johannes Pfau wrote:

 Create a package format: .dlib (simply a renamed .tar.xz)
 Contents:
     - PACKAGEINFO //meta information (version)
     - include/*   //include files
     - doc/*       //documentation in standard format for IDEs
     - lib/dmd/libfoo.a
     - lib/gdc/libfoo.a

 Write a small tool xdmd. Implement:
     //simply move to /usr/share/dlib/libfoo/version/
     //or ~/.share/dlib/libfoo/version/
     xdmd --install libfoo.dlib
     Note: when installing using a systems package manager, packages
     could be installed in the same way: pacman -S libfoo could call xdmd
     --install.

     //extract /usr/share/dlib/libfoo/0.9.3/libfoo.dlib
     //to temporary directory. Pass link and include flags and
     //rest of arguments to dmd
     xdmd --library=foo:0.9.3 test.d
     //directly use local package
     xdmd --library=libfoo-0.9.3.dlib test.d

 This is essentially are very slimmed down dub. Would be a nice weekend
 exercise to write such a tool.
So, a package manager ;). I tried that, but people were complaining that I used Ruby for the meta information and that binary packages would cause problems. Then Dub came and I gave up on that project. -- /Jacob Carlborg
Jun 01 2015
prev sibling next sibling parent Mathias Lang via Digitalmars-d <digitalmars-d puremagic.com> writes:
2015-06-01 20:03 GMT+02:00 Johannes Pfau via Digitalmars-d <
digitalmars-d puremagic.com>:

 The main problem is we don't have ABI compatibility. This means we can't
 share the libraries between compilers. So we could make 'import foo'
 work but linking with -lfoo without manually adjusting linker paths is
 not possible.

 There are some solutions, all have drawbacks:
 * Have compiler specific directories for libraries
   (/usr/lib/gdc/libvibed.a, /usr/lib/dmd/libvibed.a). Only works for
   static libraries. Can only have one version of a library installed
 * have per-library directories. I guess dub uses this. Can have
   multiple versions of the same library. Drawback: compiler can't know
   the correct library path.[1]
 * Shared libraries should be installed in a common location (/usr/lib)
   anyway. This is only possible if we have ABI compatibility[2].
   Library versioning is limited to the standard C/C++ like versioning.

 I guess the main reason why we can't have a C/C++ like out of the box
 experience is ABI compatibility. There'll always be some quirks as long
 as we don't fix this.
Glad to see that post ! But in addition of the ABI compatibility between compilers, we would need to have ABI compatibility between releases of the same major version if we were to walk the C[++] route, and we do not want that, for various reason, the most obvious one being that you won't be able to add an attribute like nogc / nothrow to a free function without doing a major release. There is good thing about C way of doing thing, but there is also some bad. For example, you can't have multiple versions of the same library installed - you don't need it -, but D definitely needs that. That also leads to major releases ending up in a different package - think qt4 / qt5 -. So, while C use SemVer for ABI compatibility, we should use SemVer for source compatibility, or we'll end up with an unmanageable mess.
Jun 02 2015
prev sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 2 June 2015 at 04:03, Johannes Pfau via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 Am Mon, 1 Jun 2015 15:26:51 +1000
 schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>:

 So, DMD/LDC/GDC know where to look to find these packages?
IIRC no.
 What
 happens if the package includes a binary lib?

 That that, I still want someone to declare an official path for D
 'includes' in the *nix filesystem, so D lib packages have somewhere to
 install...
GDC looks in /usr/include/d2. Dicebot changed that to /usr/include/d/gdc for archlinux and there are some reasons why this is (for now) necessary: Sharing D sources/headers between compilers is possible, but there are some headers (druntime,phobos) which can be compiler specific. As long as these are not in a common import path it's not a problem though. The main problem is we don't have ABI compatibility. This means we can't share the libraries between compilers. So we could make 'import foo' work but linking with -lfoo without manually adjusting linker paths is not possible. There are some solutions, all have drawbacks: * Have compiler specific directories for libraries (/usr/lib/gdc/libvibed.a, /usr/lib/dmd/libvibed.a). Only works for static libraries. Can only have one version of a library installed * have per-library directories. I guess dub uses this. Can have multiple versions of the same library. Drawback: compiler can't know the correct library path.[1] * Shared libraries should be installed in a common location (/usr/lib) anyway. This is only possible if we have ABI compatibility[2]. Library versioning is limited to the standard C/C++ like versioning. I guess the main reason why we can't have a C/C++ like out of the box experience is ABI compatibility. There'll always be some quirks as long as we don't fix this.
I'm inclined to suggest, this would be an excellent focus for 2.068... The compilers need to become aligned, and ABI issues seems like a critical issue to focus on.
Jun 02 2015
parent "David Nadlinger" <code klickverbot.at> writes:
On Wednesday, 3 June 2015 at 01:23:37 UTC, Manu wrote:
 I'm inclined to suggest, this would be an excellent focus for 
 2.068...
 The compilers need to become aligned, and ABI issues seems like 
 a
 critical issue to focus on.
There are much bigger fish to fry at this point. Phobos breaks ABI compatibility with every single release anyway. - David
Jun 03 2015
prev sibling next sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 05/31/2015 11:48 PM, Manu via Digitalmars-d wrote:
 As for dub, I'd use it if it worked like a package manager; dub get
 libcurl-d libqt-d zlib-d libsdl2-d etc
 I have no use for it as a build system, and therefore it's expression
 of dependencies is no use to me. I just want something that works the
 same way as '-dev' packages already work perfectly well in linux, that
 is, they fetch headers and libs, and put them in a standard location
 that all the tooling can find.
Ditto. Dub's great if you let it be your buildsystem, but as soon as you want to use it as a package-manager-only it becomes an uphill battle with dub fighting back every step of the way (I speak from experience). That problem is worse if your project is a library that you want fetchable through dub. That's why I've been working on some PRs to fix all that. The first step is here (building/expanding on a flag another contributer recently added): https://github.com/D-Programming-Language/dub/pull/572 (Although I think I may need to change --data=libs and separate system libs from dependency libs since linkers expect those to be passed in differently.) The next step will be to add a flag to have that data get output in a compiler-ready argument list (your choice of compiler, of course). That'll let you do something comparable to "pkg-config --cflags --libs". Then finally, a shortcut flag so you can just do something kinda like: rdmd --build-only `dub describe --cmdargs=rdmd` src/main.d And all your dependencies will be properly referenced (-I..., linker flags, etc).
May 31 2015
next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Monday, 1 June 2015 at 06:20:18 UTC, Nick Sabalausky wrote:
 Ditto. Dub's great if you let it be your buildsystem, but as 
 soon as you want to use it as a package-manager-only it becomes 
 an uphill battle with dub fighting back every step of the way 
 (I speak from experience). That problem is worse if your 
 project is a library that you want fetchable through dub.
And this is when Jacob Carlborg chimes in and says "I told you so." ;) His favorite complaint about dub has always been that it combined package management and the build tool into one. dub works fantastically if you want to do everything in the standard way without any funny stuff, but it clearly doesn't have the power of a tool like make, cmake, etc. As soon as you need to do anything funny - like include anything related to other languages in your build, or put things in a specific layout because of some company-specific thing, or anything that wasn't explicitly planned for by the folks writing dub, dub just won't work. Now, to be fair, I don't know how you can be pulling in all kinds of stray libraries with who-knows-what for their build systems and expect it to work very well via a single tool (at minimum, you have the problem that the machine that you're trying to build them on will likely be missing some of those tools), but that's pretty much what we need to handle the general case. Without that, dub will work great for the normal case (which _will_ be plenty for many, many projects), but it won't work in the general case - and corporate folks in particular are probably going to have to skip out on using it because of abnormal requirements on their part. Even simply splitting dub out so that it can pull in packages without necessarily being able to build them would be nice (maybe adding a flag to the dub.json file indicating whether dub can be used as a build tool for the project or whether it's supposed to just act as a package manager for it), though that does add the problem of code.dlang.org becoming more confusing, since you wouldn't be able to rely on all projects on it being built for you by dub as part of the process of grabbing packages. It's a complicate problem, and dub went a route which works in the 90% case, but doesn't work for more complicated cases, which will make dub unusable for some projects - especially corporate ones. I don't know how fixable it is without a major redesign of dub. - Jonathan M Davis
Jun 01 2015
next sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 06/01/2015 10:49 AM, Jonathan M Davis wrote:
 Even simply splitting dub out so that it can pull in packages without
 necessarily being able to build them would be nice (maybe adding a flag
 to the dub.json file indicating whether dub can be used as a build tool
 for the project or whether it's supposed to just act as a package
 manager for it), though that does add the problem of code.dlang.org
 becoming more confusing, since you wouldn't be able to rely on all
 projects on it being built for you by dub as part of the process of
 grabbing packages.
I've brought up something like that, but it was met with very strong opposition. :(
 It's a complicate problem, and dub went a route which works in the 90%
 case, but doesn't work for more complicated cases, which will make dub
 unusable for some projects - especially corporate ones. I don't know how
 fixable it is without a major redesign of dub.
Personally, I think "90%" is generous here, but I admit that's splitting hairs. Unfortunately, "less than 100%" is a big problem in this case because those libs/projects that don't fit the "90%" effectively cannot even participate in the dub repository for others to use (at the very LEAST, they get pounded with "fails to build via dub"). So NOT using dub to build is freaking viral. And *using* certain dub-based projects inside one which doesn't build via dub can be a major pain, as anyone who's tried to use vibe.d from a non-dub-based project should be able to attest (just *try* to figure out all the -I... and linker flags you need *and* double-check/maintain them every time you upgrade vibe.d). So anything less than 100% effectively splits the whole damn ecosystem. NOT acceptable. Dub's project generation feature is NOT a solution here, because you still have to manage builds dub's way, *exactly* as if you were using dub's build system, because...really...you still ARE. So all the dealbreakers for those 10+% projects still apply equally. On the plus side though, I *do* think this is basically solvable. The most immediate way (and what I'm working towards) is to create a dub.json that feeds dub a 'dummy-src/dummy.d' to keep dub's build system happy and safely away from the real source files, and then use: "(pre|post)-build-command-posix": "./my-real-buildscript" "(pre|post)-build-command-win": "my-real-buildscript" To trick dub into using a different build tool. It'd be easily to design/publish a skeleton dub.json to do exactly that. (Kinda sucks to NEED that, but at least it'll work, and once it's in place them maybe it can be made easier. The non-buildsytem parts of dub.json really ARE very trivial, and if someone had a problem with even that trivial little bit, then I don't see how they could develop any useful software in the first place.) I'm fairly certain the ONLY real thing still preventing all that above from actually working is that dub currently provides no (practical) way to retrieve the necessary compiler/linker args in order to actually use your dependencies (again, try using vibe.d in, say, an rdmd-based project). I say no "practical" way here because "dub describe -> parse the JSON -> extract the needed info -> convert the needed info to compiler flags" is NOT exactly practical or reasonable to expect people to do *just* to use RDMD or otherwise opt-out of dub's build-system.) Again though, I'm working on that: https://github.com/D-Programming-Language/dub/pull/572 That lets you directly grab any build info you need from dub. My next PR will then let you ask dub to automatically format the output for your compiler's CLI. And then one last one to provide convenience switches for the most likely configurations (like "--just-gimme-whatever-flags-I-need-to-pass-to-rdmd-so-I-can-actually-u e-my-dependencies", but obviously less verbose :)) With all that, any project should be able to participate in the dub ecosystem AND opt-out of dub's build system WITHOUT segregating the ecosystem in any way.
Jun 01 2015
parent reply "Atila Neves" <atila.neves gmail.com> writes:
 I'm fairly certain the ONLY real thing still preventing all 
 that above from actually working is that dub currently provides 
 no (practical) way to retrieve the necessary compiler/linker 
 args in order to actually use your dependencies (again, try 
 using vibe.d in, say, an rdmd-based project). I say no 
 "practical" way here because "dub describe -> parse the JSON -> 
 extract the needed info -> convert the needed info to compiler 
 flags" is NOT exactly practical or reasonable to expect people 
 to do *just* to use RDMD or otherwise opt-out of dub's 
 build-system.)
My build system already uses the output from `dub describe` to produce, well, a build system from it. I designed it with multiple backends in mind, right now the ones that I implemented are ninja and make. If not for `dub describe` bugs (I'm working on that), you'd be able to point it at any dub project and get make/ninja files for it. Or integrate whatever you want with the existing files to compile that dub already knows about. In fact, for any dub project that doesn't use configurations (that's the bug), you can do that _right now_. Atila
Jun 01 2015
parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 06/01/2015 01:43 PM, Atila Neves wrote:
 I'm fairly certain the ONLY real thing still preventing all that above
 from actually working is that dub currently provides no (practical)
 way to retrieve the necessary compiler/linker args in order to
 actually use your dependencies (again, try using vibe.d in, say, an
 rdmd-based project). I say no "practical" way here because "dub
 describe -> parse the JSON -> extract the needed info -> convert the
 needed info to compiler flags" is NOT exactly practical or reasonable
 to expect people to do *just* to use RDMD or otherwise opt-out of
 dub's build-system.)
My build system already uses the output from `dub describe` to produce, well, a build system from it. I designed it with multiple backends in mind, right now the ones that I implemented are ninja and make. If not for `dub describe` bugs (I'm working on that), you'd be able to point it at any dub project and get make/ninja files for it. Or integrate whatever you want with the existing files to compile that dub already knows about. In fact, for any dub project that doesn't use configurations (that's the bug), you can do that _right now_. Atila
That's certainly cool. I *do* think "dub describe"'s json is an awesome thing to have, exactly for those sorts of purposes. But I do maintain that an ability to easily opt-out of dub's buildsystem *needs* to be built-in and should not *require* external tools (though external tools should certainly still be possible). And this is even more crucial if dub's to be bundled with dmd and rdmd (the latter of which dub currently does not play nicely with).
Jun 01 2015
parent "Atila Neves" <atila.neves gmail.com> writes:
 But I do maintain that an ability to easily opt-out of dub's 
 buildsystem *needs* to be built-in and should not *require* 
 external tools (though external tools should certainly still be 
 possible). And this is even more crucial if dub's to be bundled 
 with dmd and rdmd (the latter of which dub currently does not 
 play nicely with).
Maybe I can manage to get my code into dub, I don't know. I think it's extremely useful to separate concerns, which is why I started with something new instead of trying to add code to dub. Like I said before, for most people its build system is enough. When it's not, there's reggae (my tool). I hope, anyway. The other reason is that I don't want to depend on dub. Maybe I have my priorities wrong, who knows. Atila
Jun 01 2015
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2015-06-01 16:49, Jonathan M Davis wrote:

 And this is when Jacob Carlborg chimes in and says "I told you so." ;)
 His favorite complaint about dub has always been that it combined
 package management and the build tool into one.
:) -- /Jacob Carlborg
Jun 01 2015
prev sibling parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com> writes:
Am 01.06.2015 um 16:49 schrieb Jonathan M Davis:
 On Monday, 1 June 2015 at 06:20:18 UTC, Nick Sabalausky wrote:
 Ditto. Dub's great if you let it be your buildsystem, but as soon as
 you want to use it as a package-manager-only it becomes an uphill
 battle with dub fighting back every step of the way (I speak from
 experience). That problem is worse if your project is a library that
 you want fetchable through dub.
And this is when Jacob Carlborg chimes in and says "I told you so." ;) His favorite complaint about dub has always been that it combined package management and the build tool into one.
This is actually completely irrelevant to the issue. Making the build part separate would not at all solve this. Any work on such a separate build tool (e.g. to support other languages) could instead as well go into DUB itself. Of course you could make the system completely build tool agnostic, but then you'd basically lose interoperability between packages, as each package might choose its own build tool.
 dub works fantastically if you want to do everything in the standard way
 without any funny stuff, but it clearly doesn't have the power of a tool
 like make, cmake, etc. As soon as you need to do anything funny - like
 include anything related to other languages in your build, or put things
 in a specific layout because of some company-specific thing, or anything
 that wasn't explicitly planned for by the folks writing dub, dub just
 won't work.
This is really not true. There is currently just one limitation w.r.t. the directory structure and that is the naming scheme of the folder to which packages are downloaded ("somepackage-1.0.0"). There are some packages that put their sources into the root of the repository and expect to be put into a folder with the name of the repository, which will then be added as an import folder. We are going to solve that. Everything else can be done with the sourcePaths, sourceFiles, importPaths, ... settings. You basically have the same possibilities as you have when directly invoking the compiler. Separate language support is still missing, but you always have the possibility to add generic "preBuildCommands" or "preGenerateCommands" to invoke separate tools or scripts. Of course it's desirable to get everything done with the same tool instead of relying on additional build time dependencies, but it's still *possible* to do these things.
 Now, to be fair, I don't know how you can be pulling in all kinds of
 stray libraries with who-knows-what for their build systems and expect
 it to work very well via a single tool (at minimum, you have the problem
 that the machine that you're trying to build them on will likely be
 missing some of those tools), but that's pretty much what we need to
 handle the general case. Without that, dub will work great for the
 normal case (which _will_ be plenty for many, many projects), but it
 won't work in the general case - and corporate folks in particular are
 probably going to have to skip out on using it because of abnormal
 requirements on their part.

 Even simply splitting dub out so that it can pull in packages without
 necessarily being able to build them would be nice (maybe adding a flag
 to the dub.json file indicating whether dub can be used as a build tool
 for the project or whether it's supposed to just act as a package
 manager for it), though that does add the problem of code.dlang.org
 becoming more confusing, since you wouldn't be able to rely on all
 projects on it being built for you by dub as part of the process of
 grabbing packages.
I guess using the proper targetType and preBuildCommand settings it should already be possible to invoke a separate build tool and still integrate the generated object/library files into the rest of the DUB build process. But I'd have to try that out - it would definitely be good to have something like this added to the cookbook [1]. Of course things like these always potentially degrade interoperability. DUB's build description was chosen explicitly to avoid unnecessary dependencies on a particular compiler/platform, which is where the most general build tools (especially "make") utterly fail.
 It's a complicate problem, and dub went a route which works in the 90%
 case, but doesn't work for more complicated cases, which will make dub
 unusable for some projects - especially corporate ones. I don't know how
 fixable it is without a major redesign of dub.
I don't see anything that wouldn't be fixable. In fact, there are really few things that are not possible now, even if not in the most comfortable way.
Jun 04 2015
parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 06/04/2015 04:39 AM, Sönke Ludwig wrote:
 Of course you could make the system completely build
 tool agnostic, but then you'd basically lose interoperability between
 packages, as each package might choose its own build tool.
With this PR and the upcoming follow-up to add --format=(list|some_compiler|whatever), there is NO issue with package interoperability (as long as a package's "import-paths" is correct): https://github.com/D-Programming-Language/dub/pull/572 Note that even WITHOUT dub, packages from different buildsystems *already* interop just fine. Even in C-land. All you ever need for packages to work together is the appropriate "build this package" command and the import/lib paths for the compiler/linker. Buildsystem is a package-local issue, not a cross-package issue.
Jun 04 2015
parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com> writes:
Am 04.06.2015 um 17:03 schrieb Nick Sabalausky:
 On 06/04/2015 04:39 AM, Sönke Ludwig wrote:
 Of course you could make the system completely build
 tool agnostic, but then you'd basically lose interoperability between
 packages, as each package might choose its own build tool.
With this PR and the upcoming follow-up to add --format=(list|some_compiler|whatever), there is NO issue with package interoperability (as long as a package's "import-paths" is correct): https://github.com/D-Programming-Language/dub/pull/572
Just to get this straight: Supporting external build tools like that and by writing the proper generator has always been the goal of DUB (being build tool agnostic). The built-in builder is basically mainly there for convenience (although there is no reason not to improve it to the point where it is a worthy competitor). But you still use DUB's build description in that case. What I really meant (and didn't really say) was if you completely remove all build description related fields from dub.json - then you lose interoperability.
 Note that even WITHOUT dub, packages from different buildsystems
 *already* interop just fine. Even in C-land. All you ever need for
 packages to work together is the appropriate "build this package"
 command and the import/lib paths for the compiler/linker. Buildsystem is
 a package-local issue, not a cross-package issue.
That's not what I had in mind with "interoperability". That would rather be gluing them together manually. Of course you can always do that, but then you lose a lot of potential for avoiding useless/redundant work. It means you are doing the job that the tool should (could) do. This also gets especially interesting and annoying when compiler flags (e.g. version identifiers) don't match, or the dependencies require advanced compiler flags. I really think C/C++ is the last thing we should take as our role model when it comes to this topic (apart from C's strict ABI perhaps).
Jun 04 2015
parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 06/04/2015 11:33 AM, Sönke Ludwig wrote:
 Am 04.06.2015 um 17:03 schrieb Nick Sabalausky:
 On 06/04/2015 04:39 AM, Sönke Ludwig wrote:
 Of course you could make the system completely build
 tool agnostic, but then you'd basically lose interoperability between
 packages, as each package might choose its own build tool.
With this PR and the upcoming follow-up to add --format=(list|some_compiler|whatever), there is NO issue with package interoperability (as long as a package's "import-paths" is correct): https://github.com/D-Programming-Language/dub/pull/572
Just to get this straight: Supporting external build tools like that and by writing the proper generator has always been the goal of DUB (being build tool agnostic). The built-in builder is basically mainly there for convenience (although there is no reason not to improve it to the point where it is a worthy competitor). But you still use DUB's build description in that case. What I really meant (and didn't really say) was if you completely remove all build description related fields from dub.json - then you lose interoperability.
Oh, right. I have no issue with dub *having* a built-in buildsystem. (And it is fine and convenient for a lot of things, even if not everything.) I've just hit roadblocks trying to use other buildsystems instead and am only just now getting around to addressing those issues.
Jun 04 2015
prev sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 06/01/2015 02:20 AM, Nick Sabalausky wrote:
 On 05/31/2015 11:48 PM, Manu via Digitalmars-d wrote:
 As for dub, I'd use it if it worked like a package manager; dub get
 libcurl-d libqt-d zlib-d libsdl2-d etc
 I have no use for it as a build system, and therefore it's expression
 of dependencies is no use to me. I just want something that works the
 same way as '-dev' packages already work perfectly well in linux, that
 is, they fetch headers and libs, and put them in a standard location
 that all the tooling can find.
Ditto. Dub's great if you let it be your buildsystem, but as soon as you want to use it as a package-manager-only it becomes an uphill battle with dub fighting back every step of the way (I speak from experience). That problem is worse if your project is a library that you want fetchable through dub. That's why I've been working on some PRs to fix all that. The first step is here (building/expanding on a flag another contributer recently added): https://github.com/D-Programming-Language/dub/pull/572 (Although I think I may need to change --data=libs and separate system libs from dependency libs since linkers expect those to be passed in differently.) The next step will be to add a flag to have that data get output in a compiler-ready argument list (your choice of compiler, of course). That'll let you do something comparable to "pkg-config --cflags --libs".
All the above is now included in the PR: https://github.com/D-Programming-Language/dub/pull/572 Working example: ------------------------------ $ cd /home/nick/proj/dub $ dub describe --data=main-source-file --data=options \ --data=versions --data=import-paths '/home/nick/proj/dub/source/app.d' -debug -g -w -version=DubUseCurl -version=Have_dub '-I/home/nick/proj/dub/source/' $ dub describe --compiler=ldc --data=options --data=versions -d-debug -g -w -oq -od=.dub/obj -oq -od=.dub/obj -d-version=DubUseCurl -d-version=Have_dub $ dub describe --data-list --data=options --data=versions debugMode debugInfo warningsAsErrors DubUseCurl Have_dub ------------------------------ Of course, "dub describe --help" for the full list of options. Only thing not included yet is this:
 Then finally, a shortcut flag so you can just do something kinda like:

 rdmd --build-only `dub describe --cmdargs=rdmd` src/main.d

 And all your dependencies will be properly referenced (-I..., linker
 flags, etc).
Jun 05 2015
parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 06/05/2015 12:45 PM, Nick Sabalausky wrote:
 https://github.com/D-Programming-Language/dub/pull/572

 Working example:

 ------------------------------
 $ cd /home/nick/proj/dub

 $ dub describe --data=main-source-file --data=options \
 --data=versions --data=import-paths
 '/home/nick/proj/dub/source/app.d' -debug -g -w -version=DubUseCurl
 -version=Have_dub '-I/home/nick/proj/dub/source/'

 $ dub describe --compiler=ldc --data=options --data=versions
 -d-debug -g -w -oq -od=.dub/obj -oq -od=.dub/obj -d-version=DubUseCurl
 -d-version=Have_dub

 $ dub describe --data-list --data=options --data=versions
 debugMode
 debugInfo
 warningsAsErrors

 DubUseCurl
 Have_dub
 ------------------------------
It also works even if your project doesn't have a dub.json :) $ mkdir ~/my-proj $ cd ~/my-proj $ dub fetch vibe-d $ dub describe vibe-d --data=versions --data=import-paths -version=VibeLibeventDriver -version=Have_vibe_d -version=Have_libevent -version=Have_openssl '-I/home/nick/.dub/packages/vibe-d-0.7.23-beta.1/source/' '-I/home/nick/.dub/packages/libevent-2.0.1_2.0.16/' '-I/home/nick/.dub/packages/openssl-1.1.4_1.0.1g/'
Jun 05 2015
prev sibling next sibling parent reply Bruno Medeiros <bruno.do.medeiros+dng gmail.com> writes:
On 01/06/2015 04:48, Manu via Digitalmars-d wrote:
 Please declare a standard unix location for D 'includes'. Nobody
 agrees where in the filesystem D files should be.
 I use /usr/include/d2/ for my stuff (I saw it precedented a few times
 before, but it doesn't seem that great), but I want a standard place
 that stuff bundled by linux package managers can agree on.
If you use DUB to build, isn't this transparent anyways? -- Bruno Medeiros https://twitter.com/brunodomedeiros
Jun 05 2015
parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 06/05/2015 02:56 PM, Bruno Medeiros wrote:
 On 01/06/2015 04:48, Manu via Digitalmars-d wrote:
 Please declare a standard unix location for D 'includes'. Nobody
 agrees where in the filesystem D files should be.
 I use /usr/include/d2/ for my stuff (I saw it precedented a few times
 before, but it doesn't seem that great), but I want a standard place
 that stuff bundled by linux package managers can agree on.
If you use DUB to build, isn't this transparent anyways?
Yes, but dub's buildsystem isn't always suitable for all projects.
Jun 05 2015
prev sibling parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Mon, 1 Jun 2015 13:48:21 +1000
schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>:

 Please declare a standard unix location for D 'includes'. Nobody
 agrees where in the filesystem D files should be.
 I use /usr/include/d2/ for my stuff [=E2=80=A6].
I would like to mention that 40 out of 62 ppl by the end of 2013 did agreed on /usr/include/dlang : http://forum.dlang.org/thread/20131112205019.12585bbd marco-leise That's a few more than "nobody". Today I would include a "other" choice at least, but it wouldn't have changed the picture much. --=20 Marco
Jun 08 2015
parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 9 June 2015 at 15:52, Marco Leise via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 Am Mon, 1 Jun 2015 13:48:21 +1000
 schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>:

 Please declare a standard unix location for D 'includes'. Nobody
 agrees where in the filesystem D files should be.
 I use /usr/include/d2/ for my stuff […].
I would like to mention that 40 out of 62 ppl by the end of 2013 did agreed on /usr/include/dlang : http://forum.dlang.org/thread/20131112205019.12585bbd marco-leise That's a few more than "nobody". Today I would include a "other" choice at least, but it wouldn't have changed the picture much.
There was no 'd2' option, which is the only one I've ever encountered. Is there a convention in use? Does anyone else bother with this? I just want someone to decide and publish it somewhere... I don't care what's chosen, but we really need to just agree on something.
Jun 09 2015
parent reply "Dicebot" <public dicebot.lv> writes:
I don't see how it matters as long as packaging is done properly. 
Developer should not ever be allowed to write to /usr anyway.
Jun 09 2015
parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Tuesday, 9 June 2015 at 14:09:03 UTC, Dicebot wrote:
 I don't see how it matters as long as packaging is done 
 properly. Developer should not ever be allowed to write to /usr 
 anyway.
+1 I don't expect to be able to rely on the location of anything under /usr except maybe /usr/bin/env and I don't see why D should be any different.
Jun 09 2015
prev sibling next sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 05/31/2015 07:01 PM, Andrei Alexandrescu wrote:
 Let's make this part of 2.068:

 https://issues.dlang.org/show_bug.cgi?id=14636

 It's preapproved. Who would want to work on it?
IIRC, I think Sonke wanted to hold off on that until a dub 1.0. I'm not certain exactly what work he had in mind for that though, versus post-1.0.
May 31 2015
next sibling parent "ponce" <contact gam3sfrommars.fr> writes:
On Monday, 1 June 2015 at 06:35:38 UTC, Nick Sabalausky wrote:
 On 05/31/2015 07:01 PM, Andrei Alexandrescu wrote:
 Let's make this part of 2.068:

 https://issues.dlang.org/show_bug.cgi?id=14636

 It's preapproved. Who would want to work on it?
IIRC, I think Sonke wanted to hold off on that until a dub 1.0. I'm not certain exactly what work he had in mind for that though, versus post-1.0.
DUB has worked equally well and almost unchanged for me in 2015.
May 31 2015
prev sibling parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com> writes:
Am 01.06.2015 um 08:35 schrieb Nick Sabalausky:
 On 05/31/2015 07:01 PM, Andrei Alexandrescu wrote:
 Let's make this part of 2.068:

 https://issues.dlang.org/show_bug.cgi?id=14636

 It's preapproved. Who would want to work on it?
IIRC, I think Sonke wanted to hold off on that until a dub 1.0. I'm not certain exactly what work he had in mind for that though, versus post-1.0.
We have collected some things that would be good to have done. There are no big issues left, though, except for making the API stable and supporting the improved build description language. This should be done within another compiler release cycle.
Jun 04 2015
parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 06/04/2015 04:43 AM, Sönke Ludwig wrote:
 Am 01.06.2015 um 08:35 schrieb Nick Sabalausky:
 On 05/31/2015 07:01 PM, Andrei Alexandrescu wrote:
 Let's make this part of 2.068:

 https://issues.dlang.org/show_bug.cgi?id=14636

 It's preapproved. Who would want to work on it?
IIRC, I think Sonke wanted to hold off on that until a dub 1.0. I'm not certain exactly what work he had in mind for that though, versus post-1.0.
We have collected some things that would be good to have done. There are no big issues left, though,
Cool.
 except for making the API stable and
 supporting the improved build description language.
Is there anything you need from me on the latter? (If a particular matter quiets down or I get distracted, I'm liable to forget about it.)
Jun 04 2015
parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com> writes:
Am 04.06.2015 um 17:08 schrieb Nick Sabalausky:
 On 06/04/2015 04:43 AM, Sönke Ludwig wrote:
(...)
 except for making the API stable and
 supporting the improved build description language.
Is there anything you need from me on the latter? (If a particular matter quiets down or I get distracted, I'm liable to forget about it.)
I think Dicebot did some work recently (or intended to do?). It's actually not that much work overall. I could also pick up what's there and finalize things over the next days.
Jun 04 2015
parent reply "Dicebot" <public dicebot.lv> writes:
On Thursday, 4 June 2015 at 15:39:02 UTC, Sönke Ludwig wrote:
 Am 04.06.2015 um 17:08 schrieb Nick Sabalausky:
 On 06/04/2015 04:43 AM, Sönke Ludwig wrote:
(...)
 except for making the API stable and
 supporting the improved build description language.
Is there anything you need from me on the latter? (If a particular matter quiets down or I get distracted, I'm liable to forget about it.)
I think Dicebot did some work recently (or intended to do?). It's actually not that much work overall. I could also pick up what's there and finalize things over the next days.
I got it to the point where I could build simple hello-world using dub.sdl description but haven't worked on it since then. This issue is neither interesting nor important to me, thus finding motivation is rather hard. Dump of last sources can be found here : https://github.com/Dicebot/dub/tree/sdl
Jun 04 2015
parent =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com> writes:
Am 04.06.2015 um 19:55 schrieb Dicebot:
 I got it to the point where I could build simple hello-world using
 dub.sdl description but haven't worked on it since then. This issue is
 neither interesting nor important to me, thus finding motivation is
 rather hard. Dump of last sources can be found here :
 https://github.com/Dicebot/dub/tree/sdl
I'll pick it up from there in a few days.
Jun 04 2015
prev sibling next sibling parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
Since this thread seems to have turned into a wishlist for dub 
features, I'm going to add one:

Undeprecate dependencies on git branches. They are really not 
different from dependencies with fuzzy "~>" versions. For both, 
the exact selected version can be stored in dub.selections.json.

Use case:
Making a fix to a third-party project and sharing that fix with 
other team members. Having to create a temporary project on 
code.dlang.org just for that purpose is both a waste of time for 
developers and a waste of resources of community infrastructure.
Jun 01 2015
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2015-06-01 17:16, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>" 
wrote:
 Since this thread seems to have turned into a wishlist for dub features,
 I'm going to add one:

 Undeprecate dependencies on git branches. They are really not different
 from dependencies with fuzzy "~>" versions. For both, the exact selected
 version can be stored in dub.selections.json.
That's a good point, as long as people understand how to use dub.selections.json [1]. But I would rather not allow branches in the registry. I would prefer that a git URL could be added directly to dub.json instead. [1] https://github.com/github/gitignore/pull/1444 -- /Jacob Carlborg
Jun 02 2015
prev sibling parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com> writes:
Am 01.06.2015 um 17:16 schrieb "Marc =?UTF-8?B?U2Now7x0eiI=?= 
<schuetzm gmx.net>":
 Since this thread seems to have turned into a wishlist for dub features,
 I'm going to add one:

 Undeprecate dependencies on git branches. They are really not different
 from dependencies with fuzzy "~>" versions. For both, the exact selected
 version can be stored in dub.selections.json.

 Use case:
 Making a fix to a third-party project and sharing that fix with other
 team members. Having to create a temporary project on code.dlang.org
 just for that purpose is both a waste of time for developers and a waste
 of resources of community infrastructure.
They are only deprecated within dub.json. You can still use them within dub.selections.json. But really, branch based dependencies are simply a broken feature. They are completely different to version ranges, because they are not at all comparable (to other branches or versions) - at least without completely committing to GIT and starting to walk the GIT commit graph. You can also check out certain branches locally and use "dub add-local" or "dub add-path" to let DUB use it instead of the publicly registered one.
Jun 04 2015
parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Thursday, 4 June 2015 at 08:51:02 UTC, Sönke Ludwig wrote:
 Am 01.06.2015 um 17:16 schrieb "Marc =?UTF-8?B?U2Now7x0eiI=?= 
 <schuetzm gmx.net>":
 Since this thread seems to have turned into a wishlist for dub 
 features,
 I'm going to add one:

 Undeprecate dependencies on git branches. They are really not 
 different
 from dependencies with fuzzy "~>" versions. For both, the 
 exact selected
 version can be stored in dub.selections.json.

 Use case:
 Making a fix to a third-party project and sharing that fix 
 with other
 team members. Having to create a temporary project on 
 code.dlang.org
 just for that purpose is both a waste of time for developers 
 and a waste
 of resources of community infrastructure.
They are only deprecated within dub.json. You can still use them within dub.selections.json.
This is exactly the wrong way round. dub.selections.json must only contain exact versions. That's its purpose after all. See Ruby's bundler as an example: http://bundler.io/v1.9/rationale.html Especially the parts about Gemfile.lock, which is their equivalent of dub.selections.json.
 But really, branch based dependencies are simply a broken 
 feature. They are completely different to version ranges, 
 because they are not at all comparable (to other branches or 
 versions) - at least without completely committing to GIT and 
 starting to walk the GIT commit graph.
Why would you even want to compare them, except for equality? There are only the following cases AFAICS, none of which require an order: 1) Package is not yet in dub.selections.json: => Clone/update repo, checkout the branch, and put the exact commit id into dub.selections.json. 2) Package is already included in dub.selections.json: => All fine, nothing needs to be done. 3) An upgrade of the package has been requested: => Same as 1) There could be situations during dependency resolution where you want to check whether a given commit is part of a branch, but that's a different (and easier) problem.
 You can also check out certain branches locally and use "dub 
 add-local" or "dub add-path" to let DUB use it instead of the 
 publicly registered one.
This is not an option in a larger team. Everyone would have to do that manually, or you'd need to write a custom script to do what is actually the package manager's task.
Jun 04 2015
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2015-06-04 12:30, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>" 
wrote:
 On Thursday, 4 June 2015 at 08:51:02 UTC, Sönke Ludwig wrote:
 Am 01.06.2015 um 17:16 schrieb "Marc =?UTF-8?B?U2Now7x0eiI=?=
 <schuetzm gmx.net>":
 You can also check out certain branches locally and use "dub
 add-local" or "dub add-path" to let DUB use it instead of the publicly
 registered one.
This is not an option in a larger team. Everyone would have to do that manually, or you'd need to write a custom script to do what is actually the package manager's task.
I would prefer to just stick a Git URL + branch in the dub.json file, just as with the Gemfile in Bundler. -- /Jacob Carlborg
Jun 04 2015
prev sibling parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com> writes:
Am 04.06.2015 um 12:30 schrieb "Marc =?UTF-8?B?U2Now7x0eiI=?= 
<schuetzm gmx.net>":
 On Thursday, 4 June 2015 at 08:51:02 UTC, Sönke Ludwig wrote:
 Am 01.06.2015 um 17:16 schrieb "Marc =?UTF-8?B?U2Now7x0eiI=?=
 <schuetzm gmx.net>":
 Since this thread seems to have turned into a wishlist for dub features,
 I'm going to add one:

 Undeprecate dependencies on git branches. They are really not different
 from dependencies with fuzzy "~>" versions. For both, the exact selected
 version can be stored in dub.selections.json.

 Use case:
 Making a fix to a third-party project and sharing that fix with other
 team members. Having to create a temporary project on code.dlang.org
 just for that purpose is both a waste of time for developers and a waste
 of resources of community infrastructure.
They are only deprecated within dub.json. You can still use them within dub.selections.json.
This is exactly the wrong way round. dub.selections.json must only contain exact versions. That's its purpose after all.
A branch *is* an exact version as far as DUB goes (just one that changes over time). But to make use of any commit relationships within a branch, it would have to gain specific knowledge of GIT or other version management systems. Adding support for commit hashes in dub.selections.json is still TBD, but would be another alternative. BTW, my primary concern with all of these things is retaining a healthy public package eco system. I personally suspect that this is much more important for the language as a whole rather than pleasing every single taste. This doesn't mean that the latter isn't a worthwhile goal, but to me the former has always been more important. What is missing is a proposal how to solve both issues at the same time.
 See Ruby's bundler as an example:
 http://bundler.io/v1.9/rationale.html
 Especially the parts about Gemfile.lock, which is their equivalent of
 dub.selections.json.

 But really, branch based dependencies are simply a broken feature.
 They are completely different to version ranges, because they are not
 at all comparable (to other branches or versions) - at least without
 completely committing to GIT and starting to walk the GIT commit graph.
Why would you even want to compare them, except for equality? (...)
For example for conflict resolution. This is the foremost reason why the *have* to be banned from dub.json. You can't really know if a branch matches/is compatible with any particular version (range) or another branch, so you have to either make an arbitrary guess (always prefer the branch; always prefer the version; always prefer "master", but a tagged version otherwise; etc.), or always issue a version conflict error. The practical effect that was already very visible in the existing ecosystem was that packages tended to separate into two clusters. One cluster that used branch dependencies and another that used version dependencies. Both incompatible with each other. It also meant that a lot of people didn't bother to make any version tags at all, which is understandable, but also a very bad influence to the stability of the ecosystem. Semantic versioning, if used correctly, is a very important factor in that regard. The compromise solution to allow branch based dependencies only in the version selection file doesn't really have drawbacks, but dodges all of those issues by moving the decision to the root of the dependency tree.
 You can also check out certain branches locally and use "dub
 add-local" or "dub add-path" to let DUB use it instead of the publicly
 registered one.
This is not an option in a larger team. Everyone would have to do that manually, or you'd need to write a custom script to do what is actually the package manager's task.
Using dub.selections.json should be the right tool then. You could of course also use things like git submodules + path based dependencies in this case, there is no need to force everything into one schema (especially when it doesn't fit well, such as versions and branches).
Jun 04 2015
parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Thursday, 4 June 2015 at 13:47:41 UTC, Sönke Ludwig wrote:
 A branch *is* an exact version as far as DUB goes (just one 
 that changes over time).
Well, that's the problem then.
 But to make use of any commit relationships within a branch, it 
 would have to gain specific knowledge of GIT or other version 
 management systems.

 Adding support for commit hashes in dub.selections.json is 
 still TBD, but would be another alternative.
I'd say it's the only consistent way.
 BTW, my primary concern with all of these things is retaining a 
 healthy public package eco system. I personally suspect that 
 this is much more important for the language as a whole rather 
 than pleasing every single taste. This doesn't mean that the 
 latter isn't a worthwhile goal, but to me the former has always 
 been more important. What is missing is a proposal how to solve 
 both issues at the same time.
I can understand that. But from what I've seen with rubygems, it is not a problem in practice. You're totally right that released packages shouldn't have such dependencies. A warning message in dub or a check on code.dlang.org is totally okay IMO.
 For example for conflict resolution. This is the foremost 
 reason why the *have* to be banned from dub.json. You can't 
 really know if a branch matches/is compatible with any 
 particular version (range) or another branch, so you have to 
 either make an arbitrary guess (always prefer the branch; 
 always prefer the version; always prefer "master", but a tagged 
 version otherwise; etc.), or always issue a version conflict 
 error.
Definitely always prefer the branch. The goal is to let the developer override the decisions made by upstream library authors.
 The practical effect that was already very visible in the 
 existing ecosystem was that packages tended to separate into 
 two clusters. One cluster that used branch dependencies and 
 another that used version dependencies. Both incompatible with 
 each other. It also meant that a lot of people didn't bother to 
 make any version tags at all, which is understandable, but also 
 a very bad influence to the stability of the ecosystem. 
 Semantic versioning, if used correctly, is a very important 
 factor in that regard.
Yes, but that's a social problem. We can nudge people in the right direction by printing a warning message, or even by rejecting it completely on code.dlang.org.
 The compromise solution to allow branch based dependencies only 
 in the version selection file doesn't really have drawbacks, 
 but dodges all of those issues by moving the decision to the 
 root of the dependency tree.
I does have obvious drawbacks: If dub.selections.json contains non-exact versions, different developers can get slightly different dependencies. dub.selections.json must fully determine the versions that are going to be used, otherwise it has failed its purpose.
 You can also check out certain branches locally and use "dub
 add-local" or "dub add-path" to let DUB use it instead of the 
 publicly
 registered one.
This is not an option in a larger team. Everyone would have to do that manually, or you'd need to write a custom script to do what is actually the package manager's task.
Using dub.selections.json should be the right tool then.
As explained above, it isn't.
 You could of course also use things like git submodules + path 
 based dependencies in this case, there is no need to force 
 everything into one schema (especially when it doesn't fit 
 well, such as versions and branches).
Yes, and I'm sure I can come up with even more workarounds ;-)
Jun 05 2015
parent =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com> writes:
Am 05.06.2015 um 11:56 schrieb "Marc =?UTF-8?B?U2Now7x0eiI=?= 
<schuetzm gmx.net>":
 On Thursday, 4 June 2015 at 13:47:41 UTC, Sönke Ludwig wrote:
 A branch *is* an exact version as far as DUB goes (just one that
 changes over time).
Well, that's the problem then.
 But to make use of any commit relationships within a branch, it would
 have to gain specific knowledge of GIT or other version management
 systems.

 Adding support for commit hashes in dub.selections.json is still TBD,
 but would be another alternative.
I'd say it's the only consistent way.
 BTW, my primary concern with all of these things is retaining a
 healthy public package eco system. I personally suspect that this is
 much more important for the language as a whole rather than pleasing
 every single taste. This doesn't mean that the latter isn't a
 worthwhile goal, but to me the former has always been more important.
 What is missing is a proposal how to solve both issues at the same time.
I can understand that. But from what I've seen with rubygems, it is not a problem in practice. You're totally right that released packages shouldn't have such dependencies. A warning message in dub or a check on code.dlang.org is totally okay IMO.
Well, enforcing it on code.dlang.org would indeed be an option. I still think that using the version backed approach below, or VCS facilities is superior in general, though. There is not much objective reason to duplicate that functionality in the package manager AFAICS.
 For example for conflict resolution. This is the foremost reason why
 the *have* to be banned from dub.json. You can't really know if a
 branch matches/is compatible with any particular version (range) or
 another branch, so you have to either make an arbitrary guess (always
 prefer the branch; always prefer the version; always prefer "master",
 but a tagged version otherwise; etc.), or always issue a version
 conflict error.
Definitely always prefer the branch. The goal is to let the developer override the decisions made by upstream library authors.
That's what dub.selections.json is for! It applies only to the top level project, which can make such decisions. On the other hand, if you always prefer the branch during *dependency resolution*, some deeply buried dependency can simply break your build (or any upstream build) because it chose to use an incompatible branch based dependency.
 The practical effect that was already very visible in the existing
 ecosystem was that packages tended to separate into two clusters. One
 cluster that used branch dependencies and another that used version
 dependencies. Both incompatible with each other. It also meant that a
 lot of people didn't bother to make any version tags at all, which is
 understandable, but also a very bad influence to the stability of the
 ecosystem. Semantic versioning, if used correctly, is a very important
 factor in that regard.
Yes, but that's a social problem. We can nudge people in the right direction by printing a warning message, or even by rejecting it completely on code.dlang.org.
 The compromise solution to allow branch based dependencies only in the
 version selection file doesn't really have drawbacks, but dodges all
 of those issues by moving the decision to the root of the dependency
 tree.
I does have obvious drawbacks: If dub.selections.json contains non-exact versions, different developers can get slightly different dependencies. dub.selections.json must fully determine the versions that are going to be used, otherwise it has failed its purpose.
 You can also check out certain branches locally and use "dub
 add-local" or "dub add-path" to let DUB use it instead of the publicly
 registered one.
This is not an option in a larger team. Everyone would have to do that manually, or you'd need to write a custom script to do what is actually the package manager's task.
Using dub.selections.json should be the right tool then.
As explained above, it isn't.
If you want commit granularity then that's a different thing. The intended way to support this is to still use version tags - DUB will then associate a branch with the latest matching version tag, so that normal version based dependencies can be used. If that case applies, dub.selections.json will contain something like {"somedep": "1.0.0+commit.14.f23a56gb"}. Currently this information isn't used to drive GIT to fetch the right tag, but a ticket for that is open and this could be a viable middleground solution.
 You could of course also use things like git submodules + path based
 dependencies in this case, there is no need to force everything into
 one schema (especially when it doesn't fit well, such as versions and
 branches).
Yes, and I'm sure I can come up with even more workarounds ;-)
It's not really a workaround. The package manager has a version based dependency handling logic that has the proper semantics for this kind of task. Trying to squeeze branches and commits into this isn't necessarily a good idea or even necessary at all. The idea here is to let you do such things and not get in the way, but why should the package manager duplicate functionality of the VCS when that can solve the task just fine? It adds additional and redundant complexity for no real benefit.
Jun 10 2015
prev sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 5/31/15 7:01 PM, Andrei Alexandrescu wrote:
 Let's make this part of 2.068:

 https://issues.dlang.org/show_bug.cgi?id=14636

 It's preapproved. Who would want to work on it?
Just a reminder to everyone that at one point, we wanted the compiler to do kind of what 'dub' does -- fetch packages as dependencies for a project. Remember this DIP? http://wiki.dlang.org/DIP11 The problem I have with dub is it makes you as a consumer of libraries live within its realm. You have to make a dub file to use it, and that's not a small burden. That being said, can we get a tool that does what DIP11 asked for by using dub? We do have quite a large repository of code that's dub-enabled, and I have no problem requiring extra steps for *libraries* to make them usable with this system. Basically, a builder that says "hey, if dmd (or whatever) cannot find an import, see if dub can do it, and then use dub to download the package". -Steve
Jun 01 2015
parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 06/01/2015 11:16 AM, Steven Schveighoffer wrote:
 On 5/31/15 7:01 PM, Andrei Alexandrescu wrote:
 Let's make this part of 2.068:

 https://issues.dlang.org/show_bug.cgi?id=14636

 It's preapproved. Who would want to work on it?
Just a reminder to everyone that at one point, we wanted the compiler to do kind of what 'dub' does -- fetch packages as dependencies for a project. Remember this DIP? http://wiki.dlang.org/DIP11
That sort of did what dub does, but badly. Among other things, it directly encouraged developers to hardcode their exact dependencies. It would have been a mess. Dub's package management side already handles that DIP's goals WAY better. The *only* issue now is trying to opt-out of dub's buildsystem side without causing problems (and I absolutely believe that to be easily solvable, see my other posts). There is one part of that old proposal that I have come to like though, and wouldn't mind seeing: There was an idea to have DMD automatically invoke an external tool
 The problem I have with dub is it makes you as a consumer of libraries
 live within its realm. You have to make a dub file to use it, and that's
 not a small burden.
The dub.json file is an absolutely trivial "burden". It's a really, REALLY simple little file. Or at least, it should be... The ONLY real burden comes when trying to teach dub how to build your project using dub's own buildsystem. If your project doesn't do everything "the dub way" then it certainly can be a bother. And I'm pushing very hard to relieve projects of that requirement. But really, the following is NOT a burden unless extreme lazyness is in play: { "name": "my-project", "dependencies": { "libblah": "~>1.7.0", "libfoobar": "~>2.1.0" } } Yes, a very, very, *VERY* tiny "burden". And then to also get dub's buildsystem out of the way: $ mkdir dummy-src-dir # If lib: $ touch dummy-src-dir/dummy.d # Or if app: $ echo "void main() {}" > dummy-src-dir/dummy.d { "name": "my-project", "dependencies": { "libblah": "~>1.7.0", "libfoobar": "~>2.1.0" }, "sourcePaths": "dummy-src-dir/", "pre-build-command-posix": "./buildscript", "pre-build-command-win": "buildscript" } Done. Trivial. No real burden. (Well, aside from JSON's irritating lack of comments or trailing commas.)
Jun 01 2015
next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/1/15 1:36 PM, Nick Sabalausky wrote:
 On 06/01/2015 11:16 AM, Steven Schveighoffer wrote:
 On 5/31/15 7:01 PM, Andrei Alexandrescu wrote:
 Let's make this part of 2.068:

 https://issues.dlang.org/show_bug.cgi?id=14636

 It's preapproved. Who would want to work on it?
Just a reminder to everyone that at one point, we wanted the compiler to do kind of what 'dub' does -- fetch packages as dependencies for a project. Remember this DIP? http://wiki.dlang.org/DIP11
That sort of did what dub does, but badly. Among other things, it directly encouraged developers to hardcode their exact dependencies. It would have been a mess. Dub's package management side already handles that DIP's goals WAY better. The *only* issue now is trying to opt-out of dub's buildsystem side without causing problems (and I absolutely believe that to be easily solvable, see my other posts).
I'm not defending DIP11, just that the fact that it could be done without requiring an extra "dependencies" file. One thing I absolutely LOVE about git, is that when I do git status, if it sees a not-just-cloned repository, it gives me all the commands I would need to run to do all the various things. I rarely have to go to the git manual. I'd love to see something like the following:
 dmd x.d
Error: Your import of some.lib.module didn't work. Running dub --with-some-cool-arg some.lib.module would fetch the correct module from code.dlang.org
 dub --with-some-cool-arg some.lib.module
Dub : got it, fetched.
 dmd x.d
 echo 'yay, it worked! Too bad this is a fantasy :('
 The problem I have with dub is it makes you as a consumer of libraries
 live within its realm. You have to make a dub file to use it, and that's
 not a small burden.
The dub.json file is an absolutely trivial "burden". It's a really, REALLY simple little file. Or at least, it should be...
Sorry, I suppose it's a small burden if you use dub frequently. I don't. Look what happens when I initialize a dub project: { "name": "testdub", "description": "A minimal D application.", "copyright": "Copyright © 2015, steves", "authors": ["steves"], "dependencies": { } } Cool, I can figure out how to massage all the existing strings, except, hm... dependencies. What goes in there? Now, it's time to get out the dub manual. annoying. Where is the comment that says "put your dependencies in here, they should look like this: ..."??? And dub uses versions. How does that work? Again no comment. And where does the e.g. "name" get stuck into the executable? Oh, it doesn't. Why is it generated? I don't freaking know (except that it was the directory I was in when running dub init). I'm sure dub is great for people who want to learn dub. For those who don't, it should work in a minimal way, and ESPECIALLY if we are to include it with the distribution. -Steve
Jun 01 2015
next sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 06/01/2015 02:49 PM, Steven Schveighoffer wrote:
 On 6/1/15 1:36 PM, Nick Sabalausky wrote:
 That sort of did what dub does, but badly. Among other things, it
 directly encouraged developers to hardcode their exact dependencies. It
 would have been a mess. Dub's package management side already handles
 that DIP's goals WAY better. The *only* issue now is trying to opt-out
 of dub's buildsystem side without causing problems (and I absolutely
 believe that to be easily solvable, see my other posts).
I'm not defending DIP11, just that the fact that it could be done without requiring an extra "dependencies" file. One thing I absolutely LOVE about git, is that when I do git status, if it sees a not-just-cloned repository, it gives me all the commands I would need to run to do all the various things. I rarely have to go to the git manual. I'd love to see something like the following: > dmd x.d Error: Your import of some.lib.module didn't work. Running dub --with-some-cool-arg some.lib.module would fetch the correct module from code.dlang.org > dub --with-some-cool-arg some.lib.module Dub : got it, fetched. > dmd x.d > echo 'yay, it worked! Too bad this is a fantasy :('
Yea, true, that would be nice. However, once the dub PRs I'm working on now are finished, then you should be able to do something like this:
 dmd x.d
Error: Your import of some.lib.module didn't work.
 dub fetch libfoobar
 dmd `dub describe libfoobar --dmd-args` x.d
And it should even be possible to add support for this stuff, even without a dub.json for the current project:
 dmd x.d
Error: Your import of some.lib.module didn't work.
 dub find-module some.lib.module
(searches the repo) libfoobar
 dmd `dub use libfoobar [optional-version] --dmd-args` x.d
(Auto-fetches libfoobar if it hasn't already been fetched, and then outputs the necessary CLI args for DMD to use.)
 The problem I have with dub is it makes you as a consumer of libraries
 live within its realm. You have to make a dub file to use it, and that's
 not a small burden.
The dub.json file is an absolutely trivial "burden". It's a really, REALLY simple little file. Or at least, it should be...
Sorry, I suppose it's a small burden if you use dub frequently. I don't. Look what happens when I initialize a dub project: { "name": "testdub", "description": "A minimal D application.", "copyright": "Copyright © 2015, steves", "authors": ["steves"], "dependencies": { } } Cool, I can figure out how to massage all the existing strings, except, hm... dependencies. What goes in there? Now, it's time to get out the dub manual. annoying. Where is the comment that says "put your dependencies in here, they should look like this: ..."??? And dub uses versions. How does that work? Again no comment.
There's really not much to remember. All you really need to remember is this: "dependencies": { "libname": "==1.3.0" } That's really it. There are fancier things you *can* do, but that right there is really all you need, nothing more. Once dub supports a format that can actually freaking handle comments (*grumble grumble* stupid json *grumble grumble*), then dub init's generated file could be way more self-descriptive: "dependencies": { // Uncomment these to use them: // Use an exact version: // "name-of-lib": "==1.3.4", // Restrict to a certain minor version, // equivalent to ">=1.3.4 <1.4.0": // "name-of-lib": "~>1.3.4", }
 And where does the e.g. "name" get stuck into the executable? Oh, it
 doesn't. Why is it generated? I don't freaking know (except that it was
 the directory I was in when running dub init).
Name is just the name of the project. To identify it. Just like a Github project name. Or, pretty much anything called a "name", really. Every project has a name, dub or not. Not really hard to remember. There's no magic going on there. And again, dub needs a format here that actually freaking supports comments.
 I'm sure dub is great for people who want to learn dub. For those who
 don't, it should work in a minimal way, and ESPECIALLY if we are to
 include it with the distribution.
It could be improved in some ways, but I think dub.json is way simpler then non-users really expect it to be. It just needs to communicate the minimal basics a little better, so people don't have to go trying to find <http://code.dlang.org/package-format> and then get intimidated by all the (mostly non-essential) stuff there. And of course, it needs the PRs I'm doing to make it way more friendly to non-dub buildsystems.
Jun 01 2015
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/1/15 3:53 PM, Nick Sabalausky wrote:
 On 06/01/2015 02:49 PM, Steven Schveighoffer wrote:
 On 6/1/15 1:36 PM, Nick Sabalausky wrote:
 That sort of did what dub does, but badly. Among other things, it
 directly encouraged developers to hardcode their exact dependencies. It
 would have been a mess. Dub's package management side already handles
 that DIP's goals WAY better. The *only* issue now is trying to opt-out
 of dub's buildsystem side without causing problems (and I absolutely
 believe that to be easily solvable, see my other posts).
I'm not defending DIP11, just that the fact that it could be done without requiring an extra "dependencies" file. One thing I absolutely LOVE about git, is that when I do git status, if it sees a not-just-cloned repository, it gives me all the commands I would need to run to do all the various things. I rarely have to go to the git manual. I'd love to see something like the following: > dmd x.d Error: Your import of some.lib.module didn't work. Running dub --with-some-cool-arg some.lib.module would fetch the correct module from code.dlang.org > dub --with-some-cool-arg some.lib.module Dub : got it, fetched. > dmd x.d > echo 'yay, it worked! Too bad this is a fantasy :('
Yea, true, that would be nice. However, once the dub PRs I'm working on now are finished, then you should be able to do something like this: > dmd x.d Error: Your import of some.lib.module didn't work. > dub fetch libfoobar > dmd `dub describe libfoobar --dmd-args` x.d And it should even be possible to add support for this stuff, even without a dub.json for the current project: > dmd x.d Error: Your import of some.lib.module didn't work. > dub find-module some.lib.module (searches the repo) libfoobar > dmd `dub use libfoobar [optional-version] --dmd-args` x.d (Auto-fetches libfoobar if it hasn't already been fetched, and then outputs the necessary CLI args for DMD to use.)
I would like to see more automation like this, even if it means there are some generated meta-files. But I'd like it to be dmd and dub working together. For instance, dmd should check to see if dub can has already fetched the dependency and use it (maybe with a switch enabled). Having to do the whole "`dub use blah blah`" on the command line is hacky. A protocol to be used between dmd and dub (or any other package fetcher) would be a good first step.
 And where does the e.g. "name" get stuck into the executable? Oh, it
 doesn't. Why is it generated? I don't freaking know (except that it was
 the directory I was in when running dub init).
Name is just the name of the project. To identify it. Just like a Github project name. Or, pretty much anything called a "name", really. Every project has a name, dub or not. Not really hard to remember. There's no magic going on there.
I think I mistakenly hastily chose "name" as the first thing that stuck out as not being used, but it's actually probably the only one that *is* used (to name the resulting binary), which actually makes sense. Pick any of the other ones :)
 And again, dub needs a format here that actually freaking supports
 comments.
Yes, definitely. Does it have to be straight json? -Steve
Jun 01 2015
parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Mon, 01 Jun 2015 16:05:17 -0400, Steven Schveighoffer wrote:

 A protocol to be used between dmd and dub (or any other package fetcher)
 would be a good first step.
that was the thing i once proposed. see, we have a powerful scripting=20 language inside DMD: D! yet we never used all it's power to do something=20 really exciting -- like, for example, preparing command lines for=20 external package fetching tool and parsing the answers. instead of=20 providing a simple module for that, it all goes to be hardcoded. i extended CTFE engine with simple file i/o functions and "system" call=20 with config-defined executables (something like sudoers list) just to see=20 if it will be usable. and it's surprisingly fun even with all the limits=20 and without hooks for module imports and so on. this system can be extended to allow writing arbitrary subcommands (like=20 git). just import the corresponding subcommand module, if any, and CTFE=20 it's invocation point. bingo! the system that can be extended without=20 recompiling the compiler. and user can add subcommands on per-project=20 base! sadly, this seems to get no "wow!"s. :-(=
Jun 02 2015
next sibling parent reply Rikki Cattermole <alphaglosined gmail.com> writes:
On 2/06/2015 8:54 p.m., ketmar wrote:
 On Mon, 01 Jun 2015 16:05:17 -0400, Steven Schveighoffer wrote:

 A protocol to be used between dmd and dub (or any other package fetcher)
 would be a good first step.
that was the thing i once proposed. see, we have a powerful scripting language inside DMD: D! yet we never used all it's power to do something really exciting -- like, for example, preparing command lines for external package fetching tool and parsing the answers. instead of providing a simple module for that, it all goes to be hardcoded. i extended CTFE engine with simple file i/o functions and "system" call with config-defined executables (something like sudoers list) just to see if it will be usable. and it's surprisingly fun even with all the limits and without hooks for module imports and so on. this system can be extended to allow writing arbitrary subcommands (like git). just import the corresponding subcommand module, if any, and CTFE it's invocation point. bingo! the system that can be extended without recompiling the compiler. and user can add subcommands on per-project base! sadly, this seems to get no "wow!"s. :-(
I say wow. Okay not really I know it is possible and not all the hard. Personally I think it is a rather an awesome possibility. Really what I want is to add the ability to add on the compiler side symbols that can be CTFE'd. Registered at start of runtime. Add on shared library support and wamo. You can add stuff like this pretty arbitrary :)
Jun 02 2015
parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Tue, 02 Jun 2015 21:12:14 +1200, Rikki Cattermole wrote:

 I say wow.
=20
 Okay not really I know it is possible and not all the hard. Personally I
 think it is a rather an awesome possibility.
adding some CTFE functions is not that hard at all. it's not documented,=20 but one can look at "import()" and "pragma()" to see how they're doing=20 their things and do the same. that's what i did, and it took me one day=20 to write a working PoC. i promised to make a series of articles on that for TWiD, but still=20 didn't done that. mea maxima culpa.
 Really what I want is to add the ability to add on the compiler side
 symbols that can be CTFE'd.
 Registered at start of runtime.
i'm afraid i didn't get it. can you provide a sample?
 Add on shared library support and wamo. You can add stuff like this
 pretty arbitrary :)
you mean adding symbols with plugin-like system? i'm not sure that it=20 will be easy. anyway, with well-defined CTFE API to access file system=20 and execute commands (with restrictions, of course, so D programs will=20 not go wild doing "rm -rf ~/" ;-) one can write modules that contains=20 functions that acts like "filters", executing external binaries and=20 massaging input/output. with some hooks added (like "call this CTFE function (if it present) when=20 module importing fails") and ability to add -I/-J pathes in that hooks=20 (restricted to some "library root dir") one can write "CTFE rdmd with=20 automatic package downloading". i think i have to make a draft and PoC for that, so people can try it and=20 see how k00l it is. ;-)=
Jun 02 2015
parent reply Rikki Cattermole <alphaglosined gmail.com> writes:
On 2/06/2015 9:32 p.m., ketmar wrote:
 On Tue, 02 Jun 2015 21:12:14 +1200, Rikki Cattermole wrote:

 I say wow.

 Okay not really I know it is possible and not all the hard. Personally I
 think it is a rather an awesome possibility.
adding some CTFE functions is not that hard at all. it's not documented, but one can look at "import()" and "pragma()" to see how they're doing their things and do the same. that's what i did, and it took me one day to write a working PoC. i promised to make a series of articles on that for TWiD, but still didn't done that. mea maxima culpa.
 Really what I want is to add the ability to add on the compiler side
 symbols that can be CTFE'd.
 Registered at start of runtime.
i'm afraid i didn't get it. can you provide a sample?
 Add on shared library support and wamo. You can add stuff like this
 pretty arbitrary :)
you mean adding symbols with plugin-like system? i'm not sure that it will be easy. anyway, with well-defined CTFE API to access file system and execute commands (with restrictions, of course, so D programs will not go wild doing "rm -rf ~/" ;-) one can write modules that contains functions that acts like "filters", executing external binaries and massaging input/output. with some hooks added (like "call this CTFE function (if it present) when module importing fails") and ability to add -I/-J pathes in that hooks (restricted to some "library root dir") one can write "CTFE rdmd with automatic package downloading". i think i have to make a draft and PoC for that, so people can try it and see how k00l it is. ;-)
Essentially its compiler plugins that can add news types + free-functions as if it was D code provided by source code only also able to modify e.g. AST directly. So while in of itself would not provide new language features, it could add some very nice behavior to existing ones. So what you want with commands ext. would be done in a shared library. It have e.g. a struct added into object.d(i). __CEIFileSystem.delete("./tmp/something"); Of course adding these plugins should be pretty explicit. Package maintainers for example shouldn't auto install any. There should be no way to auto install them. Basically dmd-plugin-dub-bin would be an example package. That would add dub support directly into dmd. ---------- __CEIDub.Dependency("vibe-d", ">=0.7.22") void main() { // what have you } ---------- Or: ---------- static assert(__CEIDub.dependency("vibe.d", ">=0.7.22"), "Requires vibe-d 0.7.22 or newer."); void main() { // what have you } ---------- That way it can be used for e.g. static-if and template if. Of course I'm sure Walter would go nuts at such an idea, if it was seriously proposed. So perhaps dmd doesn't support it. Instead a new version of dmd is also shipped (dmdext) with this enabled ;) With the community agreeing that we should try to keep as close to dmd as possible and only reverting to dmdext + plugins to help newbies and where it just wouldn't be possible to not have an external tool. Which most likely uses multiple compilation or something else not so nice.
Jun 02 2015
parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Tue, 02 Jun 2015 21:51:54 +1200, Rikki Cattermole wrote:

 Essentially its compiler plugins that can add news types +
 free-functions as if it was D code provided by source code only also
 able to modify e.g. AST directly.
 So while in of itself would not provide new language features, it could
 add some very nice behavior to existing ones.
=20
 So what you want with commands ext. would be done in a shared library.
 It have e.g. a struct added into object.d(i).
=20
 __CEIFileSystem.delete("./tmp/something");
=20
 Of course adding these plugins should be pretty explicit. Package
 maintainers for example shouldn't auto install any. There should be no
 way to auto install them.
=20
 Basically dmd-plugin-dub-bin would be an example package. That would add
 dub support directly into dmd.
=20
 ----------
  __CEIDub.Dependency("vibe-d", ">=3D0.7.22")
 void main() {
 	// what have you
 }
 ----------
=20
 Or:
=20
 ----------
 static assert(__CEIDub.dependency("vibe.d", ">=3D0.7.22"), "Requires
 vibe-d 0.7.22 or newer.");
=20
 void main() {
 	// what have you
 }
 ----------
=20
 That way it can be used for e.g. static-if and template if.
=20
 Of course I'm sure Walter would go nuts at such an idea, if it was
 seriously proposed.
 So perhaps dmd doesn't support it. Instead a new version of dmd is also
 shipped (dmdext) with this enabled ;)
=20
 With the community agreeing that we should try to keep as close to dmd
 as possible and only reverting to dmdext + plugins to help newbies and
 where it just wouldn't be possible to not have an external tool. Which
 most likely uses multiple compilation or something else not so nice.
actually, that should be doable with DDMD on non-windows systems*, i=20 believe. DDMD -- to avoid writing plugins in other languages. ;-) and=20 with DDMD plugins will have full access to frontend internals, including=20 AST manipulation. * non-windows, as dll support on windows is still not here, i believe. the downside of this approach is that it's highly compiler-specific. i.e.=20 one will force to have different plugins for GDC/LDC/SDC/etc. ;-) and=20 with pure CTFE API without plugins, only with external executables, it=20 will be compiler-agnostic, any compiler using DMDFE will get it for free.=20 only SDC will be forced to rewrite the CTFE part, but other than that all=20 CTFE D code will work the same. i mean, 'cmon, we have such great built-in scripting engine, let's use it! your samples can be built ontop of my "subcommand" proposal, for example.=20 let's say that compiler will try to automatically do static import=20 "DMD.subcommands.dub" when programmer writing something like `DMD.dub.Dependency("vibe-d", ">=3D0.7.22");`, falling back to "DMD.failure (name, args)" if there is no "dub" subcommand package. and then subcommand can execute dub, parse it's output and do what it=20 wants. or one can make a local override for subcommand -- without even=20 compiling separate plugin. that's how i see it. =
Jun 02 2015
parent reply Rikki Cattermole <alphaglosined gmail.com> writes:
On 2/06/2015 10:15 p.m., ketmar wrote:
 On Tue, 02 Jun 2015 21:51:54 +1200, Rikki Cattermole wrote:

 Essentially its compiler plugins that can add news types +
 free-functions as if it was D code provided by source code only also
 able to modify e.g. AST directly.
 So while in of itself would not provide new language features, it could
 add some very nice behavior to existing ones.

 So what you want with commands ext. would be done in a shared library.
 It have e.g. a struct added into object.d(i).

 __CEIFileSystem.delete("./tmp/something");

 Of course adding these plugins should be pretty explicit. Package
 maintainers for example shouldn't auto install any. There should be no
 way to auto install them.

 Basically dmd-plugin-dub-bin would be an example package. That would add
 dub support directly into dmd.

 ----------
  __CEIDub.Dependency("vibe-d", ">=0.7.22")
 void main() {
 	// what have you
 }
 ----------

 Or:

 ----------
 static assert(__CEIDub.dependency("vibe.d", ">=0.7.22"), "Requires
 vibe-d 0.7.22 or newer.");

 void main() {
 	// what have you
 }
 ----------

 That way it can be used for e.g. static-if and template if.

 Of course I'm sure Walter would go nuts at such an idea, if it was
 seriously proposed.
 So perhaps dmd doesn't support it. Instead a new version of dmd is also
 shipped (dmdext) with this enabled ;)

 With the community agreeing that we should try to keep as close to dmd
 as possible and only reverting to dmdext + plugins to help newbies and
 where it just wouldn't be possible to not have an external tool. Which
 most likely uses multiple compilation or something else not so nice.
actually, that should be doable with DDMD on non-windows systems*, i believe. DDMD -- to avoid writing plugins in other languages. ;-) and with DDMD plugins will have full access to frontend internals, including AST manipulation. * non-windows, as dll support on windows is still not here, i believe.
I've been able to do it a little bit already with my Web server. Haven't hit its limit so far. In other words as long as exceptions aren't used and its never unloaded, should be ok. Anyway, do you really think we would get that sort of support for a couple of releases?
 the downside of this approach is that it's highly compiler-specific. i.e.
 one will force to have different plugins for GDC/LDC/SDC/etc. ;-) and
 with pure CTFE API without plugins, only with external executables, it
 will be compiler-agnostic, any compiler using DMDFE will get it for free.
 only SDC will be forced to rewrite the CTFE part, but other than that all
 CTFE D code will work the same.
Yeah it is, but it would also mean better customization of the build process!
 i mean, 'cmon, we have such great built-in scripting engine, let's use it!
Indeed lets. But can we also not go modifying the language? I'm just suggesting injecting of symbols that execute compiler side actions. Your suggesting language changes. Even if it is a new pragma.
 your samples can be built ontop of my "subcommand" proposal, for example.
 let's say that compiler will try to automatically do static import
 "DMD.subcommands.dub" when programmer writing something like
 `DMD.dub.Dependency("vibe-d", ">=0.7.22");`, falling back to "DMD.failure
 (name, args)" if there is no "dub" subcommand package.

 and then subcommand can execute dub, parse it's output and do what it
 wants. or one can make a local override for subcommand -- without even
 compiling separate plugin.

 that's how i see it.
We really need to toy with these ideas properly and implement each. Then its just a matter of time to convince the higher ups that it should be merged.
Jun 02 2015
parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Tue, 02 Jun 2015 22:30:47 +1200, Rikki Cattermole wrote:

 * non-windows, as dll support on windows is still not here, i believe.
=20 I've been able to do it a little bit already with my Web server. Haven't hit its limit so far. In other words as long as exceptions aren't used and its never unloaded, should be ok. Anyway, do you really think we would get that sort of support for a couple of releases?
it seems that nobody on windows really needs that. ;-)
 Indeed lets. But can we also not go modifying the language? I'm just
 suggesting injecting of symbols that execute compiler side actions. Your
 suggesting language changes. Even if it is a new pragma.
actually, no language changes at all. CTFE is already here, and it is=20 used in semantic analysis stage. there are no changes to the language,=20 besides adding some hooks and defining an API. it's invisible from the=20 user's POV and doesn't require changing of specs. now CTFE is started=20 explicitly, by using mixins and template evaluation. with my idea it will=20 be started implicitly when some condition is met. sure, that conditions=20 must be documented, but it will not change the language drastically.
 We really need to toy with these ideas properly and implement each. Then
 its just a matter of time to convince the higher ups that it should be
 merged.
yes, i believe that this is the best approach. implement both and let the=20 best solution win in a honest competition. ;-) i don't feel very creative right now, but implementing that idea is in my=20 TODO list, and it's not at the bottom of list. so eventually (month? two?=20 ten years?) i'll write a PoC.=
Jun 02 2015
parent reply Rikki Cattermole <alphaglosined gmail.com> writes:
On 3/06/2015 1:33 a.m., ketmar wrote:
 On Tue, 02 Jun 2015 22:30:47 +1200, Rikki Cattermole wrote:

 * non-windows, as dll support on windows is still not here, i believe.
I've been able to do it a little bit already with my Web server. Haven't hit its limit so far. In other words as long as exceptions aren't used and its never unloaded, should be ok. Anyway, do you really think we would get that sort of support for a couple of releases?
it seems that nobody on windows really needs that. ;-)
 Indeed lets. But can we also not go modifying the language? I'm just
 suggesting injecting of symbols that execute compiler side actions. Your
 suggesting language changes. Even if it is a new pragma.
actually, no language changes at all. CTFE is already here, and it is used in semantic analysis stage. there are no changes to the language, besides adding some hooks and defining an API. it's invisible from the user's POV and doesn't require changing of specs. now CTFE is started explicitly, by using mixins and template evaluation. with my idea it will be started implicitly when some condition is met. sure, that conditions must be documented, but it will not change the language drastically.
Nope, it can be just another function call. ------ auto something(string text); something("...") void another() { } ------ Where something is specified on the compiler side. UDA's + static assert imply compile time literals, which is what we want. Something that is important to note is, these symbols like something defined by the compiler should never hit the backend. If they are referenced they should be forced to be called.
 We really need to toy with these ideas properly and implement each. Then
 its just a matter of time to convince the higher ups that it should be
 merged.
yes, i believe that this is the best approach. implement both and let the best solution win in a honest competition. ;-) i don't feel very creative right now, but implementing that idea is in my TODO list, and it's not at the bottom of list. so eventually (month? two? ten years?) i'll write a PoC.
I think we want something very similar. We just haven't quite merged our ideas just yet.
Jun 02 2015
parent ketmar <ketmar ketmar.no-ip.org> writes:
On Wed, 03 Jun 2015 01:48:13 +1200, Rikki Cattermole wrote:

 Nope, it can be just another function call.
=20
 ------
 auto something(string text);
=20
  something("...")
 void another() {
=20
 }
 ------
=20
 Where something is specified on the compiler side.
 UDA's + static assert imply compile time literals, which is what we
 want.
for me UDA syntax that actually calling some function seems=20 counterintuitive. like "well, i know that it's a simple declaration...=20 WUT?! how it managed to execute the code?!"
 I think we want something very similar. We just haven't quite merged our
 ideas just yet.
yes, seems that it's the same goal, but slightly different approaches. i=20 think i should do a draft for API and write some explanations first, so=20 it can be discussed in details. 'cause now i have a foggy picture in my=20 head which is hard to describe.=
Jun 02 2015
prev sibling next sibling parent reply "Atila Neves" <atila.neves gmail.com> writes:
On Tuesday, 2 June 2015 at 08:54:46 UTC, ketmar wrote:
 On Mon, 01 Jun 2015 16:05:17 -0400, Steven Schveighoffer wrote:

 A protocol to be used between dmd and dub (or any other 
 package fetcher)
 would be a good first step.
that was the thing i once proposed. see, we have a powerful scripting language inside DMD: D! yet we never used all it's power to do something really exciting -- like, for example, preparing command lines for external package fetching tool and parsing the answers. instead of providing a simple module for that, it all goes to be hardcoded.
https://github.com/atilaneves/reggae Atila
Jun 02 2015
next sibling parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Tue, 02 Jun 2015 09:15:12 +0000, Atila Neves wrote:

 On Tuesday, 2 June 2015 at 08:54:46 UTC, ketmar wrote:
 On Mon, 01 Jun 2015 16:05:17 -0400, Steven Schveighoffer wrote:

 A protocol to be used between dmd and dub (or any other package
 fetcher)
 would be a good first step.
that was the thing i once proposed. see, we have a powerful scripting language inside DMD: D! yet we never used all it's power to do something really exciting -- like, for example, preparing command lines for external package fetching tool and parsing the answers. instead of providing a simple module for that, it all goes to be hardcoded.
=20 https://github.com/atilaneves/reggae =20 Atila
hm. looks interesting, albeit a little too noisy for my taste.=
Jun 02 2015
parent reply "Atila Neves" <atila.neves gmail.com> writes:
On Tuesday, 2 June 2015 at 09:33:24 UTC, ketmar wrote:
 On Tue, 02 Jun 2015 09:15:12 +0000, Atila Neves wrote:

 On Tuesday, 2 June 2015 at 08:54:46 UTC, ketmar wrote:
 On Mon, 01 Jun 2015 16:05:17 -0400, Steven Schveighoffer 
 wrote:

 A protocol to be used between dmd and dub (or any other 
 package
 fetcher)
 would be a good first step.
that was the thing i once proposed. see, we have a powerful scripting language inside DMD: D! yet we never used all it's power to do something really exciting -- like, for example, preparing command lines for external package fetching tool and parsing the answers. instead of providing a simple module for that, it all goes to be hardcoded.
https://github.com/atilaneves/reggae Atila
hm. looks interesting, albeit a little too noisy for my taste.
It depends on what you want to do: the idea is to use high-level utility functions as much as possible, while at the same time offering low-level primitives that those high-level ones are based on. If it can be made simpler, I'd really like to know. Atila
Jun 02 2015
parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Tue, 02 Jun 2015 12:12:06 +0000, Atila Neves wrote:

 https://github.com/atilaneves/reggae
=20
 Atila
hm. looks interesting, albeit a little too noisy for my taste.
=20 It depends on what you want to do: the idea is to use high-level utility functions as much as possible, while at the same time offering low-level primitives that those high-level ones are based on. =20 If it can be made simpler, I'd really like to know.
not that i have something better to offer. it's readable and looks fine.=20 besides, i'm really sux at design things. ;-)=
Jun 02 2015
parent "Atila Neves" <atila.neves gmail.com> writes:
Well, there's this thread now:

http://forum.dlang.org/thread/ranqlmrjornlvopsuris forum.dlang.org#post-ranqlmrjornlvopsuris:40forum.dlang.org

Atila

On Tuesday, 2 June 2015 at 13:22:25 UTC, ketmar wrote:
 On Tue, 02 Jun 2015 12:12:06 +0000, Atila Neves wrote:

 https://github.com/atilaneves/reggae
 
 Atila
hm. looks interesting, albeit a little too noisy for my taste.
It depends on what you want to do: the idea is to use high-level utility functions as much as possible, while at the same time offering low-level primitives that those high-level ones are based on. If it can be made simpler, I'd really like to know.
not that i have something better to offer. it's readable and looks fine. besides, i'm really sux at design things. ;-)
Jun 02 2015
prev sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Tuesday, 2 June 2015 at 09:15:13 UTC, Atila Neves wrote:
 https://github.com/atilaneves/reggae

 Atila
If this works up to the expectation, I will only need to write a simple code.dlang.org package fetcher and will never need to use dub again :)
Jun 02 2015
parent "Atila Neves" <atila.neves gmail.com> writes:
On Tuesday, 2 June 2015 at 18:56:22 UTC, Dicebot wrote:
 On Tuesday, 2 June 2015 at 09:15:13 UTC, Atila Neves wrote:
 https://github.com/atilaneves/reggae

 Atila
If this works up to the expectation, I will only need to write a simple code.dlang.org package fetcher and will never need to use dub again :)
Destroy away! If it doesn't work like you like, let me know. Atila
Jun 02 2015
prev sibling next sibling parent reply "Wyatt" <wyatt.epp gmail.com> writes:
On Tuesday, 2 June 2015 at 08:54:46 UTC, ketmar wrote:
 sadly, this seems to get no "wow!"s. :-(
If no one knows about it, "wow!"s will not happen. Make PRs. -Wyatt
Jun 02 2015
parent ketmar <ketmar ketmar.no-ip.org> writes:
On Tue, 02 Jun 2015 13:44:10 +0000, Wyatt wrote:

 On Tuesday, 2 June 2015 at 08:54:46 UTC, ketmar wrote:
 sadly, this seems to get no "wow!"s. :-(
=20 If no one knows about it, "wow!"s will not happen. Make PRs.
i mentioned the idea several times in NG, it fades almost without any=20 attention. honestly, i have alot of hobby projects, so before writing=20 such complex things (and it's fairly complex, considering design and=20 documentation) i want to know if there will be enough attention to it. as people seems to pass it by, i have no motivation of doing that work. i=20 have alot of things i want to add to Aliced, so no "wow, that would be=20 nice to try!" replies means "move that down in my TODO list".=
Jun 02 2015
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-06-02 10:54, ketmar wrote:

 that was the thing i once proposed. see, we have a powerful scripting
 language inside DMD: D! yet we never used all it's power to do something
 really exciting -- like, for example, preparing command lines for
 external package fetching tool and parsing the answers. instead of
 providing a simple module for that, it all goes to be hardcoded.

 i extended CTFE engine with simple file i/o functions and "system" call
 with config-defined executables (something like sudoers list) just to see
 if it will be usable. and it's surprisingly fun even with all the limits
 and without hooks for module imports and so on.

 this system can be extended to allow writing arbitrary subcommands (like
 git). just import the corresponding subcommand module, if any, and CTFE
 it's invocation point. bingo! the system that can be extended without
 recompiling the compiler. and user can add subcommands on per-project
 base!

 sadly, this seems to get no "wow!"s. :-(
I think it's pretty cool. But I like the approach SDC has taken even more. Using the LLVM JIT to run CTFE code. -- /Jacob Carlborg
Jun 03 2015
parent ketmar <ketmar ketmar.no-ip.org> writes:
On Wed, 03 Jun 2015 13:45:06 +0200, Jacob Carlborg wrote:

 On 2015-06-02 10:54, ketmar wrote:
=20
 that was the thing i once proposed. see, we have a powerful scripting
 language inside DMD: D! yet we never used all it's power to do
 something really exciting -- like, for example, preparing command lines
 for external package fetching tool and parsing the answers. instead of
 providing a simple module for that, it all goes to be hardcoded.

 i extended CTFE engine with simple file i/o functions and "system" call
 with config-defined executables (something like sudoers list) just to
 see if it will be usable. and it's surprisingly fun even with all the
 limits and without hooks for module imports and so on.

 this system can be extended to allow writing arbitrary subcommands
 (like git). just import the corresponding subcommand module, if any,
 and CTFE it's invocation point. bingo! the system that can be extended
 without recompiling the compiler. and user can add subcommands on
 per-project base!

 sadly, this seems to get no "wow!"s. :-(
=20 I think it's pretty cool. But I like the approach SDC has taken even more. Using the LLVM JIT to run CTFE code.
so SDC will benefit from JITting, as it's not required to modify CTFE=20 evaluator in any way to implement my idea. just adding some more=20 "internal functions" (like `import()`), which are nothing more than "call=20 `interpret`, use results). porting that entry points to SDC will be=20 trivial.=
Jun 03 2015
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-06-01 20:49, Steven Schveighoffer wrote:

 I'm not defending DIP11, just that the fact that it could be done
 without requiring an extra "dependencies" file.
One usually needs a file to set all compiler and linker flags and other kinds of configurations. Dub allows you to do this as well in cross-platform way. So when you already have all that, just put the dependencies there and be done with it. -- /Jacob Carlborg
Jun 02 2015
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/2/15 3:12 AM, Jacob Carlborg wrote:
 On 2015-06-01 20:49, Steven Schveighoffer wrote:

 I'm not defending DIP11, just that the fact that it could be done
 without requiring an extra "dependencies" file.
One usually needs a file to set all compiler and linker flags and other kinds of configurations. Dub allows you to do this as well in cross-platform way. So when you already have all that, just put the dependencies there and be done with it.
Actually, I don't. dmd *.d generally works fine. -Steve
Jun 02 2015
parent Jacob Carlborg <doob me.com> writes:
On 2015-06-02 16:52, Steven Schveighoffer wrote:

 Actually, I don't. dmd *.d generally works fine.
I if it works for you, great. It doesn't work for me. And that shell globbing doesn't work on Windows. -- /Jacob Carlborg
Jun 03 2015
prev sibling parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com> writes:
Am 01.06.2015 um 20:49 schrieb Steven Schveighoffer:
 (...)
 Sorry, I suppose it's a small burden if you use dub frequently. I don't.
 Look what happens when I initialize a dub project:

 {
      "name": "testdub",
      "description": "A minimal D application.",
      "copyright": "Copyright © 2015, steves",
      "authors": ["steves"],
      "dependencies": {
      }
 }

 Cool, I can figure out how to massage all the existing strings, except,
 hm... dependencies. What goes in there?
As of recently, you can also directly specify dependencies to the "dub init" command, for example "dub init myproject tango derelict-gl". There was also the suggestion to make dub init (without arguments) interactive, which would make this a really easy matter.
Jun 04 2015
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/4/15 4:58 AM, Sönke Ludwig wrote:
 Am 01.06.2015 um 20:49 schrieb Steven Schveighoffer:
 (...)
 Sorry, I suppose it's a small burden if you use dub frequently. I don't.
 Look what happens when I initialize a dub project:

 {
      "name": "testdub",
      "description": "A minimal D application.",
      "copyright": "Copyright © 2015, steves",
      "authors": ["steves"],
      "dependencies": {
      }
 }

 Cool, I can figure out how to massage all the existing strings, except,
 hm... dependencies. What goes in there?
As of recently, you can also directly specify dependencies to the "dub init" command, for example "dub init myproject tango derelict-gl".
That's nice. But it still isn't self-explanatory. Nor additive.
 There
 was also the suggestion to make dub init (without arguments)
 interactive, which would make this a really easy matter.
This would be better, but I still think a way to add dependencies should be supported on the command line. Something like dub depend someproject. You don't always know what a project will need at the beginning. Of course, you could just say "well, edit the json file!". You really need a format that supports comments... In any case, this doesn't negate the concerns others have raised. I don't mean to bash dub, it's better than nothing. But I haven't used it for any real project yet. And when I did use it, it was not a straightforward experience. -Steve
Jun 04 2015
next sibling parent "wobbles" <grogan.colin gmail.com> writes:
On Thursday, 4 June 2015 at 13:07:50 UTC, Steven Schveighoffer 
wrote:
 That's nice. But it still isn't self-explanatory. Nor additive.
Do you think it'd be better to have a list of flags to specify dependancies? Like : dub init myProj -d tango -d derelict-gl --dependency=vibe-d
 This would be better, but I still think a way to add 
 dependencies should be supported on the command line. Something 
 like dub depend someproject. You don't always know what a 
 project will need at the beginning.
Agree here, this is something I plan on working on soon.
 Of course, you could just say "well, edit the json file!". You 
 really need a format that supports comments...

 In any case, this doesn't negate the concerns others have 
 raised.

 I don't mean to bash dub, it's better than nothing. But I 
 haven't used it for any real project yet. And when I did use 
 it, it was not a straightforward experience.

 -Steve
Jun 04 2015
prev sibling parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com> writes:
Am 04.06.2015 um 15:07 schrieb Steven Schveighoffer:
 On 6/4/15 4:58 AM, Sönke Ludwig wrote:
 As of recently, you can also directly specify dependencies to the "dub
 init" command, for example "dub init myproject tango derelict-gl".
That's nice. But it still isn't self-explanatory. Nor additive.
A command to add/remove dependencies might be useful. But the question is where to stop. Does it make sense to have a command to edit the description? Or add compiler flags? Or manage configurations or sub packages? It also gets a little hairy w.r.t. keeping possible user formatting in the package description file, especially once support for comments gets added.
 There
 was also the suggestion to make dub init (without arguments)
 interactive, which would make this a really easy matter.
This would be better, but I still think a way to add dependencies should be supported on the command line. Something like dub depend someproject. You don't always know what a project will need at the beginning. Of course, you could just say "well, edit the json file!". You really need a format that supports comments...
Agreed, that is really needed. It's almost on top of the TODO list.
 In any case, this doesn't negate the concerns others have raised.

 I don't mean to bash dub, it's better than nothing. But I haven't used
 it for any real project yet. And when I did use it, it was not a
 straightforward experience.
The important part is to not stop with voicing criticism, but to actively help to improve the situation. Opening tickets or "voting" on existing ones, or even making an enhancement proposal or an actual PR would be the most constructive. It's a little sad how some people seem to perceive DUB as being static and unable to evolve. That is not the case. It has started out with a small core functionality to get something that benefits a lot of people right away. But everything has been done in a way that allows to extend or generalize things later. Not everyone will agree with its philosophy to provide high level declarative abstractions instead of a low level system that can be used to create abstractions within the build language, but I'm very positive that this approach is much more useful for the vast majority of users (and especially for newcomers). Retrofitting existing build approaches can of course sometimes come with friction, but even that should be solvable one way or another in most cases. In any case, probably all of the mentioned criticism so far seems to be based on nothing more than missing functionality or bugs and nothing that would go against the existing design. An exception is support for other high level tasks, such as "deployment" targets or similar. This has never been the scope and I don't think we should go that way. On a related note, it's also a pity that Reggae mixes incremental build improvements (from which DUB itself could greatly profit, too - just as a Ninja generator for DUB would be a nice feature) with a separate, layered build description. I mean there is of course no reason to not have alternative approaches for build descriptions available in general, but when mixed with a public package repository, it just leads to fragmentation. Sorry for the slightly OT reply, most of this isn't targeted at you, I just went through the newsgroup posts for the first time after a while and needed to get this out.
Jun 04 2015
next sibling parent reply "Atila Neves" <atila.neves gmail.com> writes:
 On a related note, it's also a pity that Reggae mixes 
 incremental build improvements (from which DUB itself could 
 greatly profit, too - just as a Ninja generator for DUB would 
 be a nice feature) with a separate, layered build description. 
 I mean there is of course no reason to not have alternative 
 approaches for build descriptions available in general, but 
 when mixed with a public package repository, it just leads to 
 fragmentation.
I can add a ninja generator to dub if you want, with the default being per-package compilation (since right now for dub it's one file or everything at once). But I wrote reggae because: 1. Declarative is preferable but imperative is needed. My (our?) favourite imperative language is D, so I want to specify builds in it. 2. I want a build tool that builds on dub but doesn't require it 3. I wanted it to make it easy to link with C and C++ As I've mentioned before, I know the kind of things I'd want to do with the build system if I had a large and complicated enough project, and I know I wouldn't be able to do it easily using dub alone. As I've also mentioned before, building with dub is just fine for most people. I don't know if fragmentation would be an issue. The packages are still dub packages and I for one will use dub.json/sdl to list my dependencies even if reggae is actually generating the build. Atila
Jun 04 2015
parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com> writes:
Am 04.06.2015 um 17:47 schrieb Atila Neves:
 On a related note, it's also a pity that Reggae mixes incremental
 build improvements (from which DUB itself could greatly profit, too -
 just as a Ninja generator for DUB would be a nice feature) with a
 separate, layered build description. I mean there is of course no
 reason to not have alternative approaches for build descriptions
 available in general, but when mixed with a public package repository,
 it just leads to fragmentation.
I can add a ninja generator to dub if you want, with the default being per-package compilation (since right now for dub it's one file or everything at once).
That would be great! If the per-package mode has such practical benefits, we could also simply make it the general default, for all generators that can support it. However, I just briefly looked at your benchmark results, the performance boost seems to be solely because of parallel compiler invocations. We already had issues with per-module parallel compilation and out-of-memory errors, so this optimization seems to be only suitable for certain projects or build machines. So at least with the current compiler memory requirements I'm not sure if this would be a good default build mode.
 But I wrote reggae because:

 1. Declarative is preferable but imperative is needed. My (our?)
 favourite imperative language is D, so I want to specify builds in it.
This is of course perfectly fine and reasonable (although I personally find the D syntax to be a bit too verbose for this task, but thats really just personal taste - oh, and JSON is awful, too ;). The problems just start to creep in as soon as public DUB packages start to depend on Reggae - then we'll possibly get another split in the ecosystem. I'd say we should generally try to focus on a single standard solution, however that looks and however the tool is split into executables or packages.
 2. I want a build tool that builds on dub but doesn't require it
Fair enough.
 3. I wanted it to make it easy to link with C and C++
This really also needs to be solved within DUB (i.e. without resorting to preBuildCommands). That is going to be the next priority after getting the 1.0.0 features ready.
 As I've mentioned before, I know the kind of things I'd want to do with
 the build system if I had a large and complicated enough project, and I
 know I wouldn't be able to do it easily using dub alone. As I've also
 mentioned before, building with dub is just fine for most people.
Getting some of those use cases on a Wiki page or something would be great. There have been some ideas to approach the procedural aspect from a different angle - using procedural DUB plugins that can be invoked from within the declarative build description. This would have the advantage that a lot of information can be gathered about a package without executing procedural code (with the possible accompanying security risks and performance issues or the efforts needed to mitigate those).
 I don't know if fragmentation would be an issue. The packages are still
 dub packages and I for one will use dub.json/sdl to list my dependencies
 even if reggae is actually generating the build.
Yeah that would definitely not be an issue. I just fear (maybe unnecessarily) that people might start to put packages in the registry that can *only* be built using Reggae (or some other build tool). At least for libraries that would really be bad.
Jun 10 2015
next sibling parent reply "Atila Neves" <atila.neves gmail.com> writes:
 That would be great! If the per-package mode has such practical 
 benefits, we could also simply make it the general default, for 
 all generators that can support it. However, I just briefly 
 looked at your benchmark results, the performance boost seems 
 to be solely because of parallel compiler invocations. We 
 already had issues with per-module parallel compilation and 
 out-of-memory errors, so this optimization seems to be only 
 suitable for certain projects or build machines. So at least 
 with the current compiler memory requirements I'm not sure if 
 this would be a good default build mode.
From mine and Andrei's results it seems to be a much better default.
 This is of course perfectly fine and reasonable (although I 
 personally find the D syntax to be a bit too verbose for this 
 task, but thats really just personal taste - oh, and JSON is 
 awful, too ;). The problems just start to creep in as soon as 
 public DUB packages start to depend on Reggae - then we'll 
 possibly get another split in the ecosystem. I'd say we should 
 generally try to focus on a single standard solution, however 
 that looks and however the tool is split into executables or 
 packages.
I'm working on the syntax ;)
 As I've mentioned before, I know the kind of things I'd want 
 to do with
 the build system if I had a large and complicated enough 
 project, and I
 know I wouldn't be able to do it easily using dub alone. As 
 I've also
 mentioned before, building with dub is just fine for most 
 people.
Getting some of those use cases on a Wiki page or something would be great.
I'd have to go back to my old work project and read all the CMake code...
 I don't know if fragmentation would be an issue. The packages 
 are still
 dub packages and I for one will use dub.json/sdl to list my 
 dependencies
 even if reggae is actually generating the build.
Yeah that would definitely not be an issue. I just fear (maybe unnecessarily) that people might start to put packages in the registry that can *only* be built using Reggae (or some other build tool). At least for libraries that would really be bad.
I think that if a project is a dub package, then it needs to be able to be built with dub. If anything on code.dlang.org doesn't (correctly) build with `dub build` then that would be incredibly wrong. Also, I'm not as sure as you that reggae will catch on that much :) Atila
Jun 10 2015
parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 06/10/2015 09:06 AM, Atila Neves wrote:
 I think that if a project is a dub package, then it needs to be able to
 be built with dub. If anything on code.dlang.org doesn't (correctly)
 build with `dub build` then that would be incredibly wrong.
Right. And to summarize what I said elsewhere about that: "Building WITH dub" need NOT imply "building using dub's internal buildsystem". It only needs to mean "The CORRECT buildsystem can be trigged via 'dub build', whatever buildsystem that may be". That's why the current situation is backwards and CAUSES splits rather than preventing them: We can always trigger dub's internal buildsystem via "dub build", but trying to get "dub build" to trigger the CORRECT one INSTEAD of built-in (whenever a different one is needed) is a world of hurt.
Jun 10 2015
prev sibling next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 06/10/2015 08:11 AM, Sönke Ludwig wrote:
 Am 04.06.2015 um 17:47 schrieb Atila Neves:
 I don't know if fragmentation would be an issue. The packages are still
 dub packages and I for one will use dub.json/sdl to list my dependencies
 even if reggae is actually generating the build.
Yeah that would definitely not be an issue. I just fear (maybe unnecessarily) that people might start to put packages in the registry that can *only* be built using Reggae (or some other build tool). At least for libraries that would really be bad.
I've found the opposite to be true. We can *already* get packages in code.dlang.org that work with a custom buildscript, but then fail when doing "dub build". I know this because I've *already* done it by accident multiple times. And then I get bug reports of "X or Y build problem under dub". But if we can easily tell dub via dub.json, "Hey, don't use your internal build system to build this, I've stuck the appropriate command in pre/postBuildCommands", then "dub build" will automatically just simply work. Major bonus compatibility points the more information we can pass from dub to the custom command (ex: "Which 'built type' did the user request with '--build='?" or...well, everything provided by dub describe --data). Remember, libraries do NOT need to use the same build system to be used together. As long as there's a common interface to... 1. Trigger a build, and 2. Get the lists of import paths and output binaries (and maybe any other required flags), ...then everything's golden. This is the job of a package manager. Each library author tells the package manager: 1. "When someone asks you to build me, here's how you do it" and 2. "When someone asks you for my import paths and output binaries (etc.), this is what you tell them...". For DUB, #1 above can be done via pre/postBuildCommands as long as we can tell it "Don't go trying to build this yourself". (I'm wondering if "targetType": "none" would help here.) And for #2, DUB can do that via my --data= pull request. Ok, only one possible issue left: Source libraries (as opposed to binary libraries) *could* be an issue just because of the differences in collecting the list of files to be compiled. But that's solvable two ways: 1. If your lib *requires* something like RDMD-style dependency collecting, then the lib must be a binary lib. Or much, much better yet: 2. Option in dub.json to say "when you collect this package's list of source files, do it RDMD-style". Sure, that may not be appropriate for ALL packages, but that's exactly why it'd be a per-package OPTION. And there it is, the *existing* issues of splitting the dub ecosystem are thus eliminated, without creating new splits.
Jun 10 2015
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2015-06-10 14:11, Sönke Ludwig wrote:

 Getting some of those use cases on a Wiki page or something would be
 great. There have been some ideas to approach the procedural aspect from
 a different angle - using procedural DUB plugins that can be invoked
 from within the declarative build description. This would have the
 advantage that a lot of information can be gathered about a package
 without executing procedural code (with the possible accompanying
 security risks and performance issues or the efforts needed to mitigate
 those).
Orbit, the package manager I was working on, uses Ruby as the description language. It uses the more traditional way of uploading packages to the registry, basically a ZIP with with a special directory layout with every file necessary to build the package. With this model it'a straight forward to "compile" the Ruby code to JSON or YAML when building the package locally (same thing can be done with D). On the server side, the registry only needs to read the JSON file to get the metadata of the package. When installing the package it would use the Ruby file again (or JSON description, never got that far), to be able to run pre and post actions. This part shouldn't be any less secure than the current pre/post commands. -- /Jacob Carlborg
Jun 10 2015
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/4/15 11:07 AM, Sönke Ludwig wrote:
 Am 04.06.2015 um 15:07 schrieb Steven Schveighoffer:
 On 6/4/15 4:58 AM, Sönke Ludwig wrote:
 As of recently, you can also directly specify dependencies to the "dub
 init" command, for example "dub init myproject tango derelict-gl".
That's nice. But it still isn't self-explanatory. Nor additive.
A command to add/remove dependencies might be useful. But the question is where to stop. Does it make sense to have a command to edit the description? Or add compiler flags? Or manage configurations or sub packages? It also gets a little hairy w.r.t. keeping possible user formatting in the package description file, especially once support for comments gets added.
Editing strings that don't affect the build doesn't seem like a necessary thing. An interactive init would probably solve that. However, things like adding or removing a dependency would be very nice without having to know the format of the file. I'm trying to think of things that a simple first-time user would use dub for, aside from adding dependencies. Perhaps creating a debug build. Other than that, I think editing the file for nitty gritty details is probably OK. The use case I'm thinking of is basically: import somepackage.somemodule; ... compile -> can't find somepackage.somemodule. did you mean somepackage.somemodule from libgeneric on code.dlang.org? (url) to add: dub depend add libgeneric dub depend add libgeneric => found it, added it, added dependency I don't have to look up anything on code.dlang.org, I don't have to do anything, dub figures it all out from the description file online. I think of things like linux shell on my linuxmint installation (ubuntu based), which uses some cool things: user> blah No command 'blah' found, did you mean: Command 'blam' from package 'blam' (universe)
 Of course, you could just say "well, edit the json file!". You really
 need a format that supports comments...
Agreed, that is really needed. It's almost on top of the TODO list.
That is good!
 In any case, this doesn't negate the concerns others have raised.

 I don't mean to bash dub, it's better than nothing. But I haven't used
 it for any real project yet. And when I did use it, it was not a
 straightforward experience.
The important part is to not stop with voicing criticism, but to actively help to improve the situation. Opening tickets or "voting" on existing ones, or even making an enhancement proposal or an actual PR would be the most constructive.
Not using dub frequently means I have a very sparse relationship with it. It's kind of that chicken and egg thing. I'm sure if I started having to use dub for a project, I would become more involved :)
 Sorry for the slightly OT reply, most of this isn't targeted at you, I
 just went through the newsgroup posts for the first time after a while
 and needed to get this out.
No problem, please keep up the dialogue! -Steve
Jun 04 2015
parent reply Jacob Carlborg <doob me.com> writes:
On 2015-06-04 19:44, Steven Schveighoffer wrote:

 The use case I'm thinking of is basically:

 import somepackage.somemodule;
 ...

 compile -> can't find somepackage.somemodule.
 did you mean somepackage.somemodule from libgeneric on code.dlang.org?
     (url)
     to add: dub depend add libgeneric

 dub depend add libgeneric => found it, added it, added dependency

 I don't have to look up anything on code.dlang.org, I don't have to do
 anything, dub figures it all out from the description file online.
Sounds like a nice feature but how would that work in practice? I mean, there's nothing that connects a particular D module or package to a particular Dub package. -- /Jacob Carlborg
Jun 05 2015
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/5/15 5:07 AM, Jacob Carlborg wrote:
 On 2015-06-04 19:44, Steven Schveighoffer wrote:

 The use case I'm thinking of is basically:

 import somepackage.somemodule;
 ...

 compile -> can't find somepackage.somemodule.
 did you mean somepackage.somemodule from libgeneric on code.dlang.org?
     (url)
     to add: dub depend add libgeneric

 dub depend add libgeneric => found it, added it, added dependency

 I don't have to look up anything on code.dlang.org, I don't have to do
 anything, dub figures it all out from the description file online.
Sounds like a nice feature but how would that work in practice? I mean, there's nothing that connects a particular D module or package to a particular Dub package.
The source of the package does. code.dlang.org can create this automatically. -Steve
Jun 05 2015
parent reply Jacob Carlborg <doob me.com> writes:
On 2015-06-05 15:17, Steven Schveighoffer wrote:

 The source of the package does. code.dlang.org can create this
 automatically.
I see two problems with this: 1. Does the registry (code.dlang.org) really know which files a Dub package contains? I would guess the registry mostly contains information where to find the source for a package, i.e. GitHub Rubygems, for example, on the other hand would be able to do this because a Ruby gem (package) is ZIP archive with all the sources for the package included. Also, when building a gem the package description is "compiled", that is, all shell globs are expanded to locate all files. 2. It's possible to have a file with a completely different module name in D -- /Jacob Carlborg
Jun 06 2015
parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 06/06/2015 09:16 AM, Jacob Carlborg wrote:
 On 2015-06-05 15:17, Steven Schveighoffer wrote:

 The source of the package does. code.dlang.org can create this
 automatically.
I see two problems with this: 1. Does the registry (code.dlang.org) really know which files a Dub package contains? I would guess the registry mostly contains information where to find the source for a package, i.e. GitHub Rubygems, for example, on the other hand would be able to do this because a Ruby gem (package) is ZIP archive with all the sources for the package included. Also, when building a gem the package description is "compiled", that is, all shell globs are expanded to locate all files. 2. It's possible to have a file with a completely different module name in D
Even if that stuff isn't currently known to code.dlang.org, it'd all be easy for it to obtain: $ mkdir some_temp && cd some_temp $ git clone package_URL dir && cd dir $ dmd (flags dub already knows) -v | grep import Would only need to do that when it detects a new version tag (which it already detects).
Jun 07 2015
parent reply Jacob Carlborg <doob me.com> writes:
On 2015-06-07 17:27, Nick Sabalausky wrote:

 Even if that stuff isn't currently known to code.dlang.org, it'd all be
 easy for it to obtain:

 $ mkdir some_temp && cd some_temp
 $ git clone package_URL dir && cd dir
 $ dmd (flags dub already knows) -v | grep import

 Would only need to do that when it detects a new version tag (which it
 already detects).
So the registry basically needs to compile all packages? Isn't it better to "compile" and upload packages then? -- /Jacob Carlborg
Jun 07 2015
parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com> writes:
Am 07.06.2015 um 19:57 schrieb Jacob Carlborg:
 On 2015-06-07 17:27, Nick Sabalausky wrote:

 Even if that stuff isn't currently known to code.dlang.org, it'd all be
 easy for it to obtain:

 $ mkdir some_temp && cd some_temp
 $ git clone package_URL dir && cd dir
 $ dmd (flags dub already knows) -v | grep import

 Would only need to do that when it detects a new version tag (which it
 already detects).
So the registry basically needs to compile all packages? Isn't it better to "compile" and upload packages then?
Isn't just the list of *modules* needed and not the list of *imports*? The registry can easily query the list of files of a package using the GitHub API. The package description is obviously already known, too, so this would definitely be possible to do without compiling anything.
Jun 07 2015
parent reply Jacob Carlborg <doob me.com> writes:
On 2015-06-08 08:04, Sönke Ludwig wrote:

 Isn't just the list of *modules* needed and not the list of *imports*?
 The registry can easily query the list of files of a package using the
 GitHub API. The package description is obviously already known, too, so
 this would definitely be possible to do without compiling anything.
There's still the issue with a module not having a matching filename. -- /Jacob Carlborg
Jun 07 2015
parent =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com> writes:
Am 08.06.2015 um 08:47 schrieb Jacob Carlborg:
 On 2015-06-08 08:04, Sönke Ludwig wrote:

 Isn't just the list of *modules* needed and not the list of *imports*?
 The registry can easily query the list of files of a package using the
 GitHub API. The package description is obviously already known, too, so
 this would definitely be possible to do without compiling anything.
There's still the issue with a module not having a matching filename.
I think that will only work with a target type "sourceLibrary" anyway (since such modules always have to be on the compiler command line), so it will always be at least somewhat problematic. But there is already some code [1] to extract the module name for "dub test", so that would still be easy to do without actually invoking the compiler. [1]: https://github.com/D-Programming-Language/dub/blob/b3ea9e9f026bd096c60b4d1de55e2ea62bd313cd/source/dub/dub.d#L777
Jun 09 2015
prev sibling parent reply "ponce" <contact gam3sfrommars.fr> writes:
On Thursday, 4 June 2015 at 15:07:58 UTC, Sönke Ludwig wrote:
 Am 04.06.2015 um 15:07 schrieb Steven Schveighoffer:
 This would be better, but I still think a way to add 
 dependencies should
 be supported on the command line. Something like dub depend 
 someproject.
 You don't always know what a project will need at the 
 beginning.
imho if a program consumes a particular configuration file, it should not also write it. Counter examples: Visual Studio projects, MFC, IRC bouncers, and other technologies from Hell.
 On Thursday, 4 June 2015 at 15:07:58 UTC, Sönke Ludwig wrote:
 Sorry for the slightly OT reply, most of this isn't targeted at 
 you, I just went through the newsgroup posts for the first time 
 after a while and needed to get this out.
Be sure your work is appreciated, I couldn't imagine working without DUB everyday. As always the NG isn't the masses but some vocal subset of D users. Personally I won't use any library which is not on the DUB registry, learning how to build something is disposable knowledge.
Jun 04 2015
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/4/15 3:49 PM, ponce wrote:
 On Thursday, 4 June 2015 at 15:07:58 UTC, Sönke Ludwig wrote:
 Am 04.06.2015 um 15:07 schrieb Steven Schveighoffer:
 This would be better, but I still think a way to add dependencies should
 be supported on the command line. Something like dub depend someproject.
 You don't always know what a project will need at the beginning.
imho if a program consumes a particular configuration file, it should not also write it. Counter examples: Visual Studio projects, MFC, IRC bouncers, and other technologies from Hell.
That program is not consuming a configuration file, it's editing it. Call it dub-depend if you want. -Steve
Jun 04 2015
prev sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 06/01/2015 01:36 PM, Nick Sabalausky wrote:
 There is one part of that old proposal that I have come to like though,
 and wouldn't mind seeing: There was an idea to have DMD automatically
 invoke an external tool
Oops, forgot to finish writing that thought... As part of that old proposal, there was an idea to have DMD, when importing a module it couldn't find, automatically invoke an external tool which would tell DMD where to find the module. That way, a tool like RDMD would NOT need to invoke DMD twice (once to find the dependencies, and again to actually compile the dependencies). I've sort of come around more to that particular idea (not the entire proposal though), although actually using it would likely increase compile times on windows (notoriously slow at launching processes, just look at dustmite on win), and frankly it'd probably be better just to incorporate rdmd's dependency searching as an optional feature of dmd.
Jun 01 2015
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/1/15 12:16 PM, Nick Sabalausky wrote:
 On 06/01/2015 01:36 PM, Nick Sabalausky wrote:
 There is one part of that old proposal that I have come to like though,
 and wouldn't mind seeing: There was an idea to have DMD automatically
 invoke an external tool
Oops, forgot to finish writing that thought... As part of that old proposal, there was an idea to have DMD, when importing a module it couldn't find, automatically invoke an external tool which would tell DMD where to find the module. That way, a tool like RDMD would NOT need to invoke DMD twice (once to find the dependencies, and again to actually compile the dependencies).
rdmd could collect dependencies while performing the actual compilation. Then, if the dependencies are stable (compare with the saved ones), all done in one run. Otherwise, discard the output and rebuild. This can be implemented today. Would you want to work on it? Andrei
Jun 01 2015
next sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 06/01/2015 03:46 PM, Andrei Alexandrescu wrote:
 rdmd could collect dependencies while performing the actual compilation.
 Then, if the dependencies are stable (compare with the saved ones), all
 done in one run. Otherwise, discard the output and rebuild.
Hmm, actually, that's a good idea.
 This can be implemented today. Would you want to work on it?
I've gotten rusty on RDMD's internals, but I'll take a look, see what I can manage.
Jun 01 2015
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/1/15 12:59 PM, Nick Sabalausky wrote:
 I'll take a look, see what I
 can manage.
Thanks! -- Andrei P.S. Note the crafty editing of the quote :o).
Jun 01 2015
prev sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 06/01/2015 03:59 PM, Nick Sabalausky wrote:
 On 06/01/2015 03:46 PM, Andrei Alexandrescu wrote:
 rdmd could collect dependencies while performing the actual compilation.
 Then, if the dependencies are stable (compare with the saved ones), all
 done in one run. Otherwise, discard the output and rebuild.
Hmm, actually, that's a good idea.
 This can be implemented today. Would you want to work on it?
I've gotten rusty on RDMD's internals, but I'll take a look, see what I can manage.
Unfortunately, I've hit a bit of a problem: "dmd -v" doesn't emit an "import module_name (path/to)" line for imports that were provided on the command line. Aside from the minor issue of it complicating parts of the code, this makes it impossible (AFAICT) to distinguish between "the deps have not changed" vs "a former dep is no longer imported and should no longer be compiled".
Jun 01 2015
parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 06/02/2015 01:03 AM, Nick Sabalausky wrote:
 On 06/01/2015 03:59 PM, Nick Sabalausky wrote:
 On 06/01/2015 03:46 PM, Andrei Alexandrescu wrote:
 rdmd could collect dependencies while performing the actual compilation.
 Then, if the dependencies are stable (compare with the saved ones), all
 done in one run. Otherwise, discard the output and rebuild.
Hmm, actually, that's a good idea.
 This can be implemented today. Would you want to work on it?
I've gotten rusty on RDMD's internals, but I'll take a look, see what I can manage.
Unfortunately, I've hit a bit of a problem: "dmd -v" doesn't emit an "import module_name (path/to)" line for imports that were provided on the command line. Aside from the minor issue of it complicating parts of the code, this makes it impossible (AFAICT) to distinguish between "the deps have not changed" vs "a former dep is no longer imported and should no longer be compiled".
Related to this, does anyone happen to recall why rdmd uses "dmd -v" to get dependencies instead of "dmd -deps"? IIRC, it used to use -deps back at one time.
Jun 02 2015
parent Jacob Carlborg <doob me.com> writes:
On 2015-06-02 17:01, Nick Sabalausky wrote:

 Related to this, does anyone happen to recall why rdmd uses "dmd -v" to
 get dependencies instead of "dmd -deps"? IIRC, it used to use -deps back
 at one time.
I would assumed it uses "-v" because it existed before "-deps"? I guess "-deps" was explicitly create because of the rdmd use case. -- /Jacob Carlborg
Jun 03 2015
prev sibling parent "Atila Neves" <atila.neves gmail.com> writes:
 rdmd could collect dependencies while performing the actual 
 compilation. Then, if the dependencies are stable (compare with 
 the saved ones), all done in one run. Otherwise, discard the 
 output and rebuild.
I wrote a silly tool that gathers dependencies as it compiles. Essentially, it calls dmd -v and parses the result as rdmd does. The net effect is something akin to gcc's -MF -MD combo. It's part of the build system I keep pestering everyone about. I had to write it because Ninja depends on it for proper dependency tracking. Make needs it too, but the Makefile code I have to generate is best not looked at. Make's fault, not mine. Atila
Jun 01 2015
prev sibling parent "Rikki Cattermole" <alphaglosined gmail.com> writes:
On Monday, 1 June 2015 at 19:16:31 UTC, Nick Sabalausky wrote:
 On 06/01/2015 01:36 PM, Nick Sabalausky wrote:
 There is one part of that old proposal that I have come to 
 like though,
 and wouldn't mind seeing: There was an idea to have DMD 
 automatically
 invoke an external tool
Oops, forgot to finish writing that thought... As part of that old proposal, there was an idea to have DMD, when importing a module it couldn't find, automatically invoke an external tool which would tell DMD where to find the module. That way, a tool like RDMD would NOT need to invoke DMD twice (once to find the dependencies, and again to actually compile the dependencies). I've sort of come around more to that particular idea (not the entire proposal though), although actually using it would likely increase compile times on windows (notoriously slow at launching processes, just look at dustmite on win), and frankly it'd probably be better just to incorporate rdmd's dependency searching as an optional feature of dmd.
Of course there is always shared libraries as an alternative to executables.
Jun 01 2015