www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Error after installing DMD v2.063

reply "Gary Willoughby" <dev kalekold.net> writes:
Hi,

I get the following error while running a simple hello world 
program compiled with the new v2.063 compiler. This is on Ubuntu 
12.04.

error while loading shared libraries: libphobos2.so.0.63: cannot 
open shared object file: No such file or directory
Jun 02 2013
next sibling parent reply "Gary Willoughby" <dev kalekold.net> writes:
I've just run:

sudo ln -s /usr/lib/x86_64-linux-gnu/libphobos2.so 
/usr/lib/x86_64-linux-gnu/libphobos2.so.0.63

for now but i've never had to do that before. Is this a problem 
with the installer?
Jun 02 2013
next sibling parent Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 06/02/2013 06:47 AM, Gary Willoughby wrote:
 I've just run:

 sudo ln -s /usr/lib/x86_64-linux-gnu/libphobos2.so
 /usr/lib/x86_64-linux-gnu/libphobos2.so.0.63

 for now but i've never had to do that before. Is this a problem with the
 installer?

same problem with rpm installer. http://d.puremagic.com/issues/show_bug.cgi?id=10245
Jun 02 2013
prev sibling next sibling parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 06/02/2013 02:00 PM, Russel Winder wrote:
 is non-standard and not compliant. The standard structure should be:

 	libphobos2.so.0.63 the file
 	libphobos2.so.0 a symbolic link to libphobos2.so.0.63
 	libphobos2.so a symbolic link to libphobos2.so.0

what is libphobos2.0 ?
Jun 02 2013
parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 06/02/2013 02:51 PM, Russel Winder wrote:
 On Sun, 2013-06-02 at 14:13 -0700, Ellery Newcomer wrote:
 On 06/02/2013 02:00 PM, Russel Winder wrote:
 is non-standard and not compliant. The standard structure should be:

 	libphobos2.so.0.63 the file
 	libphobos2.so.0 a symbolic link to libphobos2.so.0.63
 	libphobos2.so a symbolic link to libphobos2.so.0

what is libphobos2.0

It is the soname concept. A soname specifies the number of API compatible versions. Basically it relates to the major version of an API. cf. for example: http://en.wikipedia.org/wiki/Soname http://tldp.org/HOWTO/Program-Library-HOWTO/shared-libraries.html http://www.netfort.gr.jp/~dancer/column/libpkg-guide/libpkg-guide.html#shlibpkgs

$ objdump -p libphobos2.so | grep SONAME SONAME libphobos2.so.0.63
Jun 02 2013
parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 06/02/2013 04:12 PM, Russel Winder wrote:
 On Sun, 2013-06-02 at 16:03 -0700, Ellery Newcomer wrote:
 […]
 $ objdump -p libphobos2.so | grep SONAME
     SONAME               libphobos2.so.0.63

Exactly, the actual file should have the fully qualified soname and all other filenames should be symbolic links to that file. Currently the DMD deb reverses this and therefore violates the standard for deb installation.

actually, your resource above says that the soname should have the format lib{lib}.so.X and the real name should have the format lib{lib}.so.X.Y.Z where X = version number Y = minor version number Z = release number so the generated .so itself violates the standard.
Jun 02 2013
parent reply Mike Wey <mike-wey example.com> writes:
On 06/03/2013 01:30 AM, Ellery Newcomer wrote:
 On 06/02/2013 04:12 PM, Russel Winder wrote:
 On Sun, 2013-06-02 at 16:03 -0700, Ellery Newcomer wrote:
 […]
 $ objdump -p libphobos2.so | grep SONAME
     SONAME               libphobos2.so.0.63

Exactly, the actual file should have the fully qualified soname and all other filenames should be symbolic links to that file. Currently the DMD deb reverses this and therefore violates the standard for deb installation.

actually, your resource above says that the soname should have the format lib{lib}.so.X and the real name should have the format lib{lib}.so.X.Y.Z where X = version number Y = minor version number Z = release number so the generated .so itself violates the standard.

Currently the Phobos make file generates: libphobos2.so.0.63.0 and creates two simlinks libphobos2.so and libphobos2.so.0.63, and sets the soname to libphobos2.s0.0.63. The soname currently includes the minor version number because the compatibility currently breaks every release, when the phobos abi is more stable it should be removed from the soname. -- Mike Wey
Jun 03 2013
parent Mike Wey <mike-wey example.com> writes:
On 06/03/2013 08:25 PM, Russel Winder wrote:
 Also there should be a symbolic link libphobos2.so.0 shouldn't there?

At some point yes, the minor version should be dropped from the so/symlink used by the application and is set as the soname. But i would wait for phobos/dmd to stabilize, so there whould be no need to increase the so version every release. -- Mike Wey
Jun 03 2013
prev sibling parent Martin Nowak <code dawg.eu> writes:
On 06/02/2013 03:47 PM, Gary Willoughby wrote:
 I've just run:

 sudo ln -s /usr/lib/x86_64-linux-gnu/libphobos2.so
 /usr/lib/x86_64-linux-gnu/libphobos2.so.0.63

 for now but i've never had to do that before. Is this a problem with the
 installer?

Sorry that was my fault, I missed that when fixing up the installers. https://github.com/D-Programming-Language/installer/pull/19
Jun 05 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, June 02, 2013 15:47:36 Gary Willoughby wrote:
 I've just run:
 
 sudo ln -s /usr/lib/x86_64-linux-gnu/libphobos2.so
 /usr/lib/x86_64-linux-gnu/libphobos2.so.0.63
 
 for now but i've never had to do that before. Is this a problem
 with the installer?

If you had to do that, then yes. - Jonathan M Davis
Jun 02 2013
prev sibling next sibling parent Russel Winder <russel winder.org.uk> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Sun, 2013-06-02 at 13:11 -0700, Jonathan M Davis wrote:
 On Sunday, June 02, 2013 15:47:36 Gary Willoughby wrote:
 I've just run:
=20
 sudo ln -s /usr/lib/x86_64-linux-gnu/libphobos2.so
 /usr/lib/x86_64-linux-gnu/libphobos2.so.0.63
=20
 for now but i've never had to do that before. Is this a problem
 with the installer?

If you had to do that, then yes.

I didn't have to do that on my Debian Unstable, but maybe I was lucky. However the structure: libphobos2.so the file libphobos2.so.0.63 a symbolic link to libphobos2.so is non-standard and not compliant. The standard structure should be: libphobos2.so.0.63 the file libphobos2.so.0 a symbolic link to libphobos2.so.0.63 libphobos2.so a symbolic link to libphobos2.so.0 --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Jun 02 2013
prev sibling next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, June 02, 2013 22:00:43 Russel Winder wrote:
 On Sun, 2013-06-02 at 13:11 -0700, Jonathan M Davis wrote:
 On Sunday, June 02, 2013 15:47:36 Gary Willoughby wrote:
 I've just run:
 
 sudo ln -s /usr/lib/x86_64-linux-gnu/libphobos2.so
 /usr/lib/x86_64-linux-gnu/libphobos2.so.0.63
 
 for now but i've never had to do that before. Is this a problem
 with the installer?

If you had to do that, then yes.

I didn't have to do that on my Debian Unstable, but maybe I was lucky. However the structure: libphobos2.so the file libphobos2.so.0.63 a symbolic link to libphobos2.so is non-standard and not compliant. The standard structure should be: libphobos2.so.0.63 the file libphobos2.so.0 a symbolic link to libphobos2.so.0.63 libphobos2.so a symbolic link to libphobos2.so.0

Create a bug report for it. I don't think that Walter realizes what's standard. And this whole problem probably stems from the fact that he can't put symlinks in the zip file that he generates, so he was talking about making it up to the maintainers to create the various symlinks, which IMHO is unacceptable long term, but it seems to require either changing the main release from a zip to something like 7zip or breaking it up per OS like a number of people have been asking for for ages. Regardless, the fact that the installer and rpm and whatnot are broken is unacceptable. - Jonathan M Davis
Jun 02 2013
next sibling parent Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 06/02/2013 02:20 PM, Jonathan M Davis wrote:
 he was talking about making
 it up to the maintainers to create the various symlinks, which IMHO is
 unacceptable long term

why?
Jun 02 2013
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/2/2013 2:20 PM, Jonathan M Davis wrote:
 Create a bug report for it. I don't think that Walter realizes what's
 standard. And this whole problem probably stems from the fact that he can't
 put symlinks in the zip file that he generates, so he was talking about making
 it up to the maintainers to create the various symlinks, which IMHO is
 unacceptable long term, but it seems to require either changing the main
 release from a zip to something like 7zip or breaking it up per OS like a
 number of people have been asking for for ages. Regardless, the fact that the
 installer and rpm and whatnot are broken is unacceptable.

It already is broken up per OS: http://dlang.org/download.html As for creating the installers and rpms, here are the scripts that do it: https://github.com/D-Programming-Language/installer Pull requests are welcome.
Jun 02 2013
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/2/2013 4:54 PM, Jonathan M Davis wrote:
 On Sunday, June 02, 2013 16:50:00 Walter Bright wrote:
 It already is broken up per OS:

     http://dlang.org/download.html

But the _zip_ file isn't. We keep asking for _that_ to be broken up.

There's no reason to, as broken up ones are available for each system.
Jun 02 2013
next sibling parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 06/02/2013 05:15 PM, Jonathan M Davis wrote:
 On Sunday, June 02, 2013 17:08:41 Walter Bright wrote:

 Regardless, the symlink issue alone shows that using the zip file format for
 *nix is a mistake. Any packages released for *nix needs to support symlinks
 correctly.

 - Jonathan M Davis

Is it so hard to convert the zip to tarball in the installer scripts? (hint: no)
Jun 02 2013
next sibling parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 06/02/2013 05:29 PM, Jonathan M Davis wrote:
 On Sunday, June 02, 2013 17:24:47 Ellery Newcomer wrote:
 On 06/02/2013 05:15 PM, Jonathan M Davis wrote:
 On Sunday, June 02, 2013 17:08:41 Walter Bright wrote:

 Regardless, the symlink issue alone shows that using the zip file format
 for *nix is a mistake. Any packages released for *nix needs to support
 symlinks correctly.

 - Jonathan M Davis

Is it so hard to convert the zip to tarball in the installer scripts? (hint: no)

But unless the zip itself is fixed, then anyone downloading it is going to end up with a screwed up *nix setup unless they fix it themselves. It would be far better to just fix the zip. - Jonathan M Davis

wouldn't they have a screwed up setup with a tarball as well? They'd still need to fix $PATH and whatever you need to fix to make -lphobos2 work.
Jun 02 2013
parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 06/02/2013 05:47 PM, Jonathan M Davis wrote:
 On Sunday, June 02, 2013 17:43:20 Ellery Newcomer wrote:
 On 06/02/2013 05:29 PM, Jonathan M Davis wrote:
 On Sunday, June 02, 2013 17:24:47 Ellery Newcomer wrote:
 On 06/02/2013 05:15 PM, Jonathan M Davis wrote:
 On Sunday, June 02, 2013 17:08:41 Walter Bright wrote:

 Regardless, the symlink issue alone shows that using the zip file format
 for *nix is a mistake. Any packages released for *nix needs to support
 symlinks correctly.

 - Jonathan M Davis

Is it so hard to convert the zip to tarball in the installer scripts? (hint: no)

But unless the zip itself is fixed, then anyone downloading it is going to end up with a screwed up *nix setup unless they fix it themselves. It would be far better to just fix the zip. - Jonathan M Davis

wouldn't they have a screwed up setup with a tarball as well? They'd still need to fix $PATH and whatever you need to fix to make -lphobos2 work.

All you should have to do is set the PATH so that it has dmd in it. Everything else should just work. - Jonathan M Davis

Okay, so I unzip to /home/ellery/Downloads/dmd2, add ~/Downloads/dmd2/linux/bin64 to PATH, and try to build a shared library. At runtime, it gives me ./test1.x: error while loading shared libraries: libphobos2.so.0.63: cannot open shared object file: No such file or directory And I don't even remember how to fix this. You have to muck around with ldconfig, which requires root, or something. Maybe there is a way to make the compiler point the shared lib dependency to /home/ellery/Downloads/linux/lib64/libphobos2.so ? But yuck.
Jun 02 2013
parent Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 06/02/2013 06:20 PM, Jonathan M Davis wrote:
 I don't believe that it's not an ldconfig problem. It's the fact that there's a
 libphobos2.so and not a libphobos2.so.0.63. It's the exact same problem that
 the rpm and deb files are having. dmd.conf already makes it so that the linker
 looks in the right place.

 - Jonathan M Davis

Aha! you need to put -defaultlib=libphobos2.so for compiling the executable too! and now it complains about not having libcurl-gnutls, which doesn't even exist in fedora, so ln -s /usr/lib64/libcurl.so.4 /usr/lib64/libcurl-gnutls.so.4 then I get /home/ellery/Downloads/dmd2/linux/bin64/../lib64/libphobos2.so: undefined reference to `curl_global_init CURL_GNUTLS_3' /home/ellery/Downloads/dmd2/linux/bin64/../lib64/libphobos2.so: undefined reference to `curl_easy_cleanup CURL_GNUTLS_3' /home/ellery/Downloads/dmd2/linux/bin64/../lib64/libphobos2.so: undefined reference to `curl_slist_append CURL_GNUTLS_3' /home/ellery/Downloads/dmd2/linux/bin64/../lib64/libphobos2.so: undefined reference to `curl_easy_init CURL_GNUTLS_3' /home/ellery/Downloads/dmd2/linux/bin64/../lib64/libphobos2.so: undefined reference to `curl_easy_pause CURL_GNUTLS_3' /home/ellery/Downloads/dmd2/linux/bin64/../lib64/libphobos2.so: undefined reference to `curl_slist_free_all CURL_GNUTLS_3' /home/ellery/Downloads/dmd2/linux/bin64/../lib64/libphobos2.so: undefined reference to `curl_easy_duphandle CURL_GNUTLS_3' /home/ellery/Downloads/dmd2/linux/bin64/../lib64/libphobos2.so: undefined reference to `curl_easy_strerror CURL_GNUTLS_3' /home/ellery/Downloads/dmd2/linux/bin64/../lib64/libphobos2.so: undefined reference to `curl_easy_perform CURL_GNUTLS_3' /home/ellery/Downloads/dmd2/linux/bin64/../lib64/libphobos2.so: undefined reference to `curl_global_cleanup CURL_GNUTLS_3' /home/ellery/Downloads/dmd2/linux/bin64/../lib64/libphobos2.so: undefined reference to `curl_easy_setopt CURL_GNUTLS_3' So I concede the point about ldconfig (I'm thinking what I used was LD_LIBRARY_PATH) hey! the rpm behaves the same way! Maybe building a fedora package on ubuntu is in fact a terrible idea!
Jun 02 2013
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/2/13 8:24 PM, Ellery Newcomer wrote:
 On 06/02/2013 05:15 PM, Jonathan M Davis wrote:
 On Sunday, June 02, 2013 17:08:41 Walter Bright wrote:

 Regardless, the symlink issue alone shows that using the zip file
 format for
 *nix is a mistake. Any packages released for *nix needs to support
 symlinks
 correctly.

 - Jonathan M Davis

Is it so hard to convert the zip to tarball in the installer scripts? (hint: no)

symlinks? Andrei
Jun 02 2013
parent Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 06/02/2013 07:33 PM, Andrei Alexandrescu wrote:
 On 6/2/13 8:24 PM, Ellery Newcomer wrote:
 On 06/02/2013 05:15 PM, Jonathan M Davis wrote:
 On Sunday, June 02, 2013 17:08:41 Walter Bright wrote:

 Regardless, the symlink issue alone shows that using the zip file
 format for
 *nix is a mistake. Any packages released for *nix needs to support
 symlinks
 correctly.

 - Jonathan M Davis

Is it so hard to convert the zip to tarball in the installer scripts? (hint: no)

symlinks? Andrei

oh, hey. there's a %post thinggummy, so %post ln -s /usr/lib64/{libphobos2.so,$SONAME} would probably work great too.
Jun 02 2013
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/2/2013 5:15 PM, Jonathan M Davis wrote:
 On Sunday, June 02, 2013 17:08:41 Walter Bright wrote:
 On 6/2/2013 4:54 PM, Jonathan M Davis wrote:
 On Sunday, June 02, 2013 16:50:00 Walter Bright wrote:
 It already is broken up per OS:
      http://dlang.org/download.html

But the _zip_ file isn't. We keep asking for _that_ to be broken up.

There's no reason to, as broken up ones are available for each system.

Except that those are rpms and debs and whatnot which are different. I'm far from the first to bring this up. It has been brought up repeatedly for quite some time that folks don't want the zip to contain everything. They want to get the zip (or tarball) for their specific OS and that's it.

That's what the rpms and debs are for.
 Using an rpm and deb is fundamentally different,

The complaint from Russel was about the .deb file. In any case, anyone is free to create a script to build whatever combination they want, in any format they want, and submit it as a pull request to installer. Nobody has to wait on me to do it. https://github.com/D-Programming-Language/installer
Jun 02 2013
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/3/2013 8:59 AM, Russel Winder wrote:
 Is there a tarball or only this infamous zipfile?

I can't believe it is hard to create a tarball from a zipfile.
Jun 03 2013
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/3/2013 12:50 PM, Russel Winder wrote:
 On Mon, 2013-06-03 at 12:38 -0700, Walter Bright wrote:
 On 6/3/2013 8:59 AM, Russel Winder wrote:
 Is there a tarball or only this infamous zipfile?

I can't believe it is hard to create a tarball from a zipfile.

Actually it can be since tarballs can represent symbolic links whereas it appears zipfiles cannot.

1. unzip the zip file 2. add symbolic links 3. create tarball
 However, I think the issue here re deb files
 is deeper. The solution to the issue as far as generating a deb is
 concerned is to tag the DMD Git repository so that anyone can generate
 an official source tarball given the release tag.

It's already tagged with v2.063
 I appreciate there are licencing issues that means we cannot expect to
 get DMD into Debian itself, if we can have a Debian toolchain build that
 targets Debian non-free, we can get DMD into Ubuntu and Mint as well as
 accesible for Debian users. I believe this to be a good aim.

Sounds good to me.
Jun 03 2013
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/3/2013 5:34 PM, Jonathan M Davis wrote:
 But why create it as a zip file in the first place?

Because creating tarballs on Windows is a big problem, not the least of which you can't create symlinks on Windows, so even if you found a working Windows tar program you still haven't solved the problem.
 Just create the tarball
 correctly when it's built and don't create a zip file at all. Clearly, people
 other than you are going to be sorting this out, but we shouldn't even be
 creating a zip file for the non-Windows OSes in the first place.
 The symlinks should be created as part of the build and tar-ed from there.

It isn't any harder creating a symlink after unzipping than it is creating one before creating a tarball.
Jun 04 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-06-04 09:52, Walter Bright wrote:
 On 6/3/2013 5:34 PM, Jonathan M Davis wrote:
 But why create it as a zip file in the first place?

Because creating tarballs on Windows is a big problem, not the least of which you can't create symlinks on Windows, so even if you found a working Windows tar program you still haven't solved the problem.

You can create symlinks on Windows, on versions later than XP. Although I don't know if they're compatible with Posix. -- /Jacob Carlborg
Jun 04 2013
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/2/13 8:15 PM, Jonathan M Davis wrote:
 On Sunday, June 02, 2013 17:08:41 Walter Bright wrote:
 On 6/2/2013 4:54 PM, Jonathan M Davis wrote:
 On Sunday, June 02, 2013 16:50:00 Walter Bright wrote:
 It already is broken up per OS:
      http://dlang.org/download.html

But the _zip_ file isn't. We keep asking for _that_ to be broken up.

There's no reason to, as broken up ones are available for each system.

Except that those are rpms and debs and whatnot which are different. I'm far from the first to bring this up. It has been brought up repeatedly for quite some time that folks don't want the zip to contain everything. They want to get the zip (or tarball) for their specific OS and that's it. Using an rpm and deb is fundamentally different, and if that were enough, why do we even have the zip? Separating the OSes out and having an installer are two separate issues (much as they have to be separated out for the installers), and most of us want the zip to be separated. I don't think that much of anyone around here thinks that the zip should contain all of the OSes. I believe that you're the only one that's actually thought that it was a good idea whenever it's been discussed. And the zip is only going to get bigger and bigger as we add shared library support (and possibly more OSes in the future). Regardless, the symlink issue alone shows that using the zip file format for *nix is a mistake. Any packages released for *nix needs to support symlinks correctly.

I quoted all of the above because I so much agree with it. Packaging all OSs in a zip is really disingenuous and now it's come to a head. Let's fix this for 2.064. Andrei
Jun 02 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-06-03 04:33, Andrei Alexandrescu wrote:

 I quoted all of the above because I so much agree with it. Packaging all
 OSs in a zip is really disingenuous and now it's come to a head. Let's
 fix this for 2.064.

DVM relies on DMD being packaged as a single zip. As long as that is kept, you can do whatever you want. There are probably other tools that relies on this as well. -- /Jacob Carlborg
Jun 03 2013
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-06-03 02:15, Jonathan M Davis wrote:

 I don't think that much of anyone around here
 thinks that the zip should contain all of the OSes.

DVM relies on DMD being packaged as a single zip. As long as that is kept, you can do whatever you want. -- /Jacob Carlborg
Jun 03 2013
parent reply Jacob Carlborg <doob me.com> writes:
On 2013-06-03 09:37, Jonathan M Davis wrote:

 Except that that's _exactly_ what we want to get rid of. It's ridiculous to
 put them all in one zip. It just wastes bandwidth, and it doesn't work with
 symlinks, and now that we're adding shared libraries, we need the *nix
 packages to have symlinks in them.

 I understand that DVM currently relies on there being a single zip, but aside
 from trying not to break DVM, I see zero reason to leave it as a single zip.
 It's just causing us problems.

If it was not clear, I would like to keep the cross-platform zip in addition to the platform specific zip/tarball/packages. -- /Jacob Carlborg
Jun 03 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-06-03 10:34, Jonathan M Davis wrote:

 Well, part of the problem is that the zip is inherently broken for *nix
 systems due to the fact that symlinks don't work properly. So, now that we
 have an so version of Phobos, the zip just isn't going to work properly
 anymore. And while we aren't really looking to break DVM, AFAIK, there would
 be no other reason to keep the zip around other than for DVM. And since the
 zip isn't going to work properly on *nix systems anyway, I'm not sure that
 keeping it around for DVM really solves much.

*nix is really not correct to say. Currently only Linux 64bit supports shared libraries. It's not like it's broken on all platforms, just one. Sure, it will break once we get support for share libraries on additional platforms. If static linking is default nothing is broken. Which format can we use for these platform specific packages? For which of these are there existing bindings or libraries? -- /Jacob Carlborg
Jun 03 2013
prev sibling next sibling parent Piotr Szturmaj <bncrbme jadamspam.pl> writes:
W dniu 02.06.2013 15:05, Gary Willoughby pisze:
 Hi,

 I get the following error while running a simple hello world program
 compiled with the new v2.063 compiler. This is on Ubuntu 12.04.

 error while loading shared libraries: libphobos2.so.0.63: cannot open
 shared object file: No such file or directory

I have installed 2.063 version on Ubuntu 12.10 without problems. I was using downloaded .deb file which updated older apt-get installed version 2.062. So, at least on Ubuntu 12.10 it's working properly.
Jun 02 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, June 02, 2013 14:29:39 Ellery Newcomer wrote:
 On 06/02/2013 02:20 PM, Jonathan M Davis wrote:
 he was talking about making
 it up to the maintainers to create the various symlinks, which IMHO is
 unacceptable long term

why?

Because of the problems of zip with symlinks. He would have to change how they're created (and probably not use zip at all) in order to fix the problem, and apparently, he wasn't willing to do that for this release. That doesn't mean that the problem can't or won't be fixed for the next release, but it required too many changes to how Walter normally functions for him to be willing to do it correctly for this release. Nick Sabalausky was looking at fixing up how the zip files are generated for Walter so that they we have separate packages for each OS, and if sorts that out, then I'm sure that the symlink issue can and will be sorted out. I believe that the push is to fully automate generating the release packages (probably using the autotester), but it pretty much requires folks other than Walter making those changes, since he doesn't change his habits and procedures with regards to this sort of thing very easily. - Jonathan M Davis
Jun 02 2013
prev sibling next sibling parent Russel Winder <russel winder.org.uk> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Sun, 2013-06-02 at 14:39 -0700, Jonathan M Davis wrote:
[=E2=80=A6]
 Nick Sabalausky was looking at fixing up how the zip files are generated =

 Walter so that they we have separate packages for each OS, and if sorts t=

 out, then I'm sure that the symlink issue can and will be sorted out. I=

 believe that the push is to fully automate generating the release package=

 (probably using the autotester), but it pretty much requires folks other =

 Walter making those changes, since he doesn't change his habits and proce=

 with regards to this sort of thing very easily.

For Linux just use a tarball and the symlinks problem goes away since they can be represented. This covers all Posix compliant systems which leaves just Windows. Which I don't care about ;-) If Walter is not willing to get this right then he mut pass on the responsibility to someone who is. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Jun 02 2013
prev sibling next sibling parent Russel Winder <russel winder.org.uk> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Sun, 2013-06-02 at 14:13 -0700, Ellery Newcomer wrote:
 On 06/02/2013 02:00 PM, Russel Winder wrote:
 is non-standard and not compliant. The standard structure should be:

 	libphobos2.so.0.63 the file
 	libphobos2.so.0 a symbolic link to libphobos2.so.0.63
 	libphobos2.so a symbolic link to libphobos2.so.0

what is libphobos2.0

It is the soname concept. A soname specifies the number of API compatible versions. Basically it relates to the major version of an API. cf. for example: http://en.wikipedia.org/wiki/Soname http://tldp.org/HOWTO/Program-Library-HOWTO/shared-libraries.html http://www.netfort.gr.jp/~dancer/column/libpkg-guide/libpkg-guide.html#shli= bpkgs --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Jun 02 2013
prev sibling next sibling parent reply Russel Winder <russel winder.org.uk> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Sun, 2013-06-02 at 14:20 -0700, Jonathan M Davis wrote:
[=E2=80=A6]
 Create a bug report for it. I don't think that Walter realizes what's=20
 standard. And this whole problem probably stems from the fact that he can=

 put symlinks in the zip file that he generates, so he was talking about m=

 it up to the maintainers to create the various symlinks, which IMHO is=

 unacceptable long term, but it seems to require either changing the main=

 release from a zip to something like 7zip or breaking it up per OS like a=

 number of people have been asking for for ages. Regardless, the fact that=

 installer and rpm and whatnot are broken is unacceptable.

Not being able to put symlinks in a zipfile has nothing to do with getting deb files and RPM files correct since they should be built from tarballs. Sounds like the system for creating distributions is broken. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Jun 02 2013
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/2/2013 2:52 PM, Russel Winder wrote:
 Not being able to put symlinks in a zipfile has nothing to do with
 getting deb files and RPM files correct since they should be built from
 tarballs.

Anyone is welcome to help out with the scripts that create deb and rpm files, which are here: https://github.com/D-Programming-Language/installer
Jun 02 2013
prev sibling next sibling parent Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 06/02/2013 03:25 PM, Nick Sabalausky wrote:
 On Sun, 02 Jun 2013 22:52:59 +0100
 Russel Winder <russel winder.org.uk> wrote:
 Sounds like the system for creating distributions is broken.

Yea, I'm working on a replacement.

do tell
Jun 02 2013
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-06-03 00:25, Nick Sabalausky wrote:

 Yea, I'm working on a replacement.

Please keep the existing zip packages as well, we don't want to break DVM :) -- /Jacob Carlborg
Jun 03 2013
parent reply Jacob Carlborg <doob me.com> writes:
On 2013-06-03 23:08, Nick Sabalausky wrote:

 I intend to. But I do think it would be good if we then update DVM
 to use the platform-specific versions, to preserve bandwidth and
 improve dload/installation speeds.

I would like to rewrite the whole tool. Then we can fix all these things. Haven't taken the time to do it yet.
 Possibly related to that, I keep meaning to look into updating DVM to
 install *both* -m32 and -m64 abilities on appropriate platforms (unless
 you've already beaten me to it?)

No, I haven't done that. See above. BTW, DVM is not really working on Windows XP, or rather the installed compilers. DVM installs the compilers in a path which contains spaces. The compiler and/or the linker isn't happy with that. -- /Jacob Carlborg
Jun 03 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-06-04 09:39, Nick Sabalausky wrote:

 Just curious: What sort of direction did you have in mind for a rewrite?
 Something about the general architecture of it you'd do differently?

* All the functionality shouldn't be in the commands. The commands should invoke library functions/classes. I won't DVM to be usable as a library. The tool should just be a thin wrapper/user interface around the library. * I want to support other compilers, at least LDC (GDC doesn't seem so easy). * The Application module and command line arguments handling is not very DRY. I have factored that out in separate modules in other tools I'm building (DStep, Orbit), I plan to reuse that. I guess those are the major things I want to do. Then of course fixing all these little things we have been talking about for a while. -- /Jacob Carlborg
Jun 04 2013
prev sibling next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, June 02, 2013 22:52:59 Russel Winder wrote:
 On Sun, 2013-06-02 at 14:20 -0700, Jonathan M Davis wrote:
 [=E2=80=A6]
=20
 Create a bug report for it. I don't think that Walter realizes what=


 standard. And this whole problem probably stems from the fact that =


 can't
 put symlinks in the zip file that he generates, so he was talking a=


 making it up to the maintainers to create the various symlinks, whi=


 IMHO is unacceptable long term, but it seems to require either chan=


 the main release from a zip to something like 7zip or breaking it u=


 OS like a number of people have been asking for for ages. Regardles=


 fact that the installer and rpm and whatnot are broken is unaccepta=


=20
 Not being able to put symlinks in a zipfile has nothing to do with
 getting deb files and RPM files correct since they should be built fr=

 tarballs.
=20
 Sounds like the system for creating distributions is broken.

It's done entirely by Walter on his own systems, and I suspect that the= deb=20 and rpm files are created from the zip file (though I'm not sure if he = creates=20 those or someone else does). We need to change it so that the process f= or=20 generating them is automated and does not rely on Walter. Work is being= done=20 in that area, but it does not appear to be a priority for Walter. - Jonathan M Davis
Jun 02 2013
next sibling parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 06/02/2013 03:05 PM, Jonathan M Davis wrote:
 On Sunday, June 02, 2013 22:52:59 Russel Winder wrote:
 On Sun, 2013-06-02 at 14:20 -0700, Jonathan M Davis wrote:
 […]

 Create a bug report for it. I don't think that Walter realizes what's
 standard. And this whole problem probably stems from the fact that he
 can't
 put symlinks in the zip file that he generates, so he was talking about
 making it up to the maintainers to create the various symlinks, which
 IMHO is unacceptable long term, but it seems to require either changing
 the main release from a zip to something like 7zip or breaking it up per
 OS like a number of people have been asking for for ages. Regardless, the
 fact that the installer and rpm and whatnot are broken is unacceptable.

Not being able to put symlinks in a zipfile has nothing to do with getting deb files and RPM files correct since they should be built from tarballs. Sounds like the system for creating distributions is broken.

It's done entirely by Walter on his own systems, and I suspect that the deb and rpm files are created from the zip file (though I'm not sure if he creates those or someone else does). We need to change it so that the process for generating them is automated and does not rely on Walter. Work is being done in that area, but it does not appear to be a priority for Walter. - Jonathan M Davis

I thought the packages were generated using the scripts at https://github.com/D-Programming-Language/installer/tree/master/linux which pull the zip files from ftp.digitalmars.com What would an automated process look like? I would think something like a script which makes a tarball out of the git repos and then some more which turn the tarball into rpm or deb or what have you?
Jun 02 2013
next sibling parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 06/02/2013 04:50 PM, Jonathan M Davis wrote:
 On Sunday, June 02, 2013 16:42:12 Ellery Newcomer wrote:
 I thought the packages were generated using the scripts at

 https://github.com/D-Programming-Language/installer/tree/master/linux

 which pull the zip files from ftp.digitalmars.com

They may. I don't know what the current process is with regards to the rpms and debs, but that would mean that they are indeed using the zips, which is why you're having the problems that you're having.
 What would an automated process look like? I would think something like
 a script which makes a tarball out of the git repos and then some more
 which turn the tarball into rpm or deb or what have you?

Something like that. The autotester is already doing that sort of thing for building and running the unit tests. It shouldn't be that hard to make it then generate the correct packages as part of that for the cases where we want it, and if done right, the same process could be used on other boxes. - Jonathan M Davis

where is this mythical autotester, anyways?
Jun 02 2013
parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 06/02/2013 05:08 PM, Jonathan M Davis wrote:
 On Sunday, June 02, 2013 16:57:08 Ellery Newcomer wrote:
 where is this mythical autotester, anyways?

Mythical? Oh ye of little faith: http://d.puremagic.com/test-results/ dmd, druntime, and Phobos are built are their unit tests run after every commit. And the pull tester ( http://d.puremagic.com/test-results/pulls.ghtml?projectid=1 ) tests every pull request the same way (if the submitter is on the whitelist anyway). The code for the auto tester can be found here: https://github.com/braddr/d-tester - Jonathan M Davis

what distro are those linux tests performed on?
Jun 02 2013
parent Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 06/02/2013 05:29 PM, Jonathan M Davis wrote:
 On Sunday, June 02, 2013 17:20:37 Ellery Newcomer wrote:
 what distro are those linux tests performed on?

I don't know. You'd probably have to ask Brad Roberts. - Jonathan M Davis

just asking because the rpm script in installer is set up to only run on debian.
Jun 02 2013
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/2/13 7:42 PM, Ellery Newcomer wrote:
 I thought the packages were generated using the scripts at

 https://github.com/D-Programming-Language/installer/tree/master/linux

 which pull the zip files from ftp.digitalmars.com

 What would an automated process look like? I would think something like
 a script which makes a tarball out of the git repos and then some more
 which turn the tarball into rpm or deb or what have you?

Yah, get a specific tag from github and start from there. Andrei
Jun 02 2013
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/2/2013 3:05 PM, Jonathan M Davis wrote:
 It's done entirely by Walter on his own systems, and I suspect that the deb
 and rpm files are created from the zip file (though I'm not sure if he creates
 those or someone else does). We need to change it so that the process for
 generating them is automated and does not rely on Walter. Work is being done
 in that area, but it does not appear to be a priority for Walter.

Creating deps and rpms is done from the zip file, and the scripts are here: https://github.com/D-Programming-Language/installer I know I'm sounding like a broken record, but those scripts are not a mystery and anyone can produce pull requests to add to them or fix them.
Jun 03 2013
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/3/13 2:30 PM, Russel Winder wrote:
 On Mon, 2013-06-03 at 10:29 -0700, Walter Bright wrote:
 On 6/2/2013 3:05 PM, Jonathan M Davis wrote:
 It's done entirely by Walter on his own systems, and I suspect that the deb
 and rpm files are created from the zip file (though I'm not sure if he creates
 those or someone else does). We need to change it so that the process for
 generating them is automated and does not rely on Walter. Work is being done
 in that area, but it does not appear to be a priority for Walter.

Creating deps and rpms is done from the zip file, and the scripts are here: https://github.com/D-Programming-Language/installer I know I'm sounding like a broken record, but those scripts are not a mystery and anyone can produce pull requests to add to them or fix them.

As I noted earlier, the deb creation scripts in that repository are just fundamentally the wrong way of creating debs. Also as noted earlier I can't do anything about this till August. The result will not be a pull request on the script in the repository, but a new Git repository specifically designed to use the Debian standard deb creation toolchain and a request to delete the script mentioned above from the whole release toolset. Sorry guv but you have to use the right tool for the job, and deb creation should use the deb creation tools. The start point has to be a tarball of the source. If this is not part of the distribution release then we need to agree an officially acceptable process for creating a release source tarball. Thanks.

Instead of planning to work on it, one alternative would be to post bits and pieces of information in a bug report and guide others how to do it. Just a thought. Andrei
Jun 03 2013
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/3/13 2:47 PM, Russel Winder wrote:
 On Mon, 2013-06-03 at 14:32 -0400, Andrei Alexandrescu wrote:
 […]
 Instead of planning to work on it, one alternative would be to post bits
 and pieces of information in a bug report and guide others how to do it.
 Just a thought.

OK. How about trying: http://www.debian.org/doc/manuals/maint-guide/build.en.html http://wiki.debian.org/PackagingWithGit http://lpenz.org/articles/debgit/index.html http://honk.sigxcpu.org/projects/git-buildpackage/manual-html/gbp.html The core of this is having the source tarball and a debian directory with all the appropriate files so that git-buildpackage can do its stuff. I am not sure a bug report helps, what is needed is action and the creation of the Git repository to do the stuff. My problem is that I have Groovy, Python and Scala stuff to do for the next 2 months :-( I hope this helps.

Thanks. We've learned with time to organized ourselves really well around bug reports (both discussions and guides for implementation), so: http://d.puremagic.com/issues/show_bug.cgi?id=10256 Andrei
Jun 03 2013
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-06-03 19:29, Walter Bright wrote:

 Creating deps and rpms is done from the zip file, and the scripts are here:

     https://github.com/D-Programming-Language/installer

The Mac OS X installer is built from the zip file as well. If we change format or anything similar we need to update the script that builds the installer. -- /Jacob Carlborg
Jun 04 2013
prev sibling next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Sun, 02 Jun 2013 22:52:59 +0100
Russel Winder <russel winder.org.uk> wrote:
 
 Sounds like the system for creating distributions is broken.
 

Yea, I'm working on a replacement.
Jun 02 2013
prev sibling next sibling parent Russel Winder <russel winder.org.uk> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Sun, 2013-06-02 at 16:03 -0700, Ellery Newcomer wrote:
[=E2=80=A6]
 $ objdump -p libphobos2.so | grep SONAME
    SONAME               libphobos2.so.0.63

Exactly, the actual file should have the fully qualified soname and all other filenames should be symbolic links to that file. Currently the DMD deb reverses this and therefore violates the standard for deb installation. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Jun 02 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, June 02, 2013 16:42:12 Ellery Newcomer wrote:
 I thought the packages were generated using the scripts at
 
 https://github.com/D-Programming-Language/installer/tree/master/linux
 
 which pull the zip files from ftp.digitalmars.com

They may. I don't know what the current process is with regards to the rpms and debs, but that would mean that they are indeed using the zips, which is why you're having the problems that you're having.
 What would an automated process look like? I would think something like
 a script which makes a tarball out of the git repos and then some more
 which turn the tarball into rpm or deb or what have you?

Something like that. The autotester is already doing that sort of thing for building and running the unit tests. It shouldn't be that hard to make it then generate the correct packages as part of that for the cases where we want it, and if done right, the same process could be used on other boxes. - Jonathan M Davis
Jun 02 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, June 02, 2013 16:50:00 Walter Bright wrote:
 It already is broken up per OS:
 
    http://dlang.org/download.html

But the _zip_ file isn't. We keep asking for _that_ to be broken up. That's completely separate from whatever is going on with rpms or debs or exe installers. Plenty of people just use the zip file, and they don't want to have to download dmd for every OS it supports. And we are now having issues with symlinks because of the way the zip currently works. If we had a tarball for linux (or FreeBSD or Mac OS X, etc.), then that could trivially have the symlinks be correct, and the rpms and debs colud be correctly generated as well. Clearly, having everything in one zip is causing problems. - Jonathan M Davis
Jun 02 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, June 02, 2013 16:52:03 Walter Bright wrote:
 On 6/2/2013 2:52 PM, Russel Winder wrote:
 Not being able to put symlinks in a zipfile has nothing to do with
 getting deb files and RPM files correct since they should be built from
 tarballs.

Anyone is welcome to help out with the scripts that create deb and rpm files, which are here: https://github.com/D-Programming-Language/installer

Well, Nick is looking into some of that stuff. Hopefully, he'll be able to sort it out. - Jonathan M Davis
Jun 02 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, June 02, 2013 16:57:08 Ellery Newcomer wrote:
 where is this mythical autotester, anyways?

Mythical? Oh ye of little faith: http://d.puremagic.com/test-results/ dmd, druntime, and Phobos are built are their unit tests run after every commit. And the pull tester ( http://d.puremagic.com/test-results/pulls.ghtml?projectid=1 ) tests every pull request the same way (if the submitter is on the whitelist anyway). The code for the auto tester can be found here: https://github.com/braddr/d-tester - Jonathan M Davis
Jun 02 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, June 02, 2013 17:08:41 Walter Bright wrote:
 On 6/2/2013 4:54 PM, Jonathan M Davis wrote:
 On Sunday, June 02, 2013 16:50:00 Walter Bright wrote:
 It already is broken up per OS:
     http://dlang.org/download.html

But the _zip_ file isn't. We keep asking for _that_ to be broken up.

There's no reason to, as broken up ones are available for each system.

Except that those are rpms and debs and whatnot which are different. I'm far from the first to bring this up. It has been brought up repeatedly for quite some time that folks don't want the zip to contain everything. They want to get the zip (or tarball) for their specific OS and that's it. Using an rpm and deb is fundamentally different, and if that were enough, why do we even have the zip? Separating the OSes out and having an installer are two separate issues (much as they have to be separated out for the installers), and most of us want the zip to be separated. I don't think that much of anyone around here thinks that the zip should contain all of the OSes. I believe that you're the only one that's actually thought that it was a good idea whenever it's been discussed. And the zip is only going to get bigger and bigger as we add shared library support (and possibly more OSes in the future). Regardless, the symlink issue alone shows that using the zip file format for *nix is a mistake. Any packages released for *nix needs to support symlinks correctly. - Jonathan M Davis
Jun 02 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, June 02, 2013 17:24:47 Ellery Newcomer wrote:
 On 06/02/2013 05:15 PM, Jonathan M Davis wrote:
 On Sunday, June 02, 2013 17:08:41 Walter Bright wrote:
 
 Regardless, the symlink issue alone shows that using the zip file format
 for *nix is a mistake. Any packages released for *nix needs to support
 symlinks correctly.
 
 - Jonathan M Davis

Is it so hard to convert the zip to tarball in the installer scripts? (hint: no)

But unless the zip itself is fixed, then anyone downloading it is going to end up with a screwed up *nix setup unless they fix it themselves. It would be far better to just fix the zip. - Jonathan M Davis
Jun 02 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, June 02, 2013 17:20:37 Ellery Newcomer wrote:
 what distro are those linux tests performed on?

I don't know. You'd probably have to ask Brad Roberts. - Jonathan M Davis
Jun 02 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, June 02, 2013 17:39:50 Ellery Newcomer wrote:
 On 06/02/2013 05:29 PM, Jonathan M Davis wrote:
 On Sunday, June 02, 2013 17:20:37 Ellery Newcomer wrote:
 what distro are those linux tests performed on?

I don't know. You'd probably have to ask Brad Roberts. - Jonathan M Davis

just asking because the rpm script in installer is set up to only run on debian.

My guess would be Ubuntu, but I really have no idea. - Jonathan M Davis
Jun 02 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, June 02, 2013 17:43:20 Ellery Newcomer wrote:
 On 06/02/2013 05:29 PM, Jonathan M Davis wrote:
 On Sunday, June 02, 2013 17:24:47 Ellery Newcomer wrote:
 On 06/02/2013 05:15 PM, Jonathan M Davis wrote:
 On Sunday, June 02, 2013 17:08:41 Walter Bright wrote:
 
 Regardless, the symlink issue alone shows that using the zip file format
 for *nix is a mistake. Any packages released for *nix needs to support
 symlinks correctly.
 
 - Jonathan M Davis

Is it so hard to convert the zip to tarball in the installer scripts? (hint: no)

But unless the zip itself is fixed, then anyone downloading it is going to end up with a screwed up *nix setup unless they fix it themselves. It would be far better to just fix the zip. - Jonathan M Davis

wouldn't they have a screwed up setup with a tarball as well? They'd still need to fix $PATH and whatever you need to fix to make -lphobos2 work.

All you should have to do is set the PATH so that it has dmd in it. Everything else should just work. - Jonathan M Davis
Jun 02 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, June 02, 2013 17:50:46 Walter Bright wrote:
 In any case, anyone is free to create a script to build whatever combination
 they want, in any format they want, and submit it as a pull request to
 installer. Nobody has to wait on me to do it.
 
    https://github.com/D-Programming-Language/installer

I believe that that's what Nick is working on (and I think that he's probably the number one complainer about this issue). - Jonathan M Davis
Jun 02 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, June 02, 2013 18:12:06 Ellery Newcomer wrote:
 On 06/02/2013 05:47 PM, Jonathan M Davis wrote:
 On Sunday, June 02, 2013 17:43:20 Ellery Newcomer wrote:
 On 06/02/2013 05:29 PM, Jonathan M Davis wrote:
 On Sunday, June 02, 2013 17:24:47 Ellery Newcomer wrote:
 On 06/02/2013 05:15 PM, Jonathan M Davis wrote:
 On Sunday, June 02, 2013 17:08:41 Walter Bright wrote:
 
 Regardless, the symlink issue alone shows that using the zip file
 format
 for *nix is a mistake. Any packages released for *nix needs to support
 symlinks correctly.
 
 - Jonathan M Davis

Is it so hard to convert the zip to tarball in the installer scripts? (hint: no)

But unless the zip itself is fixed, then anyone downloading it is going to end up with a screwed up *nix setup unless they fix it themselves. It would be far better to just fix the zip. - Jonathan M Davis

wouldn't they have a screwed up setup with a tarball as well? They'd still need to fix $PATH and whatever you need to fix to make -lphobos2 work.>

Everything else should just work. - Jonathan M Davis

Okay, so I unzip to /home/ellery/Downloads/dmd2, add ~/Downloads/dmd2/linux/bin64 to PATH, and try to build a shared library. At runtime, it gives me ./test1.x: error while loading shared libraries: libphobos2.so.0.63: cannot open shared object file: No such file or directory And I don't even remember how to fix this. You have to muck around with ldconfig, which requires root, or something. Maybe there is a way to make the compiler point the shared lib dependency to /home/ellery/Downloads/linux/lib64/libphobos2.so ? But yuck.

I don't believe that it's not an ldconfig problem. It's the fact that there's a libphobos2.so and not a libphobos2.so.0.63. It's the exact same problem that the rpm and deb files are having. dmd.conf already makes it so that the linker looks in the right place. - Jonathan M Davis
Jun 02 2013
prev sibling next sibling parent Brad Roberts <braddr puremagic.com> writes:
On 6/2/13 7:33 PM, Andrei Alexandrescu wrote:
 On 6/2/13 8:15 PM, Jonathan M Davis wrote:
 On Sunday, June 02, 2013 17:08:41 Walter Bright wrote:
 On 6/2/2013 4:54 PM, Jonathan M Davis wrote:
 On Sunday, June 02, 2013 16:50:00 Walter Bright wrote:
 It already is broken up per OS:
      http://dlang.org/download.html

But the _zip_ file isn't. We keep asking for _that_ to be broken up.

There's no reason to, as broken up ones are available for each system.

Except that those are rpms and debs and whatnot which are different. I'm far from the first to bring this up. It has been brought up repeatedly for quite some time that folks don't want the zip to contain everything. They want to get the zip (or tarball) for their specific OS and that's it. Using an rpm and deb is fundamentally different, and if that were enough, why do we even have the zip? Separating the OSes out and having an installer are two separate issues (much as they have to be separated out for the installers), and most of us want the zip to be separated. I don't think that much of anyone around here thinks that the zip should contain all of the OSes. I believe that you're the only one that's actually thought that it was a good idea whenever it's been discussed. And the zip is only going to get bigger and bigger as we add shared library support (and possibly more OSes in the future). Regardless, the symlink issue alone shows that using the zip file format for *nix is a mistake. Any packages released for *nix needs to support symlinks correctly.

I quoted all of the above because I so much agree with it. Packaging all OSs in a zip is really disingenuous and now it's come to a head. Let's fix this for 2.064. Andrei

The pull requests I submitted for dmd/druntime/phobos are the start of automating the process. A few iterations of this and the auto-tester can create both snapshot builds as well as release builds against tags.
Jun 02 2013
prev sibling next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Sun, 02 Jun 2013 16:58:29 -0700
Jonathan M Davis <jmdavisProg gmx.com> wrote:

 On Sunday, June 02, 2013 16:52:03 Walter Bright wrote:
 On 6/2/2013 2:52 PM, Russel Winder wrote:
 Not being able to put symlinks in a zipfile has nothing to do with
 getting deb files and RPM files correct since they should be
 built from tarballs.

Anyone is welcome to help out with the scripts that create deb and rpm files, which are here: https://github.com/D-Programming-Language/installer

Well, Nick is looking into some of that stuff. Hopefully, he'll be able to sort it out.

What I'm doing is zip-packaging and bootstrapping. I don't really know much of anything about debs, rpms and such. Apologies if I've mislead anyone.
Jun 02 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, June 03, 2013 00:38:09 Nick Sabalausky wrote:
 On Sun, 02 Jun 2013 16:58:29 -0700
 
 Jonathan M Davis <jmdavisProg gmx.com> wrote:
 On Sunday, June 02, 2013 16:52:03 Walter Bright wrote:
 On 6/2/2013 2:52 PM, Russel Winder wrote:
 Not being able to put symlinks in a zipfile has nothing to do with
 getting deb files and RPM files correct since they should be
 built from tarballs.

Anyone is welcome to help out with the scripts that create deb and rpm files, which are here: https://github.com/D-Programming-Language/installer

Well, Nick is looking into some of that stuff. Hopefully, he'll be able to sort it out.

What I'm doing is zip-packaging and bootstrapping. I don't really know much of anything about debs, rpms and such. Apologies if I've mislead anyone.

Well, it's all part of the same thing. Maybe you won't deal with the rpms or debs explicitly, but what you're doing should enable them to be fixed. It's splitting up the zip file and making it so that we're distributing proper symlinks with *nix which is the key thing. - Jonathan M Davis
Jun 02 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 2 June 2013 at 21:00:57 UTC, Russel Winder wrote:
 On Sun, 2013-06-02 at 13:11 -0700, Jonathan M Davis wrote:
 On Sunday, June 02, 2013 15:47:36 Gary Willoughby wrote:
 I've just run:
 
 sudo ln -s /usr/lib/x86_64-linux-gnu/libphobos2.so
 /usr/lib/x86_64-linux-gnu/libphobos2.so.0.63
 
 for now but i've never had to do that before. Is this a 
 problem
 with the installer?

If you had to do that, then yes.

I didn't have to do that on my Debian Unstable, but maybe I was lucky. However the structure: libphobos2.so the file libphobos2.so.0.63 a symbolic link to libphobos2.so is non-standard and not compliant. The standard structure should be: libphobos2.so.0.63 the file libphobos2.so.0 a symbolic link to libphobos2.so.0.63 libphobos2.so a symbolic link to libphobos2.so.0

And symlink are created automagically by tooling.
Jun 02 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, June 03, 2013 09:20:58 Jacob Carlborg wrote:
 On 2013-06-03 02:15, Jonathan M Davis wrote:
 I don't think that much of anyone around here
 thinks that the zip should contain all of the OSes.

DVM relies on DMD being packaged as a single zip. As long as that is kept, you can do whatever you want.

Except that that's _exactly_ what we want to get rid of. It's ridiculous to put them all in one zip. It just wastes bandwidth, and it doesn't work with symlinks, and now that we're adding shared libraries, we need the *nix packages to have symlinks in them. I understand that DVM currently relies on there being a single zip, but aside from trying not to break DVM, I see zero reason to leave it as a single zip. It's just causing us problems. - Jonathan M Davis
Jun 03 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, June 03, 2013 10:23:10 Jacob Carlborg wrote:
 On 2013-06-03 09:37, Jonathan M Davis wrote:
 Except that that's _exactly_ what we want to get rid of. It's ridiculous
 to
 put them all in one zip. It just wastes bandwidth, and it doesn't work
 with
 symlinks, and now that we're adding shared libraries, we need the *nix
 packages to have symlinks in them.
 
 I understand that DVM currently relies on there being a single zip, but
 aside from trying not to break DVM, I see zero reason to leave it as a
 single zip. It's just causing us problems.

If it was not clear, I would like to keep the cross-platform zip in addition to the platform specific zip/tarball/packages.

Well, part of the problem is that the zip is inherently broken for *nix systems due to the fact that symlinks don't work properly. So, now that we have an so version of Phobos, the zip just isn't going to work properly anymore. And while we aren't really looking to break DVM, AFAIK, there would be no other reason to keep the zip around other than for DVM. And since the zip isn't going to work properly on *nix systems anyway, I'm not sure that keeping it around for DVM really solves much. - Jonathan M Davis
Jun 03 2013
prev sibling next sibling parent Russel Winder <russel winder.org.uk> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, 2013-06-03 at 08:18 +0200, deadalnix wrote:
[=E2=80=A6]
 	libphobos2.so.0.63 the file
 	libphobos2.so.0 a symbolic link to libphobos2.so.0.63
 	libphobos2.so a symbolic link to libphobos2.so.0

And symlink are created automagically by tooling.

For Debian the symlinks are create by the post-install script for a shared library. So yes by the tooling. The moral of this story is that the current mechanisms for creating the DMD deb file are not compliant with the correct tool chain for creating debs. i.e. it is wrong. There should be a Git repository holding the DMD deb meta data and then just use all the git-buildpackage stuff. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Jun 03 2013
prev sibling next sibling parent "Dicebot" <m.strashun gmail.com> writes:
On Monday, 3 June 2013 at 12:19:49 UTC, Russel Winder wrote:
 ...

In most basic form there should be just set of instructions for packagers to conform post-install hook to. Especially when it comes to main repos and stuff is build from SVN/Git whenever it is possible. Providing .deb and .rpm may be convenient sometimes but is not the right way in general.
Jun 03 2013
prev sibling next sibling parent Russel Winder <russel winder.org.uk> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Sun, 2013-06-02 at 17:50 -0700, Walter Bright wrote:
[=E2=80=A6]
 The complaint from Russel was about the .deb file.
=20
 In any case, anyone is free to create a script to build whatever combinat=

 they want, in any format they want, and submit it as a pull request to=

 installer. Nobody has to wait on me to do it.
=20
    https://github.com/D-Programming-Language/installer

The use of a script like this is just totally the wrong way of building debs. I can create the debian directory to replace the script (*) but it requires the tarball of the release to be available. Is there a tarball or only this infamous zipfile? (*) Not immediately but sometime in late August. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Jun 03 2013
prev sibling next sibling parent Russel Winder <russel winder.org.uk> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Sun, 2013-06-02 at 17:47 -0700, Jonathan M Davis wrote:
[=E2=80=A6]
 All you should have to do is set the PATH so that it has dmd in it. Every=

 else should just work.

There is also the issue about whether the compiled stuff has the correct soname in it. I think the correct solution for the debs is to build from a pure source-only tarball. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Jun 03 2013
prev sibling next sibling parent Russel Winder <russel winder.org.uk> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Sun, 2013-06-02 at 18:20 -0700, Jonathan M Davis wrote:
[=E2=80=A6]
 I don't believe that it's not an ldconfig problem. It's the fact that the=

 libphobos2.so and not a libphobos2.so.0.63. It's the exact same problem t=

 the rpm and deb files are having. dmd.conf already makes it so that the l=

 looks in the right place.

Building the deb from a pure source only tarball using the proper Debian deb building toolchain will sort all this out. I suspect the same goes for the Fedora RPM -- use the distribution specific toolchain. It is 12 years since I built any RPMs so I declare myself lacking in knowledge on that front. Debs I can deal with though. All that needs to be known is where is the official source release tarball for 2.063?=20 --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Jun 03 2013
prev sibling next sibling parent Russel Winder <russel winder.org.uk> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Sun, 2013-06-02 at 19:09 -0700, Ellery Newcomer wrote:
[=E2=80=A6]
 hey! the rpm behaves the same way! Maybe building a fedora package on=20
 ubuntu is in fact a terrible idea!

Build a Fedora package on Fedora or don't build it at all. Question Fedora 17, 18, 19=E2=80=A6 I could currently help with 18 but as soon as 19 is released (as opposed to alpha, beta, RC) I will upgrade. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Jun 03 2013
prev sibling next sibling parent Russel Winder <russel winder.org.uk> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, 2013-06-03 at 20:09 +0200, Mike Wey wrote:
[=E2=80=A6]
 Currently the Phobos make file generates: libphobos2.so.0.63.0 and=20
 creates two simlinks libphobos2.so and libphobos2.so.0.63, and sets the=

 soname to libphobos2.s0.0.63.
=20
 The soname currently includes the minor version number because the=20
 compatibility currently breaks every release, when the phobos abi is=20
 more stable it should be removed from the soname.

Uuurrr=E2=80=A6 this isn't what is in the Debian deb file. :-( Also there should be a symbolic link libphobos2.so.0 shouldn't there? --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Jun 03 2013
prev sibling next sibling parent Russel Winder <russel winder.org.uk> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, 2013-06-03 at 10:29 -0700, Walter Bright wrote:
 On 6/2/2013 3:05 PM, Jonathan M Davis wrote:
 It's done entirely by Walter on his own systems, and I suspect that the=


 and rpm files are created from the zip file (though I'm not sure if he =


 those or someone else does). We need to change it so that the process f=


 generating them is automated and does not rely on Walter. Work is being=


 in that area, but it does not appear to be a priority for Walter.

Creating deps and rpms is done from the zip file, and the scripts are her=

=20
     https://github.com/D-Programming-Language/installer
=20
 I know I'm sounding like a broken record, but those scripts are not a mys=

 and anyone can produce pull requests to add to them or fix them.

As I noted earlier, the deb creation scripts in that repository are just fundamentally the wrong way of creating debs. Also as noted earlier I can't do anything about this till August. The result will not be a pull request on the script in the repository, but a new Git repository specifically designed to use the Debian standard deb creation toolchain and a request to delete the script mentioned above from the whole release toolset. Sorry guv but you have to use the right tool for the job, and deb creation should use the deb creation tools. The start point has to be a tarball of the source. If this is not part of the distribution release then we need to agree an officially acceptable process for creating a release source tarball. Thanks. =20 --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Jun 03 2013
prev sibling next sibling parent Russel Winder <russel winder.org.uk> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, 2013-06-03 at 14:32 -0400, Andrei Alexandrescu wrote:
[=E2=80=A6]
 Instead of planning to work on it, one alternative would be to post bits=

 and pieces of information in a bug report and guide others how to do it.=

 Just a thought.

OK. How about trying: http://www.debian.org/doc/manuals/maint-guide/build.en.html http://wiki.debian.org/PackagingWithGit http://lpenz.org/articles/debgit/index.html http://honk.sigxcpu.org/projects/git-buildpackage/manual-html/gbp.html The core of this is having the source tarball and a debian directory with all the appropriate files so that git-buildpackage can do its stuff.=20 I am not sure a bug report helps, what is needed is action and the creation of the Git repository to do the stuff. My problem is that I have Groovy, Python and Scala stuff to do for the next 2 months :-( I hope this helps. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Jun 03 2013
prev sibling next sibling parent Russel Winder <russel winder.org.uk> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, 2013-06-03 at 14:53 -0400, Andrei Alexandrescu wrote:

 Thanks. We've learned with time to organized ourselves really well=20
 around bug reports (both discussions and guides for implementation), so:=

 http://d.puremagic.com/issues/show_bug.cgi?id=3D10256

OK, whatever helps progress this. I'll sign up to the issue and contribute just as soon as I have (space_x, space_y, space_z, time) to focus on this to create something useful =E2=80=93 we like 4-vectors ;-) --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Jun 03 2013
prev sibling next sibling parent Russel Winder <russel winder.org.uk> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, 2013-06-03 at 12:38 -0700, Walter Bright wrote:
 On 6/3/2013 8:59 AM, Russel Winder wrote:
 Is there a tarball or only this infamous zipfile?

I can't believe it is hard to create a tarball from a zipfile.

Actually it can be since tarballs can represent symbolic links whereas it appears zipfiles cannot. However, I think the issue here re deb files is deeper. The solution to the issue as far as generating a deb is concerned is to tag the DMD Git repository so that anyone can generate an official source tarball given the release tag. I appreciate there are licencing issues that means we cannot expect to get DMD into Debian itself, if we can have a Debian toolchain build that targets Debian non-free, we can get DMD into Ubuntu and Mint as well as accesible for Debian users. I believe this to be a good aim. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Jun 03 2013
prev sibling next sibling parent "Dicebot" <m.strashun gmail.com> writes:
On Monday, 3 June 2013 at 19:50:29 UTC, Russel Winder wrote:
 The solution to the issue as far as generating a deb is
 concerned is to tag the DMD Git repository so that anyone can 
 generate
 an official source tarball given the release tag.

It is how this is done in Arch Linux right now. Uses "v2.063.1" tag. Well, not done yet, because there is still 2.062 in main repo, but is incoming.
Jun 03 2013
prev sibling next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Mon, 03 Jun 2013 09:23:35 +0200
Jacob Carlborg <doob me.com> wrote:

 On 2013-06-03 00:25, Nick Sabalausky wrote:
 
 Yea, I'm working on a replacement.

Please keep the existing zip packages as well, we don't want to break DVM :)

I intend to. But I do think it would be good if we then update DVM to use the platform-specific versions, to preserve bandwidth and improve dload/installation speeds. Possibly related to that, I keep meaning to look into updating DVM to install *both* -m32 and -m64 abilities on appropriate platforms (unless you've already beaten me to it?)
Jun 03 2013
prev sibling next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Sun, 02 Jun 2013 22:33:05 -0700
Jonathan M Davis <jmdavisProg gmx.com> wrote:

 On Monday, June 03, 2013 00:38:09 Nick Sabalausky wrote:
 On Sun, 02 Jun 2013 16:58:29 -0700
 
 Jonathan M Davis <jmdavisProg gmx.com> wrote:
 On Sunday, June 02, 2013 16:52:03 Walter Bright wrote:
 On 6/2/2013 2:52 PM, Russel Winder wrote:
 Not being able to put symlinks in a zipfile has nothing to do
 with getting deb files and RPM files correct since they
 should be built from tarballs.

Anyone is welcome to help out with the scripts that create deb and rpm files, which are here: https://github.com/D-Programming-Language/installer

Well, Nick is looking into some of that stuff. Hopefully, he'll be able to sort it out.

What I'm doing is zip-packaging and bootstrapping. I don't really know much of anything about debs, rpms and such. Apologies if I've mislead anyone.

Well, it's all part of the same thing. Maybe you won't deal with the rpms or debs explicitly, but what you're doing should enable them to be fixed. It's splitting up the zip file and making it so that we're distributing proper symlinks with *nix which is the key thing.

Ok. It will, naturally, be generating a non-zipped DMD release directory (just so it *can* then be zipped up), so maybe that could help the other packagers.
Jun 03 2013
prev sibling next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Mon, 03 Jun 2013 19:30:16 +0100
Russel Winder <russel winder.org.uk> wrote:
 
 The start point has to be a tarball of the source. If this is not part
 of the distribution release then we need to agree an officially
 acceptable process for creating a release source tarball.
 

My tool will do this.
Jun 03 2013
prev sibling next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Sun, 02 Jun 2013 22:33:03 -0400
Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> wrote:
 
 I quoted all of the above because I so much agree with it. Packaging
 all OSs in a zip is really disingenuous and now it's come to a head.
 Let's fix this for 2.064.
 

FWIW, I fully intend to have my corner of this ready by then.
Jun 03 2013
prev sibling next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Mon, 3 Jun 2013 17:12:31 -0400
Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> wrote:

 On Mon, 03 Jun 2013 19:30:16 +0100
 Russel Winder <russel winder.org.uk> wrote:
 
 The start point has to be a tarball of the source. If this is not
 part of the distribution release then we need to agree an officially
 acceptable process for creating a release source tarball.
 

My tool will do this.

Speaking of which: Are there any special requirements I should be aware of? In particular, it would be very helpful to have a canonical list of anything/everything that should be different from what you get by taking one of the current release ZIPs we have today and extracting it. Also does ".tar.gz" vs ".tar.bz2" matter? I understand that the latter (if I spelled it right) is newer and better compression, but any reason that it might *need* to be one or the other?
Jun 03 2013
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Monday, 3 June 2013 at 21:18:54 UTC, Nick Sabalausky wrote:
 Also does ".tar.gz"

I prefer tar.gz because bzip is SLOW. There's some space savings but meh. gzip is slightly more likely to be already installed on a box too, though bzip is like 99% so that doesn't matter a lot.
Jun 03 2013
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
I have two requests if we start changing the zip, since I'm 
pretty happy with it how it is!

1) keep the folder structure the same as it is now. So the 
windows zip still has a windows folder in it with the bin and lib 
stuff. The only difference from the current thing is you remove 
the linux etc folders.

This way we don't have to update install/use scripts that assume 
these paths exist.


2) still keep the zips for all the OSes available. I don't use a 
package manager and don't care for installers. I still want a 
dmd-windows.zip and dmd-linux.zip and so on. (or 
dmd-linux.tar.gz, whatever, as long as the contents are the same)
Jun 03 2013
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 03 Jun 2013 17:21:30 -0400, Adam D. Ruppe  
<destructionator gmail.com> wrote:

 On Monday, 3 June 2013 at 21:18:54 UTC, Nick Sabalausky wrote:
 Also does ".tar.gz"

I prefer tar.gz because bzip is SLOW. There's some space savings but meh.

+1 when developing an imaging system for hard drives using cpio+compression, bzip never finished running (well, I didn't wait long enough probably), plus it doesn't stream at all. -Steve
Jun 03 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, June 03, 2013 11:42:53 Jacob Carlborg wrote:
 On 2013-06-03 10:34, Jonathan M Davis wrote:
 Well, part of the problem is that the zip is inherently broken for *nix
 systems due to the fact that symlinks don't work properly. So, now that we
 have an so version of Phobos, the zip just isn't going to work properly
 anymore. And while we aren't really looking to break DVM, AFAIK, there
 would be no other reason to keep the zip around other than for DVM. And
 since the zip isn't going to work properly on *nix systems anyway, I'm
 not sure that keeping it around for DVM really solves much.

*nix is really not correct to say. Currently only Linux 64bit supports shared libraries. It's not like it's broken on all platforms, just one. Sure, it will break once we get support for share libraries on additional platforms. If static linking is default nothing is broken.

By default sure, but people can now use Phobos as a shared library if they so choose, and at some point, it will probably change to be the default. And the symlinks _are_ broken thanks to zip. And the problems will only get worse as we add more platforms. There's really no reason to stick with zip files other than avoiding breaking something like DVM, and the zip files are inherently broken at this point anyway. And that breakage will only get worse as we add more shared library support.
 Which format can we use for these platform specific packages?

We'd probably use tar.gz or tar.bz2, which are incredibly standard. Mac OS X might want something different (I don't use it, so I don't know), but it's based on the BSDs, so it probably uses tar.gz like the rest (I'm sure that it at least supports it). zip is too Windows-specific.
 For which of these are there existing bindings or libraries?

I'm sure that there are C bindings for tar and gz, but I've never used them, and worst case, it's trivial to use std.process.system to decompress the package. - Jonathan M Davis
Jun 03 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, June 03, 2013 16:59:16 Russel Winder wrote:
 Is there a tarball or only this infamous zipfile?

Only the zip file, which is the main reason why all of the symlinks are broken. But Nick is working on changing it so that we generate separate packages for each OS, and Brad Roberts has some preliminary pull requests towards creating install targets to generate the packages. - Jonathan M Davis
Jun 03 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, June 03, 2013 23:27:00 Adam D. Ruppe wrote:
 2) still keep the zips for all the OSes available. I don't use a
 package manager and don't care for installers. I still want a
 dmd-windows.zip and dmd-linux.zip and so on. (or
 dmd-linux.tar.gz, whatever, as long as the contents are the same)

Well, they wouldn't be zip for stuff other than Windows (they'd likely be tar.gz or tar.bz2 for the other OSes), but their internals could be kept the same as far as directory structure goes. - Jonathan m Davis
Jun 03 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, June 03, 2013 14:13:26 Walter Bright wrote:
 On 6/3/2013 12:50 PM, Russel Winder wrote:
 On Mon, 2013-06-03 at 12:38 -0700, Walter Bright wrote:
 On 6/3/2013 8:59 AM, Russel Winder wrote:
 Is there a tarball or only this infamous zipfile?

I can't believe it is hard to create a tarball from a zipfile.

Actually it can be since tarballs can represent symbolic links whereas it appears zipfiles cannot.

1. unzip the zip file 2. add symbolic links 3. create tarball

But why create it as a zip file in the first place? Just create the tarball correctly when it's built and don't create a zip file at all. Clearly, people other than you are going to be sorting this out, but we shouldn't even be creating a zip file for the non-Windows OSes in the first place. The symlinks should be created as part of the build and tar-ed from there. - Jonathan M Davis
Jun 03 2013
prev sibling next sibling parent Russel Winder <russel winder.org.uk> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Mon, 2013-06-03 at 14:13 -0700, Walter Bright wrote:
[=E2=80=A6]
=20
 1. unzip the zip file
 2. add symbolic links
 3. create tarball

Not the right way of doing it. [=E2=80=A6]
 It's already tagged with v2.063

1. clone repository 2. checkout to tag 3. generate tar.gz file Definitely a better way. The symlinks is only part of the issue re the Debian deb file.=20 --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Jun 03 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday, June 04, 2013 09:16:12 Jacob Carlborg wrote:
 On 2013-06-03 19:29, Walter Bright wrote:
 Creating deps and rpms is done from the zip file, and the scripts are here:
     https://github.com/D-Programming-Language/installer

The Mac OS X installer is built from the zip file as well. If we change format or anything similar we need to update the script that builds the installer.

If we change the zip, we need to look at _all_ of the various installers/packages which are generated from it, since it looks like _everything_ generated from the zip file at this point. - Jonathan M Davis
Jun 04 2013
prev sibling next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Tue, 04 Jun 2013 08:57:49 +0200
Jacob Carlborg <doob me.com> wrote:

 On 2013-06-03 23:08, Nick Sabalausky wrote:
 
 I intend to. But I do think it would be good if we then update DVM
 to use the platform-specific versions, to preserve bandwidth and
 improve dload/installation speeds.

I would like to rewrite the whole tool. Then we can fix all these things. Haven't taken the time to do it yet.

Just curious: What sort of direction did you have in mind for a rewrite? Something about the general architecture of it you'd do differently?
 Possibly related to that, I keep meaning to look into updating DVM
 to install *both* -m32 and -m64 abilities on appropriate platforms
 (unless you've already beaten me to it?)

No, I haven't done that. See above. BTW, DVM is not really working on Windows XP, or rather the installed compilers. DVM installs the compilers in a path which contains spaces. The compiler and/or the linker isn't happy with that.

I was using an XP box as my main machine until about a year ago. It always worked fine for me as I recall. The only "path with spaces" problem I had was when using DVM to build DMD using a version of DMC that was installed by DVM. That lead to a "paths with spaces" problem. I wasn't able to figure that out, although I suspected it may have had something to do with either DMD's makefiles or DMC's make.
Jun 04 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday, June 04, 2013 00:52:27 Walter Bright wrote:
 On 6/3/2013 5:34 PM, Jonathan M Davis wrote:
 But why create it as a zip file in the first place?

Because creating tarballs on Windows is a big problem, not the least of which you can't create symlinks on Windows, so even if you found a working Windows tar program you still haven't solved the problem.

Except that they shouldn't even be being created on Windows in the first place. I understand that this is how you've been doing it, but it would make a lot more sense for a build machine like the auto tester to be generating the tarballs as part of the build. Then they're being built locally to the OSes that they're going to be used on, and the process is properly automatable. We can even have nightly dev releases that way. I believe that this is what Brad Roberts started working on with some recent pull requests towards creating make install targets. Ideally, none of the official build and packaging process would rely on your local setup. - Jonathan M Davis
Jun 04 2013
prev sibling next sibling parent "Joakim" <joakim airpost.net> writes:
On Monday, 3 June 2013 at 21:18:54 UTC, Nick Sabalausky wrote:
 Also does ".tar.gz" vs ".tar.bz2" matter? I understand that the 
 latter
 (if I spelled it right) is newer and better compression, but 
 any reason
 that it might *need* to be one or the other?

format for open source distros: http://en.wikipedia.org/wiki/Xz#Uses It usually compresses the tightest and decompresses the fastest, while beating bzip2 on compression time.
Jun 05 2013
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, June 05, 2013 19:13:46 Joakim wrote:
 On Monday, 3 June 2013 at 21:18:54 UTC, Nick Sabalausky wrote:
 Also does ".tar.gz" vs ".tar.bz2" matter? I understand that the
 latter
 (if I spelled it right) is newer and better compression, but
 any reason
 that it might *need* to be one or the other?

I'd look at xz, it's increasingly becoming the standard packaging format for open source distros: http://en.wikipedia.org/wiki/Xz#Uses It usually compresses the tightest and decompresses the fastest, while beating bzip2 on compression time.

In my experience, xz has way worse compression time than bzip2, and on smaller files, it actually compresses worse. Where xz shines are large files. It definitely beats out bzip2 by a fair bit there. But as it loses at small files (which distro packages usually are), it seems very off to me that Arch Linux switched to used xz from gzip. It would have made for more sense to switch to bzip2. I we split the dmd zip, I question that it would be large enough for xz to compress better than bzip2. And I don't think that xz is supported as well in general as gzip or bzip2. So, I doubt that switching to xz is a good idea. - Jonathan M Davis
Jun 05 2013
prev sibling parent "David Nadlinger" <code klickverbot.at> writes:
On Wednesday, 5 June 2013 at 17:24:56 UTC, Jonathan M Davis wrote:
 In my experience, xz has way worse compression time than bzip2, 
 and on smaller
 files, it actually compresses worse. Where xz shines are large 
 files. It
 definitely beats out bzip2 by a fair bit there. But as it loses 
 at small files
 (which distro packages usually are), it seems very off to me 
 that Arch Linux
 switched to used xz from gzip. It would have made for more 
 sense to switch to
 bzip2.

If I were you, I'd assume that the Arch Linux devs have done their homework, and xz actually compresses a typical package better than bzip2 does. And indeed, when I compared different compression formats to figure out how to distribute the LDC binary packages, I found that xz compresses our packages quite a bit better than bzip2 does, while being faster at *de*compressing, which is what matters for users. As far as compression speed goes, I actually find it to be mostly irrelevant for packaging binary releases: I don't care whether the archive creation part of my scripts takes 5 or 50 seconds to run, uploading the archives probably takes longer anyway, unless I'm on the university internet connection. David
Jun 05 2013