www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Practical Problems with distribution D projects

reply Assaf Gordon <agordon wi.mit.edu> writes:
Hello,

I working on a small program, which I'd like to make publicly available and
easy to distribute,install and use.

I'm trying to figure out the best way to distribute it.
Writing D code is a lot of fun, but it looks like preparing and packaging it is
not so much...

So far, I'm stuck at:

1.
Packaging for source-based distribution (ie a "tarball"):
I couldn't find a working autotools template (only few old discussions),
and so far it seems the best way is writing my own makefile (which I have).

My concern is that the parameters are so different between dmd/gdc/ldc that I
basically force the users to choose one.

I'm also not sure whether it will work on windows or not (I'm developing on
linux). For windows users, would a custom,hand-written make file just work?

Although "rdmd" is terrific, I would really prefer not to require it (since it
is 'dmd' specific and not free in the FOSS sense).


2.
How complicated it for potential users of the program to install a D compiler?

Based on my humble experience:
dmd: fine, as long as the downloadable binary version works. I tested only on
multiple debians and ubuntus. But it's not free, and it requires root
privileges.
gdc: only source code available, compiling is a bit tricky. After compilation,
it works.
ldc: the pre-compiled version doesn't work on old Ubuntu (10.4) or new Debian
(7). The source code version is messy to compile and I stopped after couple of
tries.

So, it seems, that if a random person (not unix expert) wants to compiler my
program, he/she will not have an easy time doing so.


3.
Distributing Binaries:
Since I already assume source distribution is not friendly, I consider also
making the pre-compiled binaries available for download.
But it looks like it's only possible to compile dynamically-linked binaries.

The only leads I've found are two old bugs:
http://d.puremagic.com/issues/show_bug.cgi?id=6952
http://d.puremagic.com/issues/show_bug.cgi?id=4376

And I couldn't find any better solution (on dlang.org or google).
---

All the above, makes me somewhat reconsider whether D is an optimal choice:
even if my program is useful in theory, it's of no use if very few people can
actually run it.

Are there any suggestions how to deal with these issues ?

Thanks,
  -gordon
Feb 24 2014
next sibling parent "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Tuesday, 25 February 2014 at 01:21:03 UTC, Assaf Gordon wrote:
 1.
 Packaging for source-based distribution (ie a "tarball"):
 I couldn't find a working autotools template (only few old 
 discussions),
 and so far it seems the best way is writing my own makefile 
 (which I have).

 My concern is that the parameters are so different between 
 dmd/gdc/ldc that I basically force the users to choose one.

GDC and LDC have gdmd/ldmd respectively, which wrap DMD-style command line arguments over the GCC/Clang-style arguments that `gdc` and `ldc` accept.
 I'm also not sure whether it will work on windows or not (I'm 
 developing on linux). For windows users, would a 
 custom,hand-written make file just work?

That depends almost entirely on the shell commands used in the target recipes. Also, it's easy to create reliance on a certain flavour of `make`, such as by using GNU-make-specific features; it's not a big problem though because GNU make is relatively easy to get on Windows through MinGW.
 Although "rdmd" is terrific, I would really prefer not to 
 require it (since it is 'dmd' specific and not free in the FOSS 
 sense).

`rdmd` itself should be free software, and supports `gdmd` and `ldmd` through the --compiler flag. DMD itself is indeed not entirely free software.
 2.
 How complicated it for potential users of the program to 
 install a D compiler?

 Based on my humble experience:
 dmd: fine, as long as the downloadable binary version works. I 
 tested only on multiple debians and ubuntus. But it's not free, 
 and it requires root privileges.
 gdc: only source code available, compiling is a bit tricky. 
 After compilation, it works.
 ldc: the pre-compiled version doesn't work on old Ubuntu (10.4) 
 or new Debian (7). The source code version is messy to compile 
 and I stopped after couple of tries.

 So, it seems, that if a random person (not unix expert) wants 
 to compiler my program, he/she will not have an easy time doing 
 so.

Maybe we should have a generic Linux binary package that statically links all dependencies, which would then be usable on a wide variety of Linux systems without requiring root privileges. But are there really no packages of `gdc` and `ldc` in various package repositories? I haven't used D on Linux in a while, but I find that hard to believe.
 3.
 Distributing Binaries:
 Since I already assume source distribution is not friendly, I 
 consider also making the pre-compiled binaries available for 
 download.
 But it looks like it's only possible to compile 
 dynamically-linked binaries.

I'm pretty sure the default still uses static linking, even with recent Linux releases? Either way, I'm sure there's a way to get the old behaviour. Dynamic linking of the runtime is a relatively recent ability.
Feb 24 2014
prev sibling next sibling parent "FreeSlave" <freeslave93 gmail.com> writes:
You can use hack in Makefile to find out whether the current 
platform is Windows or not. Check for SystemRoot environment 
variable in this way:

ifdef SystemRoot
	SYSTEM=Windows
else ifdef SYSTEMROOT
	SYSTEM=Windows
else
	SYSTEM=Other
endif

In non-Windows case use Unix uname command to determine current 
platform (Linux, Darwin, etc).

If you don't want to force user to install MSys or Cygwin, use 
Windows-shell commands instead Unix ones. Example:

ifeq ($(SYSTEM), Windows)
	RM=cmd //C del
	CP=cmd //C copy
else
	RM=rm -f
	CP=cp
endif

So you can achieve some flexibility here.
Sometimes developers write separate makefiles named windows.mak 
and posix.mak, but those ones still can share common parts via 
include directive.


Also you can write small build-system for your project in D 
itself. Derelict uses this approach. It also allows to ensure 
that D compiler was installed and works properly.
Feb 24 2014
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Tuesday, 25 February 2014 at 01:21:03 UTC, Assaf Gordon wrote:
 dmd: fine, as long as the downloadable binary version works. I 
 tested only on multiple debians and ubuntus. But it's not free, 
 and it requires root privileges.

dmd does not require root. The install instructions about copying stuff into /etc and such are ridiculously overcomplicated - dmd actually just works* if you run it straight out of the zip as any regular user in any location. * if the binary works on your distro, sometimes there's a version mismatch forcing you to build it yourself. That's not a hard thing to do either, the source is included and you can just make it as a regular user, but an extra step doesn't do 'just works'.
 But it looks like it's only possible to compile 
 dynamically-linked binaries.

This is more of a Linux problem than a D one. But most D programs will work normally, dynamically linking the C library *usually* works and that's all a D app really needs. If you want more binary compatibility btw I'd say stick to building Windows exes, they almost always just work, even on Linux with wine.
Feb 24 2014
prev sibling next sibling parent reply Shammah Chancellor <anonymous coward.com> writes:
On 2014-02-25 00:50:55 +0000, Assaf Gordon said:

 Hello,
 
 I working on a small program, which I'd like to make publicly available 
 and easy to distribute,install and use.
 
 I'm trying to figure out the best way to distribute it.
 Writing D code is a lot of fun, but it looks like preparing and 
 packaging it is not so much...
 
 So far, I'm stuck at:
 
 1.
 Packaging for source-based distribution (ie a "tarball"):
 I couldn't find a working autotools template (only few old discussions),
 and so far it seems the best way is writing my own makefile (which I have).
 
 My concern is that the parameters are so different between dmd/gdc/ldc 
 that I basically force the users to choose one.
 
 I'm also not sure whether it will work on windows or not (I'm 
 developing on linux). For windows users, would a custom,hand-written 
 make file just work?
 
 Although "rdmd" is terrific, I would really prefer not to require it 
 (since it is 'dmd' specific and not free in the FOSS sense).
 
 
 2.
 How complicated it for potential users of the program to install a D compiler?
 
 Based on my humble experience:
 dmd: fine, as long as the downloadable binary version works. I tested 
 only on multiple debians and ubuntus. But it's not free, and it 
 requires root privileges.
 gdc: only source code available, compiling is a bit tricky. After 
 compilation, it works.
 ldc: the pre-compiled version doesn't work on old Ubuntu (10.4) or new 
 Debian (7). The source code version is messy to compile and I stopped 
 after couple of tries.
 
 So, it seems, that if a random person (not unix expert) wants to 
 compiler my program, he/she will not have an easy time doing so.
 
 
 3.
 Distributing Binaries:
 Since I already assume source distribution is not friendly, I consider 
 also making the pre-compiled binaries available for download.
 But it looks like it's only possible to compile dynamically-linked binaries.
 
 The only leads I've found are two old bugs:
 http://d.puremagic.com/issues/show_bug.cgi?id=6952
 http://d.puremagic.com/issues/show_bug.cgi?id=4376
 
 And I couldn't find any better solution (on dlang.org or google).
 ---
 
 All the above, makes me somewhat reconsider whether D is an optimal 
 choice: even if my program is useful in theory, it's of no use if very 
 few people can actually run it.
 
 Are there any suggestions how to deal with these issues ?
 
 Thanks,
   -gordon

Have you looked at having dub be your autotool?
Feb 24 2014
next sibling parent Mike Parker <aldacron gmail.com> writes:
On 2/25/2014 11:52 PM, Assaf Gordon wrote:
 On 02/25/2014 12:09 AM, Artem Tarasov wrote:

 On 02/24/2014 09:09 PM, Adam D. Ruppe wrote:> On Tuesday, 25 February
 2014 at 01:21:03 UTC, Assaf Gordon wrote:
 dmd does not require root. The install instructions about copying
 stuff into /etc and such are ridiculously overcomplicated - dmd
 actually just works* if you run it straight out of the zip as any
 regular user in any location.

From a non-expert unix POV - Either you download and install the package (deb/rpm) which is easy but requires root, or you download and compile the sources from the source zip - which is complicated.

You don't need to compile anything. There are precompiled binaries for 32- and 64-bit Linux in the zip. I'm no Linux expert (far, far from it). I'm not even what you would call a casual user. But when I do need to boot into Linux for one thing or another (which is rare), I unzip the latest binaries into my user directory and away I go. The only thing I've ever configured is putting the appropriate bin directory on the path, but that's it.
Feb 25 2014
prev sibling parent Shammah Chancellor <anonymous coward.com> writes:
On 2014-02-25 16:21:24 +0000, Adam D. Ruppe said:

 On Tuesday, 25 February 2014 at 16:03:46 UTC, Artem Tarasov wrote:
 /usr/bin/ld: cannot find -l:libphobos2.a

Weird. That might just be a config problem.

The dmd.conf file has the wrong options set for the path. I ran into this problem on MacOS X as well. -S.
Feb 25 2014
prev sibling next sibling parent Artem Tarasov <lomereiter gmail.com> writes:
--089e0158a8e6eb5fab04f3341711
Content-Type: text/plain; charset=ISO-8859-1

On Tue, Feb 25, 2014 at 7:16 AM, Shammah Chancellor <anonymous coward.com>wrote:

 Have you looked at having dub be your autotool?

The complaint is about difficulties with installation of a compiler and rdmd, and you suggest to bring in yet another dependency? My experience is the same as described, I keep a VM on Amazon EC2 with CentOS 5, which contains installed LDC2, just so that the compiled binaries run on any Linux x86_64 machine no matter how old glibc version is. --089e0158a8e6eb5fab04f3341711 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On T= ue, Feb 25, 2014 at 7:16 AM, Shammah Chancellor <span dir=3D"ltr">&lt;<a hr= ef=3D"mailto:anonymous coward.com" target=3D"_blank">anonymous coward.com</= a>&gt;</span> wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex">Have you looked at having dub be your autoto= ol?<br></blockquote><div><br></div><div>The complaint is about difficulties= with installation of a compiler and rdmd, and you suggest to bring in yet = another dependency?<br> <br></div><div>My experience is the same as described, I keep a VM on Amazo= n EC2 with CentOS 5, which contains installed LDC2, just so that the compil= ed binaries run on any Linux x86_64 machine no matter how old glibc version= is.<br> </div></div><br></div></div> --089e0158a8e6eb5fab04f3341711--
Feb 24 2014
prev sibling next sibling parent "Jesse Phillips" <Jesse.K.Phillips+D gmail.com> writes:
On Tuesday, 25 February 2014 at 01:21:03 UTC, Assaf Gordon wrote:
 Hello,

 I working on a small program, which I'd like to make publicly 
 available and easy to distribute,install and use.

If you want to do distribution to developers, dub[1] is the way to go. If you want to distribute to desktop users, binary packages is the way to go. The issues with static linking are related to C libraries. If you develop in Linux you generally don't statically link. All your D code is statically linked, work has been done to support dynamic linking with Phobos, but for now that isn't default. 1. http://code.dlang.org/
Feb 24 2014
prev sibling next sibling parent =?UTF-8?B?IlRow6lv?= Bueno" <munrek gmx.com> writes:
On Tuesday, 25 February 2014 at 01:21:03 UTC, Assaf Gordon wrote:
 I couldn't find a working autotools template (only few old 
 discussions),
 and so far it seems the best way is writing my own makefile 
 (which I have).

https://github.com/bioinfornatics/MakefileForD
Feb 24 2014
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
For building dub and/or rdmd are de-facto standard for any D 
developers. You should just start from source tarball and proceed 
from there to finding maintainers in specific distributives, 
possibly being the one for distro or two. Then conform packaging 
rules there.

Single Linux-wide binary distribution is mostly a myth and dmd 
shows it by its own.
Feb 25 2014
prev sibling next sibling parent Assaf Gordon <agordon wi.mit.edu> writes:
On 02/25/2014 12:09 AM, Artem Tarasov wrote:
 On Tue, Feb 25, 2014 at 7:16 AM, Shammah Chancellor
 <anonymous coward.com <mailto:anonymous coward.com>> wrote:
 Have you looked at having dub be your autotool?

and rdmd, and you suggest to bring in yet another dependency? My experience is the same as described, I keep a VM on Amazon EC2 with CentOS 5, which contains installed LDC2, just so that the compiled binaries run on any Linux x86_64 machine no matter how old glibc version is.

Thanks! I was beginning to think I'm alone in this problem :) I'm getting the impression that D is either windows-focused (which is a mostly homogenous system everywhere and easy to work with), or unix-experts/developers-focused (where people feel naturally comfortable with downloading and compiling and fidgeting with programs). ----- To address some of the other responses: On 02/24/2014 09:09 PM, Adam D. Ruppe wrote:> On Tuesday, 25 February 2014 at 01:21:03 UTC, Assaf Gordon wrote:
 dmd does not require root. The install instructions about copying
 stuff into /etc and such are ridiculously overcomplicated - dmd
 actually just works* if you run it straight out of the zip as any
 regular user in any location.

From a non-expert unix POV - Either you download and install the package (deb/rpm) which is easy but requires root, or you download and compile the sources from the source zip - which is complicated. We can argue about what "complicated" means, obviously the members of this forums are mostly experts, so they don't think that compiling from source is tricky. But if you download the zip, and try to look at it from a novice's POV: The README.txt is not helpful. There's no "configure" or "CMakefile" or "Makefile" - which are the most common ways to "just make it work" for open source projects. Going into "src", there's no Makefile, and the "readme.txt" is again not helpful (talks about the license, not about the installation). If you're have some experience, you can guess that "posix.mak" is the relevant file, and indeed "make -f posix.mak" builds dmd correctly. Then you have to repeat the same for "./src/dmd", "./src/druntime" and "./src/phobos" - and then what? update $PATH, $LD_LIBRARY_PATH ? would that be enough ? All these might be easy for you - but not for a casual Linux user (which is my intended target audience). ----- On 02/25/2014 12:48 AM, Jesse Phillips wrote:
 If you want to distribute to desktop users, binary packages is the
 way to go. The issues with static linking are related to C
 libraries. If you develop in Linux you generally don't statically
 link.

You "generally don't statically link" because you can: 1. Readily and easily compile from source code on the local machine, or 2. Have binary distribution that perfectly matches your system (in the case of debs/rpms from an official repository). With D - I have none of these. And when distribution just an executable to run on an unknown linux machine, the best way is a statically linked binary.
 All your D code is statically linked, work has been done to support
 dynamic linking with Phobos, but for now that isn't default.

But I'm not asking for dynamic linking... I'm asking for static linking that works... ---- On 02/25/2014 08:13 AM, Dicebot wrote:
 For building dub and/or rdmd are de-facto standard for any D
 developers. You should just start from source tarball and proceed
 from there to finding maintainers in specific distributives, possibly
 being the one for distro or two. Then conform packaging rules there.

My target audience is not D developers - it is unix casual users (not even developers). I want to make it the easiest for them to use my program. If it's requires more than the standard "configure && make && make install" (or the cmake equivalent), then many of them will not do it.
 Single Linux-wide binary distribution is mostly a myth and dmd shows
 it by its own.

Not sure what the "myth" is - but a statically-linked binaries work well (if your program is self-contained and properly compiled), and there are many examples of it. ---- On the upside: I now see that "dub" provides a good Debian repository, and a HomeBrew formula (and by proxy, a LinuxBrew formula) - so perhaps that is the way to go. Still, if I may suggest: There have been recent threads in the mailing list about "the future of D" and creating "Killer Apps" and making D more popular - From my (biased) POV, one big step would be to make D easily useable on more platforms - so the casual (non-windows) users could also experiment with it, and developers would have easier time to distribute software written in D. Also, as long as DMD is not free, providing easy way to install LDC/GDC is highly desired. Thanks for reading so far, -gordon
Feb 25 2014
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Tuesday, 25 February 2014 at 14:51:48 UTC, Assaf Gordon wrote:
 My target audience is not D developers - it is unix casual 
 users (not even developers).
 I want to make it the easiest for them to use my program.
 If it's requires more than the standard "configure && make && 
 make install" (or the cmake equivalent), then many of them will 
 not do it.

Then push it into distro repositories. You are NOT helping by recommending them to do "make install", quite the contrary, harming whole ecosystem. Casual users should not be even aware that "make install" exists. Being one of package maintainers myself I really hate this kind of "helping" attitude, it causes only trouble.
 Single Linux-wide binary distribution is mostly a myth and dmd 
 shows
 it by its own.

Not sure what the "myth" is - but a statically-linked binaries work well (if your program is self-contained and properly compiled), and there are many examples of it.

Except you never have truly statically linked binaries, there is always at least glibc. You can do it for small set of popular distributives, especially when conforming to http://en.wikipedia.org/wiki/Linux_Standard_Base but it simply does not work in general. Please stop distributing software for Linux with Windows attitude.
Feb 25 2014
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Tuesday, 25 February 2014 at 14:51:48 UTC, Assaf Gordon wrote:
 Either you download and install the package (deb/rpm) which is 
 easy but requires root,
 or you download and compile the sources from the source zip -

OR you download and run the binary zip. The instructions aren't very good, but this is really easy. download dmd.version-you-want.zip unzip dmd.zip run dmd2/linux/bin32/dmd yourfile.d that's all you have to do. You don't have to build anything (most the time), you don't have to mess with PATH or LD_LIBRARY_PATH, you don't have to copy anything into a system folder. You just run the program straight out of the zip.
Feb 25 2014
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Tuesday, 25 February 2014 at 14:51:48 UTC, Assaf Gordon wrote:
 Also, as long as DMD is not free, providing easy way to install 
 LDC/GDC is highly desired.

btw, fresh LDC has recently made it to Debian sid : https://packages.debian.org/sid/ldc , thanks to Konstantinos Margaritis
Feb 25 2014
prev sibling next sibling parent "Francesco Cattoglio" <francesco.cattoglio gmail.com> writes:
On Tuesday, 25 February 2014 at 14:51:48 UTC, Assaf Gordon wrote:
 Also, as long as DMD is not free, providing easy way to install 
 LDC/GDC is highly desired.

On my Ubuntu laptop (standard repos as far as I remember) I have to "apt-get install gdc" to install it. I don't understand what you meant with "easy way to install". Are you talking about getting it from sources or what?
Feb 25 2014
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Tuesday, 25 February 2014 at 15:12:08 UTC, Francesco Cattoglio 
wrote:
 On my Ubuntu laptop (standard repos as far as I remember) I 
 have to "apt-get install gdc" to install it. I don't understand 
 what you meant with "easy way to install". Are you talking 
 about getting it from sources or what?

It is a rather old gdc, I can't recommend to use it ;)
Feb 25 2014
prev sibling next sibling parent "Francesco Cattoglio" <francesco.cattoglio gmail.com> writes:
On Tuesday, 25 February 2014 at 15:18:06 UTC, Dicebot wrote:
 On Tuesday, 25 February 2014 at 15:12:08 UTC, Francesco 
 Cattoglio wrote:
 On my Ubuntu laptop (standard repos as far as I remember) I 
 have to "apt-get install gdc" to install it. I don't 
 understand what you meant with "easy way to install". Are you 
 talking about getting it from sources or what?

It is a rather old gdc, I can't recommend to use it ;)

since gdc --version output is 4.8.1, what version would it be? like 2.060 or more like 2.052?
Feb 25 2014
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Tuesday, 25 February 2014 at 15:24:19 UTC, Francesco Cattoglio 
wrote:
 since gdc --version output is 4.8.1, what version would it be? 
 like 2.060 or more like 2.052?

Hm, ones I see in package list are for 4.6 - are you using some sort of testing repo? 4.8.* can possibly be pretty new, you can check fronend version by compiling test program with "pragma(msg, __VERSION__)"
Feb 25 2014
prev sibling next sibling parent "Francesco Cattoglio" <francesco.cattoglio gmail.com> writes:
On Tuesday, 25 February 2014 at 15:31:19 UTC, Dicebot wrote:
 On Tuesday, 25 February 2014 at 15:24:19 UTC, Francesco 
 Cattoglio wrote:
 since gdc --version output is 4.8.1, what version would it be? 
 like 2.060 or more like 2.052?

Hm, ones I see in package list are for 4.6 - are you using some sort of testing repo? 4.8.* can possibly be pretty new, you can check fronend version by compiling test program with "pragma(msg, __VERSION__)"

2062L! 2.062 is decently recent I'd say ;) Perhaps is from some launchpad repo, but nothing too much trascendental. Anyway, my point was that GDC and LDC are easily available under most used distros, given a bit of patience. I mean, we can probably package them faster than the distribution updates them (I'm referring to Ubuntu 6-months cycle in particular)
Feb 25 2014
prev sibling next sibling parent Artem Tarasov <lomereiter gmail.com> writes:
--001a11c11d12ddd30d04f33d3a99
Content-Type: text/plain; charset=ISO-8859-1

On Tue, Feb 25, 2014 at 7:06 PM, Adam D. Ruppe <destructionator gmail.com>wrote:

 OR you download and run the binary zip. The instructions aren't very good,
 but this is really easy.

Yes, that works great on my Mint laptop and, I guess, the majority of modern distros. But let's try your advice on CentOS 5 (and please don't ask me why some people still use it -- I have no idea, too): $ uname -a Linux ip-10-182-148-229 2.6.18-274.7.1.el5xen #1 SMP Thu Oct 20 17:06:34 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux $ wget http://downloads.dlang.org/releases/2014/dmd.2.065.0.zip -q $ unzip -q dmd.2.065.0.zip $ cat test.d import std.stdio; void main() { writeln("Hello, old Linux system!"); } $ ./dmd2/linux/bin64/dmd -L-L/usr/lib64 test.d /usr/bin/ld: cannot find -l:libphobos2.a collect2: ld returned 1 exit status --- errorlevel 1 O-ops!.. --001a11c11d12ddd30d04f33d3a99 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On T= ue, Feb 25, 2014 at 7:06 PM, Adam D. Ruppe <span dir=3D"ltr">&lt;<a href=3D= "mailto:destructionator gmail.com" target=3D"_blank">destructionator gmail.= com</a>&gt;</span> wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left:1px solid rgb(204,204,204);padding-left:1ex"><div class=3D""><br></div=

od, but this is really easy.<br></blockquote><div><br></div><div>Yes, that = works great on my Mint laptop and, I guess, the majority of modern distros.= <br> But let&#39;s try your advice on CentOS 5 (and please don&#39;t ask me why = some people still use it -- I have no idea, too):<br></div><div>$ uname -a<= br>Linux ip-10-182-148-229 2.6.18-274.7.1.el5xen #1 SMP Thu Oct 20 17:06:34= EDT 2011 x86_64 x86_64 x86_64 GNU/Linux<br> </div><div>$ wget <a href=3D"http://downloads.dlang.org/releases/2014/dmd.2= .065.0.zip">http://downloads.dlang.org/releases/2014/dmd.2.065.0.zip</a> -q= <br>$ unzip -q dmd.2.065.0.zip<br></div><div>$ cat test.d<br>import std.std= io; void main() { writeln(&quot;Hello, old Linux system!&quot;); }<br> $ ./dmd2/linux/bin64/dmd -L-L/usr/lib64 test.d<br>/usr/bin/ld: cannot find = -l:libphobos2.a<br>collect2: ld returned 1 exit status<br>--- errorlevel 1<= br><br></div><div>O-ops!..<br></div></div></div></div> --001a11c11d12ddd30d04f33d3a99--
Feb 25 2014
prev sibling next sibling parent Assaf Gordon <agordon wi.mit.edu> writes:
On 02/25/2014 10:06 AM, Dicebot wrote:
 On Tuesday, 25 February 2014 at 14:51:48 UTC, Assaf Gordon wrote:
 My target audience is not D developers - it is unix casual users
 (not even developers). I want to make it the easiest for them to
 use my program. If it's requires more than the standard "configure
 && make && make install" (or the cmake equivalent), then many of
 them will not do it.

Then push it into distro repositories. You are NOT helping by recommending them to do "make install", quite the contrary, harming whole ecosystem. Casual users should not be even aware that "make install" exists. Being one of package maintainers myself I really hate this kind of "helping" attitude, it causes only trouble.

I'm sorry it makes you upset - I can see how frustrating it is for a package maintainer. But I think "harming the ecosystem" is a bit of a hyperbole - in an ideal world where Linux Distros contain every piece of software (and up to date version), there is indeed very little need to install from source. But there are several real-world constraints, at least in the environment I work in: 1. Software needs to run on variety of servers and distributions, some are very old (e.g. CentOS 5.4 or Ubuntu 10.04). Upgrading is not an option for these environments. 2. Software needs to be installed without root - to a user directory (i.e. using "configure --prefix" or similar). 3. Software is updated too frequently, or is too new, or (sadly) too experimental (code-wise) for inclusion in a stable distribution package - but it still needs to be used. 4. It takes time for a software to be properly packaged, and users want to install it now. Also - it will not be available in the "stable" branch of the distribution. In addition, some distribution have extra requirements. For example, Debian frowns upon compiling with static libraries when a shared one is available. If there is already a package in official Debian which is written in D, it would be very helpful to see an example of the packaging process.
 Single Linux-wide binary distribution is mostly a myth and dmd
 shows it by its own.

Not sure what the "myth" is - but a statically-linked binaries work well (if your program is self-contained and properly compiled), and there are many examples of it.

Except you never have truly statically linked binaries, there is always at least glibc.

I'm not sure I understand this statement. If I link with "gcc -static" and my program is properly built, the resulting binary does not require any library, not even glibc. It uses only Linux ABI syscalls. Unless I'm mistaken?
 Please stop distributing software for Linux with Windows attitude.

Not sure what the "windows attitude" is, since I haven't developed software for windows in 12 years... My first choice would be to have a project in D which is easily compilable from source code. Either with "make" or "dub" or "configure" or anything else. Once I have that, packaging (for a linux distribution) is straight forward, and so is adding it to HomeBrew/LinuxBrew. At the moment, I don't think it's possible (in the level of "easiness" comparable with C/autotools-based projects). The alternative (for linux) is distributing a binary program, and for that, a static-binary is much more portable than a dynamic one. Having GDC and LDC in Debian is an excellent start - I was not aware of them, and it is certainly the right direction. But those are still not available in the "stable" branches, and so, not easily usable. In the future this will definitely change. -Gordon
Feb 25 2014
prev sibling next sibling parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Tuesday, 25 February 2014 at 15:06:14 UTC, Dicebot wrote:
 On Tuesday, 25 February 2014 at 14:51:48 UTC, Assaf Gordon 
 wrote:
 My target audience is not D developers - it is unix casual 
 users (not even developers).
 I want to make it the easiest for them to use my program.
 If it's requires more than the standard "configure && make && 
 make install" (or the cmake equivalent), then many of them 
 will not do it.

Then push it into distro repositories. You are NOT helping by recommending them to do "make install", quite the contrary, harming whole ecosystem. Casual users should not be even aware that "make install" exists. Being one of package maintainers myself I really hate this kind of "helping" attitude, it causes only trouble.

Personally I have to compile all my software by hand at work, because I don't have root/sudo access. Although I agree that in general one should try to work within the package managers as much as possible, many (most?) distributions don't have even half decent capabilities for source-based distribution within the package manager, especially when working without root.
Feb 25 2014
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Tuesday, 25 February 2014 at 16:04:56 UTC, Assaf Gordon wrote:
 Not sure what the "windows attitude" is, since I haven't 
 developed software for windows in 12 years...

The Windows attitude is expecting things to actually work for the end user. It rarely actually works that way on linux.
Feb 25 2014
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Tuesday, 25 February 2014 at 16:03:46 UTC, Artem Tarasov wrote:
 /usr/bin/ld: cannot find -l:libphobos2.a

Weird. That might just be a config problem.
Feb 25 2014
prev sibling next sibling parent Assaf Gordon <agordon wi.mit.edu> writes:
On 02/25/2014 11:17 AM, Adam D. Ruppe wrote:
 On Tuesday, 25 February 2014 at 16:04:56 UTC, Assaf Gordon wrote:
 Not sure what the "windows attitude" is, since I haven't developed software
for windows in 12 years...

The Windows attitude is expecting things to actually work for the end user. It rarely actually works that way on linux.

Side-stepping the flame-war potential of this statement :) , I agree that Windows is homogenous enough, and have so many "built-in" runtime DLLs that you can expect all systems to be almost identical in terms of what's available. It makes distribution very easy for users (and developers). It takes more effort to build portable software, and in windows you rarely have to worry about it. Once you do build a portable software properly, it "works" for the end user as well.
Feb 25 2014
prev sibling next sibling parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Tuesday, 25 February 2014 at 16:17:13 UTC, Adam D. Ruppe wrote:
 On Tuesday, 25 February 2014 at 16:04:56 UTC, Assaf Gordon 
 wrote:
 Not sure what the "windows attitude" is, since I haven't 
 developed software for windows in 12 years...

The Windows attitude is expecting things to actually work for the end user. It rarely actually works that way on linux.

If by "works for the end user" you mean that you go to a (different for every app) website and download a binary blob, give it administrator permissions, go through a graphical installer that by default installs 3 toolbars, an anti-virus scanner demo and changes your homepage, then downloads a whole other program that does the actual installation... then yes, I guess it does :) Honestly, my experience of linux in terms of getting new applications has always been a highlight for me ever since I first started using it years ago. apt-get install firefox emerge firefox pacman -S firefox yast -i firefox yum install firefox How can any of those be said to not work for the end user? It's really very very simple and that's without even discussing the GUIs for the point-and-clickers.
Feb 25 2014
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Tuesday, 25 February 2014 at 16:04:56 UTC, Assaf Gordon wrote:
 But I think "harming the ecosystem" is a bit of a hyperbole - 
 in an ideal world where Linux Distros contain every piece of 
 software (and up to date version), there is indeed very little 
 need to install from source.

Apologise for overly harsh reaction, it is rather frequent source of frustration for me which makes reaction unacceptably nervous. That "ideal" state is pretty much in your hands. There are as many packages in repos as people are willing to put - not like there is some important boss who decides what needs to be packages. I know it is not very convenient with Debian because of how conservative it is but in Arch just asking in #archlinux freenode is likely to result in kind soul willing to help with packaging questions. Or even easier, just contacting me as it is my direct responsibility to maintain all D-related projects in Arch [community] ;) As general rule of a thumb - if you application is useful, there always be someone willing to maintain package for it, assuming build process is not utterly insane.
 But there are several real-world constraints, at least in the 
 environment I work in:

 1. Software needs to run on variety of servers and 
 distributions, some are very old (e.g. CentOS 5.4 or Ubuntu 
 10.04).
 Upgrading is not an option for these environments.

 2. Software needs to be installed without root - to a user 
 directory (i.e. using "configure --prefix" or similar).

 3. Software is updated too frequently, or is too new, or 
 (sadly) too experimental (code-wise) for inclusion in a stable 
 distribution package - but it still needs to be used.

 4. It takes time for a software to be properly packaged, and 
 users want to install it now. Also - it will not be available 
 in the "stable" branch of the distribution.

I see, your requirements do not match use case of a typical desktop application. However, if you are fine with full static linking why not simply distribute binary tarballs for set of systems you want to support? It will put burden of having proper D compilation environment away from the end user. It is worth noting though that for any normal application requiring package manager access (not full root) is expected and desired. Fast update issue can be solved by providing own upstream repository (i.e. launchpad)
 If there is already a package in official Debian which is 
 written in D, it would be very helpful to see an example of the 
 packaging process.

Unfortunately all Debian/Ubuntu D packages I am aware about are found in non-official repo maintained by Jordi : http://d-apt.sourceforge.net/ Debian is probably hardest distro out there to deal with this kind of stuff.
 I'm not sure I understand this statement.
 If I link with "gcc -static" and my program is properly built, 
 the resulting binary does not require any library, not even 
 glibc.
 It uses only Linux ABI syscalls.
 Unless I'm mistaken?

I think it will work but I'd recommend to provide an option to build system to chose both dynamically and statically linked binary to simplify life for those who _may_ want to package it later.
 Not sure what the "windows attitude" is, since I haven't 
 developed software for windows in 12 years...

I was referring to extremely common attempts to provide all-in-one "installers" ignoring normal practices of target systems that often happen when those used to Windows world start porting their stuff to Linux. And/or thinking about all distros as single target "Linux" system.
 My first choice would be to have a project in D which is easily 
 compilable from source code.
 Either with "make" or "dub" or "configure" or anything else.
 Once I have that, packaging (for a linux distribution) is 
 straight forward, and so is adding it to HomeBrew/LinuxBrew.

dub is the way to go in general, but it is not expected to be ever installed on user systems (well, unless it is Gentoo :)) - it is to be used by you or anyone willing to (re)package your application.
 The alternative (for linux) is distributing a binary program, 
 and for that, a static-binary is much more portable than a 
 dynamic one.

Within your requirements it may be a best choice. In general though I'd say that perfect portability is a flaw rather than merit. When speaking about desktop applications, for exampe, as a user of any operating systems I usually want software that is adapted to conform practices of this specific system and hardly appreciate how portable it is.
Feb 25 2014
prev sibling next sibling parent Assaf Gordon <agordon wi.mit.edu> writes:
On 02/25/2014 05:41 PM, Dicebot wrote:
 I see, your requirements do not match use case of a typical desktop
 application.

This is the root cause of our different POVs - I should have explained it more clearly. My users are people who have access to Linux machines which they do not control. I can expect them to be able to handle the standard/easy steps of compiling software, but not much more. This is the case in many universities, were the computer "cluster" is made of whatever Linux the IT department used. You won't get root access on those servers. And many time the distribution is very old and not updated on purpose (for the sake of stability). To make matters more complicated, many additional programs are installed in places like "/usr/local/" and "/opt/". It might make you cringe you as a maintainer (it makes me cringe, as a root-less user) ... but that's life :) Now, I have to be realistic: My program will be useful to some degree, but not a life-changing Killer-App. If it's easy to use and install, people will use it (and by proxy, perhaps use D as well, if they want to hack the code). If it's too complicated, they'll give up and look for a different solution. So my goal to begin with, was to find the easiest way to distribute my D program to unix platforms.
However, if you are fine with full static linking why
 not simply distribute binary tarballs for set of systems you want to
 support? It will put burden of having proper D compilation
 environment away from the end user.

We've come full circle :) In my first message in this thread, I wrote (item #3), that unless I'm missing something, there is no way to build a statically linked binary with D on Linux. If there is a way, I would love to know about it. I would gladly spend the time to create VMs for each common system setup and package a static binary for it.
 I know it is not very convenient with Debian because of how
 conservative it is but in Arch just asking in #archlinux   freenode
 is likely to result in kind soul willing to help with packaging
 questions. Or even easier, just contacting me as it is my direct
 responsibility to maintain all D-related projects in Arch [community]
 ;)

I got one package into Debian, and it took some effort. But it was worth it, because I learned alot, and it did make my program better (distribution wise). Arch and Gentoo might be friendlier (and HomeBrew/LinuxBrew even easier), but the people I'm aiming for aren't using Arch or Gentoo... I do have high hopes for HomeBrew/LinuxBrew, as "brew install dmd" and "brew install dub" seems to work nicely on some linux systems. Best, -gordon
Feb 26 2014
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Tuesday, 25 February 2014 at 17:04:11 UTC, John Colvin wrote:
 If by "works for the end user" you mean that you go to a 
 (different for every app) website

I always thought that was the sane way to do it, different website for every app, though I loathe and despise chasing down library dependencies. static linking ftw
 other program that does the actual installation... then yes, I 
 guess it does :)

Eh, my standard is dmd: get a zip and run the program with a reasonable expectation that it will just work. I don't like installers either.
 How can any of those be said to not work for the end user?

yum install php gets php 5.1, but my other app needs 5.2 something something epel key whatever yum update php got php 5.2! .... but php-gd isn't in that repo, so it still is 5.1 and doesn't work. wget http://php.net/downloads/whatever/php-5.2.tar.gz tar zxf blah cd blah ./configure missing library X yum install libx-devel ./configure missing library Y yum install liby-devel no such package *spend half an hour searching teh web* yum install libwtf-devel # for some reason liby is in there.... ./configure missing library Z yum install libz-devel no such package wget http://libz.org/downloads/libz.tar.gz tar zxf cd ./configure missing library You get the idea. finally get that working cd php ./configure library Z is version 3.1 but php requries 2.8 here we go again *six hours later* OH NO I FORGOT TO ADD --enable-gd TO CONFIGURE :-( :-( :-( here we go again ./configure --enable-gd --enable-whatever-else .... <snip> sudo make install $ php Segmentation fault AAaaaAaaAaaaAaaRrrrReGGGHHhhhhGH. OK, I admit, I'm combining two different horror stories into one here, a php one and a recurring hassle I've had with gtk, but still, the point stands.
 It's really very very simple and that's without even
 discussing the GUIs for the point-and-clickers.

When it works, it is great, when it doesn't, you're in a world of pain. Getting into the repo can be a pain too, so can compiling everything. On Windows, you just compile on your box and it tends to work on others. On Linux, you need a dozen different distros installed to build the compatible binaries to package...
Feb 26 2014
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Wednesday, 26 February 2014 at 17:08:38 UTC, Assaf Gordon 
wrote:
 In my first message in this thread, I wrote (item #3), that 
 unless I'm missing something, there is no way to build a 
 statically linked binary with D on Linux.

They really aren't far from it out of the box: $ dmd hellod.d $ ldd hellod linux-gate.so.1 => (0xffffe000) libpthread.so.0 => /lib/libpthread.so.0 (0xf76db000) libm.so.6 => /lib/libm.so.6 (0xf76b5000) librt.so.1 => /lib/librt.so.1 (0xf76ac000) libc.so.6 => /lib/libc.so.6 (0xf7549000) /lib/ld-linux.so.2 (0xf7730000) It isn't fully static, but those libraries are all pretty basic things and generally available on any linux install. The biggest problem is probably the libc version not matching on older installations. One option might be to package those .so files right with your executable. A poor man's static link. Anyway, let's try the -static switch we can use to do statically linked libc on some programs. $ dmd hellod.d -c $ gcc hellod.o -o hellod -m32 -L/home/me/d/dmd2/linux/bin32/../lib32 -Xlinker --export-dynamic -l:libphobos2.a -lpthread -lm -lrt -static undefined reference to `__tls_get_addr' BTW if you run "dmd -v yourfile.d" the last line of output is the linker command line it runs (via gcc). Then you can copy/paste that and tweak the arguments. Here, I added -static. However, that is *not* a D problem per se, that's a libc function. Maybe my system just doesn't have the right stuff installed to build a static threaded app.
 I would gladly spend the time to create VMs for each common 
 system setup and package a static binary for it.

If you're doing that, you don't need to do anything special, since the vm will link the appropriate libc version anyway, so the default build should work. I betcha if I sent you my binary from my slackware box over to your computer it would work even if you are a different distro.
Feb 26 2014
prev sibling next sibling parent Assaf Gordon <agordon wi.mit.edu> writes:
On 02/26/2014 12:34 PM, Adam D. Ruppe wrote:
 On Wednesday, 26 February 2014 at 17:08:38 UTC, Assaf Gordon wrote:
 In my first message in this thread, I wrote (item #3), that unless I'm missing
something, there is no way to build a statically linked binary with D on Linux.

They really aren't far from it out of the box: $ dmd hellod.d $ ldd hellod linux-gate.so.1 => (0xffffe000) libpthread.so.0 => /lib/libpthread.so.0 (0xf76db000) libm.so.6 => /lib/libm.so.6 (0xf76b5000) librt.so.1 => /lib/librt.so.1 (0xf76ac000) libc.so.6 => /lib/libc.so.6 (0xf7549000) /lib/ld-linux.so.2 (0xf7730000) It isn't fully static, but those libraries are all pretty basic things and generally available on any linux install. The biggest problem is probably the libc version not matching on older installations.

This might seem "generally available" on linux, but it's not. It might be "generally available" when you assume you're dealing with "recent" "desktop" versions of linux. But it's not at all when dealing with servers (as Artem already mentioned before: http://forum.dlang.org/post/mailman.290.1393304990.6445.digital ars-d puremagic.com ) . As a quick example, try to download and run the pre-compiled binaries of "ldc2" from the dlang.org website on a Stable Debian - they will fail with incompatible libc versions.
 One option might be to package those .so files right with your executable. A
poor man's static link.

Now that's what I'd call "windows attitude" - package all dependencies together, in what I think was termed "DLL hell" :) I think that low-level libraries (e.g. ld-linux, libc, linux-gate, librt, libpthread) are tightly coupled to the system (and the kernel), you really don't want to run incompatible ones on an arbitrary linux system.
 Anyway, let's try the -static switch we can use to do statically linked libc
on some programs.

 $ dmd hellod.d -c
 $ gcc hellod.o -o hellod -m32 -L/home/me/d/dmd2/linux/bin32/../lib32 -Xlinker
--export-dynamic -l:libphobos2.a -lpthread -lm -lrt -static

 undefined reference to `__tls_get_addr'


 BTW if you run "dmd -v yourfile.d" the last line of output is the linker
command line it runs (via gcc). Then you can copy/paste that and tweak the
arguments. Here, I added -static.


 However, that is *not* a D problem per se, that's a libc function. Maybe my
system just doesn't have the right stuff installed to build a static threaded
app.

I'd argue the opposite: it is a D problem (or libphobos2/druntime problem, which is a major part of D). The missing symbol "__tls_get_addr" is referenced from the D function getTLSRange() in "druntime/src/rt/sections_linux.d". And the way druntime uses it is apparently not compatible with static linking (I know very little about this low-level stuff).
 I betcha if I sent you my binary from my slackware box over to your computer
it would work even if you are a different distro.

You'd be surprised what kind of old versions of linuxes are out there. I have tried it with C-based projects, and based on my humble experience, anything except a real static binary will eventually cause problems to users.
Feb 26 2014
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Wednesday, 26 February 2014 at 17:34:45 UTC, Adam D. Ruppe 
wrote:
 Anyway, let's try the -static switch we can use to do 
 statically linked libc on some programs.

 $ dmd hellod.d -c
 $ gcc hellod.o -o hellod -m32 
 -L/home/me/d/dmd2/linux/bin32/../lib32 -Xlinker 
 --export-dynamic -l:libphobos2.a -lpthread -lm -lrt -static

 undefined reference to `__tls_get_addr'

Huh, I was sure this works. Unpleasant surprise. Have filed a bug report : https://d.puremagic.com/issues/show_bug.cgi?id=12268 ( I don't have time to properly answer holy war topics, probably later :) )
Feb 26 2014
prev sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Wednesday, 26 February 2014 at 18:52:55 UTC, Assaf Gordon 
wrote:
 It might be "generally available" when you assume you're 
 dealing with "recent" "desktop" versions of linux.

I've had good luck with them on my servers too but I haven't used many of the distros. But I have had the same libc version problems you see too.
 I think that low-level libraries (e.g. ld-linux, libc, 
 linux-gate, librt, libpthread) are tightly coupled to the 
 system (and the kernel), you really don't want to run 
 incompatible ones on an arbitrary linux system.

If this were true, static linking would be useless! Static linking is basically just copying the library functions into your binary instead of leaving a reference to the library.
 I'd argue the opposite: it is a D problem (or 
 libphobos2/druntime problem, which is a major part of D).

I'm not sure because searching the web for people having this problem with C++ suggests the fix is to update the libc on their development computer. So that might work with us too.... but then again if it breaks on your computer too who knows.
 I have tried it with C-based projects, and based on my humble 
 experience, anything except a real static binary will 
 eventually cause problems to users.

I agree, this is why I prefer distributing Windows exes :) but if you know your target computers it might not be a big problem, just compile with a libc close enough to them.
Feb 26 2014