www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - Dub support was added to Meson

reply Filipe =?UTF-8?B?TGHDrW5z?= <lains archlinux.org> writes:
Hello,

Dub support was finally merged to the Meson's upstream.
For the ones that don't know, Meson[1] is a fast build system 
that uses ninja[2] as a backend. Until now it was impossible to 
use dependencies via the Dub and many many packages only 
supported Dub. Now you can import dependencies that already exist 
in Dub's local repo. You still need to fetch and build them 
though.

First, you need to fetch and build the dependency.

   dub fetch vibe-d
   dub build vibe-d

Then you just need to specify Dub as the dependency method.

   vibed_dep = dependency('vibe-d', method: 'dub')

Together with this, a 'dlang' module was also merged. You can use 
it to automatically generate a dub.json file.

Here's an example.



path/to/where/to/install/the/dub/file,

   dlang.generate_dub_file(meson.project_name().to_lower(), 
meson.source_root(),
                           authors: 'Foo',
                           description: 'Bar',
                           copyright: 'Copyright © 2018, Foo',
                           license: 'MIT',
                           sourceFiles: 'foobar.d',
                           targetType: 'executable',
                           dependencies: vibed_dep
   )

If you have any issue with this, contact me at 
lains archlinux.org.

Thank you,
Filipe Laíns

[1] https://github.com/mesonbuild/meson
[2] https://github.com/ninja-build/ninja
Aug 04 2018
next sibling parent reply Russel Winder <russel winder.org.uk> writes:
Great stuff.

I have failed to date finishing Dub support in SCons, I shall have to put m=
ore
effort in now. :-)

The single biggest problem with D and Meson is that unit-threaded doesn't w=
ork
with the way Meson works. Which is why I introduced the ProgramAllAtOnce
builder into SCons.

If there is a plan to try and make D/unit-threaded/Meson all work together,
count me in as a tester at the very least.

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Dr Russel Winder      t: +44 20 7585 2200
41 Buckmaster Road    m: +44 7770 465 077
London SW11 1EN, UK   w: www.russel.org.uk
Aug 04 2018
parent Filipe =?UTF-8?B?TGHDrW5z?= <lains archlinux.org> writes:
On Saturday, 4 August 2018 at 19:01:50 UTC, Russel Winder wrote:

 Great stuff.

 I have failed to date finishing Dub support in SCons, I shall 
 have to put more effort in now. :-)

 The single biggest problem with D and Meson is that 
 unit-threaded doesn't work with the way Meson works. Which is 
 why I introduced the ProgramAllAtOnce builder into SCons.

 If there is a plan to try and make D/unit-threaded/Meson all 
 work together, count me in as a tester at the very least.


I'll keep an eye out for that. Maybe I'll try to implement 
something if my free time allows it :)

PS.: Sorry for the grammar errors in the original post (mainly 
the "the Dub" :facepalm:), I rewrote some of the sentences and I 
didn't properly read the whole post again.

Thanks,
Filipe Laíns
Aug 04 2018
prev sibling next sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Saturday, 4 August 2018 at 16:07:45 UTC, Filipe Laíns wrote:

 Hello,

 Dub support was finally merged to the Meson's upstream.
 For the ones that don't know, Meson[1] is a fast build system 
 that uses ninja[2] as a backend. Until now it was impossible to 
 use dependencies via the Dub and many many packages only 
 supported Dub. Now you can import dependencies that already 
 exist in Dub's local repo. You still need to fetch and build 
 them though.

 [...]


How does it track dependencies given that ninja needs 
functionality akin to gcc's to do that? Or does it always compile 
everything if any file changes?

reggae has to wrap dmd to be able to use ninja for anything. I 
can't see any other way but to add the necessary dependencies 
generation as you compile to dmd itself.
Aug 07 2018
parent reply Mike Wey <mike-wey example.com> writes:
On 07-08-18 22:33, Atila Neves wrote:

 How does it track dependencies given that ninja needs functionality akin 
 to gcc's to do that? Or does it always compile everything if any file 
 changes?


It currently only tracks dependencies when using gdc, for dmd and ldc 
dmd pull 6961[1] would have to be merged so that support can be extended 
to the other compilers.

So when using ldc or dmd you will currently have to call 'ninja clean && 
ninja' to compile everything.

[1] https://github.com/dlang/dmd/pull/6961

-- 
Mike Wey
Aug 08 2018
parent Atila Neves <atila.neves gmail.com> writes:
On Wednesday, 8 August 2018 at 17:26:39 UTC, Mike Wey wrote:

 On 07-08-18 22:33, Atila Neves wrote:

 How does it track dependencies given that ninja needs 
 functionality akin to gcc's to do that? Or does it always 
 compile everything if any file changes?


 It currently only tracks dependencies when using gdc, for dmd 
 and ldc dmd pull 6961[1] would have to be merged so that 
 support can be extended to the other compilers.

 So when using ldc or dmd you will currently have to call 'ninja 
 clean && ninja' to compile everything.

 [1] https://github.com/dlang/dmd/pull/6961


Makes sense, thanks for the info.
Aug 08 2018
prev sibling parent reply Russel Winder <russel winder.org.uk> writes:
On Sat, 2018-08-04 at 16:07 +0000, Filipe La=C3=ADns via Digitalmars-d-anno=
unce
wrote:

 Hello,
=20
 Dub support was finally merged to the Meson's upstream.
 For the ones that don't know, Meson[1] is a fast build system=20
 that uses ninja[2] as a backend. Until now it was impossible to=20
 use dependencies via the Dub and many many packages only=20
 supported Dub. Now you can import dependencies that already exist=20
 in Dub's local repo. You still need to fetch and build them=20
 though.


Sadly not in Meson 0.47.1 is a new release due soon?


 First, you need to fetch and build the dependency.
=20
    dub fetch vibe-d
    dub build vibe-d


This feels a little unsatisfactory. Dub (and soon SCons I hope) builds hand=
le
getting the dependency as well as building it and thence linking the projec=
t
to it. Meson both depends on the dub executable and yet isn't doing the
getting, it is assuming already got.

Of course Meson relies on all dependencies being present at build
specification time, so nothing inconsistent, it just seems a wee bit
incomplete.

I haven't finished reading the Meson source to find out which Dub built
archive is used in the Meson build but if it is the one in
~/.dub/packages/<package>-<version>/<package> then stuff can go wrong since=
 it
is the result of the last build. In order to get the build with the specifi=
c
compiler on the specific platform you have to delve into
~/.dub/packages/<package>-<version>/<package>/.dub/ and know the Dub naming
strategy. For SCons I am having problems with this because of the extra stu=
ff
in the directory name.


 Then you just need to specify Dub as the dependency method.
=20
    vibed_dep =3D dependency('vibe-d', method: 'dub')


I have been getting projects such as Unit-Threaded, DInotify, TinyEndian, a=
nd
D_YAML to have Meson builds so as to create shared libraries prior to using
pkg-config to handle D project being dependent on these libraries. It's
working nicely (for me anyway) except Dub uses GitHub tags for versioning a=
nd
Meson requires version numbers in the meson.build file. This duplication is
getting quite annoying for all concerned. As is the Meson requirement to li=
st
all source files, rather than have a convention over configuration approach=
 as
Dub does =E2=80=93 SCons has a different approach but still avoids explicit=
 source
lists. I can get round it with Meson, but I am informed it is naughty and
should not be done.

These builds create shared objects and allow for dynamic linking. Using Dub
built archives means static linking. Unless Meson somehow manages to get
shared objects out of the Dub build.

Unless Dub can generate shared objects, having the dependencies build local=
ly
and creating pkg-config files actually seems a better way that accessing Du=
b
builds.

But I may be missing something!=20
=20
[=E2=80=A6]
--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Dr Russel Winder      t: +44 20 7585 2200
41 Buckmaster Road    m: +44 7770 465 077
London SW11 1EN, UK   w: www.russel.org.uk
Aug 10 2018
next sibling parent reply Matthias Klumpp <mak debian.org> writes:
On Friday, 10 August 2018 at 14:38:10 UTC, Russel Winder wrote:

 On Sat, 2018-08-04 at 16:07 +0000, Filipe Laíns via 
 Digitalmars-d-announce wrote:

 Hello,
 
 Dub support was finally merged to the Meson's upstream.
 For the ones that don't know, Meson[1] is a fast build system
 that uses ninja[2] as a backend. Until now it was impossible to
 use dependencies via the Dub and many many packages only
 supported Dub. Now you can import dependencies that already 
 exist
 in Dub's local repo. You still need to fetch and build them
 though.


 Sadly not in Meson 0.47.1 is a new release due soon?


There is no fixed release schedule, but from experience, this 
might take a few months.


 First, you need to fetch and build the dependency.
 
    dub fetch vibe-d
    dub build vibe-d


 This feels a little unsatisfactory. Dub (and soon SCons I hope) 
 builds handle getting the dependency as well as building it and 
 thence linking the project to it. Meson both depends on the dub 
 executable and yet isn't doing the getting, it is assuming 
 already got.

 Of course Meson relies on all dependencies being present at 
 build specification time, so nothing inconsistent, it just 
 seems a wee bit incomplete.


This is intentional, see this comment from Jussi: 
https://github.com/mesonbuild/meson/pull/3592#issuecomment-390421754

Unfortunately, this makes the dub feature almost useless for 
Linux distro integration.
However, I hope we might be able to implement reading dub.json 
files that are locally installed and incorporate D sources from 
the dependency into the regular Meson build process.

This would mean that Meson's dlang plugin would have to implement 
quite a bit of dub and dub.json parsing, which is some effort.
The benefit would be the creation of a near-seamless bridge 
between the dub and Meson worlds (you'd only have to locally 
install a dub package), that is also useful for Linux 
distribution packaging.


 [...]
 I have been getting projects such as Unit-Threaded, DInotify, 
 TinyEndian, and D_YAML to have Meson builds so as to create 
 shared libraries prior to using pkg-config to handle D project 
 being dependent on these libraries. It's working nicely (for me 
 anyway) except Dub uses GitHub tags for versioning and Meson 
 requires version numbers in the meson.build file. This 
 duplication is getting quite annoying for all concerned. As is 
 the Meson requirement to list all source files, rather than 
 have a convention over configuration approach as Dub does – 
 SCons has a different approach but still avoids explicit source 
 lists. I can get round it with Meson, but I am informed it is 
 naughty and should not be done.


Jup, deterministic source lists are useful for split-building 
stuff efficiently. And globbing sources is possible with Meson, 
but highly discouraged (and even looks ugly).


 These builds create shared objects and allow for dynamic 
 linking. Using Dub built archives means static linking. Unless 
 Meson somehow manages to get shared objects out of the Dub 
 build.

 Unless Dub can generate shared objects, having the dependencies 
 build locally and creating pkg-config files actually seems a 
 better way that accessing Dub builds.
 [...]


At the moment, I think for a final project, adding Meson files to 
dependencies still is the better way. However, for quickly 
testing out things and for building something locally, the new 
dub support is quite valuable.
Also, we can expand it in future, this is a first step :-)
Aug 13 2018
parent Russel Winder <russel winder.org.uk> writes:
On Mon, 2018-08-13 at 14:33 +0000, Matthias Klumpp via Digitalmars-d-announ=
ce
wrote:
[=E2=80=A6]

 This is intentional, see this comment from Jussi:=20
 https://github.com/mesonbuild/meson/pull/3592#issuecomment-390421754


I am not entirely sure I agree with everything there, but he is BDFL of the
project so his view wins.=20


 Unfortunately, this makes the dub feature almost useless for=20
 Linux distro integration.
 However, I hope we might be able to implement reading dub.json=20
 files that are locally installed and incorporate D sources from=20
 the dependency into the regular Meson build process.


I am rapidly heading this route for SCons: instead of using Dub to create t=
he
libraries just use it to get the sources and then compile the sources as a
sub-project. However I still prefer having a Meson build for the library an=
d
building that way. If however there is a way of using dub.sdl or dub.json t=
hat
would be interesting.


 This would mean that Meson's dlang plugin would have to implement=20
 quite a bit of dub and dub.json parsing, which is some effort.
 The benefit would be the creation of a near-seamless bridge=20
 between the dub and Meson worlds (you'd only have to locally=20
 install a dub package), that is also useful for Linux=20
 distribution packaging.


Perhaps there is a route whereby there can be a separate Python package to =
do
this that both Meson and SCons can use? Otherwise it risks there being
implementation of the same code for Meson and SCons separately.

[=E2=80=A6]

 Jup, deterministic source lists are useful for split-building=20
 stuff efficiently. And globbing sources is possible with Meson,=20
 but highly discouraged (and even looks ugly).


I am not entirely convinced explicit manual lists are better, nor are they
required for consistent builds. If the source files structure changes you h=
ave
to create a new build anyway.

What is the ugly globbing way you know =E2=80=93 I'm wondering if it better=
 than the
hack I have.=20

[=E2=80=A6]

 At the moment, I think for a final project, adding Meson files to=20
 dependencies still is the better way. However, for quickly=20
 testing out things and for building something locally, the new=20
 dub support is quite valuable.
 Also, we can expand it in future, this is a first step :-)


I get the feeling Meson Rust/Cargo support is easier and better than D/Dub
support, even though D seems to get more publicity in the Meson documentati=
on.

I think a really good way forward for Meson (and analogously SCons) would b=
e
to have the Dub dub.{json,sdl} file usable to create (and deploy if using
shared objects) a Meson (Scons) build of the dependency and then building t=
he
project based on the created dependencies.

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Dr Russel Winder      t: +44 20 7585 2200
41 Buckmaster Road    m: +44 7770 465 077
London SW11 1EN, UK   w: www.russel.org.uk
Aug 13 2018
prev sibling parent reply Filipe =?UTF-8?B?TGHDrW5z?= <lains archlinux.org> writes:
On Friday, 10 August 2018 at 14:38:10 UTC, Russel Winder wrote:

 First, you need to fetch and build the dependency.
 
    dub fetch vibe-d
    dub build vibe-d


 This feels a little unsatisfactory. Dub (and soon SCons I hope) 
 builds handle getting the dependency as well as building it and 
 thence linking the project to it. Meson both depends on the dub 
 executable and yet isn't doing the getting, it is assuming 
 already got.


Yeah, Meson is a build system, not a package manager. This 
problem can be solved by simply creating a script that fetches 
and builds de dependencies. When creating a script, you might 
also want to compile the dependencies against the system's 
default D compiler. This way, you won't have any issues.


 I haven't finished reading the Meson source to find out which 
 Dub built archive is used in the Meson build but if it is the 
 one in ~/.dub/packages/<package>-<version>/<package> then stuff 
 can go wrong since it is the result of the last build. In order 
 to get the build with the specific compiler on the specific 
 platform you have to delve into 
 ~/.dub/packages/<package>-<version>/<package>/.dub/ and know 
 the Dub naming strategy. For SCons I am having problems with 
 this because of the extra stuff in the directory name.


It uses 'dub describe' to find the files, this also allows us to 
check the compiler. Meson won't even let you try to compile the 
project if there's an compiler mismatch.


 I have been getting projects such as Unit-Threaded, DInotify, 
 TinyEndian, and D_YAML to have Meson builds so as to create 
 shared libraries prior to using pkg-config to handle D project 
 being dependent on these libraries. It's working nicely (for me 
 anyway) except Dub uses GitHub tags for versioning and Meson 
 requires version numbers in the meson.build file. This 
 duplication is getting quite annoying for all concerned. As is 
 the Meson requirement to list all source files, rather than 
 have a convention over configuration approach as Dub does – 
 SCons has a different approach but still avoids explicit source 
 lists. I can get round it with Meson, but I am informed it is 
 naughty and should not be done.


I mean, you can do this but as you said, it's not recommended.


 These builds create shared objects and allow for dynamic 
 linking. Using Dub built archives means static linking. Unless 
 Meson somehow manages to get shared objects out of the Dub 
 build.

 Unless Dub can generate shared objects, having the dependencies 
 build locally and creating pkg-config files actually seems a 
 better way that accessing Dub builds.


Dub is implemented this way for a reason. We really *shouldn't* 
use dynamic libraries, at least for now.
Aug 14 2018
parent reply Russel Winder <russel winder.org.uk> writes:
On Tue, 2018-08-14 at 17:51 +0000, Filipe La=C3=ADns via Digitalmars-d-anno=
unce
wrote:
[=E2=80=A6]

=20
 Yeah, Meson is a build system, not a package manager. This=20
 problem can be solved by simply creating a script that fetches=20
 and builds de dependencies. When creating a script, you might=20
 also want to compile the dependencies against the system's=20
 default D compiler. This way, you won't have any issues.


That's the battle isn't it. Should the system that builds be the system tha=
t
also manages the packages for the build (cf. Dub, Cargo, Go, Mason), or sho=
uld
the two be kept separate (cf. CMake, Meson, SCons, Make).

The debate has been had on the D mailing lists a number of times and at gre=
at
length. But in the end D, Rust, and Go are going with a single system for
everything.

Building for Debian/Fedora/etc. can be seen as separate from building a
project independent of platform. Whilst build-only clearly works for
Debian/Fedora/etc. builds, and possibly for big corporate builds, for small
projects the all-in-one approach is clearly winning the battle for hearts a=
nd
minds. =20

[=E2=80=A6]

 It uses 'dub describe' to find the files, this also allows us to=20
 check the compiler. Meson won't even let you try to compile the=20
 project if there's an compiler mismatch.


I am not sure how "dub describe" tells you which compiler is being used, it
always says "compiler": "dmd" on my projects even though I only use ldc2 fo=
r
build.

[=E2=80=A6]

 I mean, you can do this but as you said, it's not recommended.


Certainly it is not recommended for Meson builds but I see no reason why. I=
t
is just duplicating information that is held on the filestore, and DRY is
supposed to be a thing in computing. Convention over configuration really
makes things a lot easier. But I guess this debate would turn into a flame =
war
of competing prejudices, so probably best left alone.=20

[=E2=80=A6]

 Dub is implemented this way for a reason. We really *shouldn't*=20
 use dynamic libraries, at least for now.


Maybe then Dub should be better able to handle shared objects. Whilst the
whole static linking vs. shared objects is another topic that descends into
flame wars of prejudice, the de facto standard tool should not really be
taking sides in the war but should support both sides. Dub supports shared
objects as dependencies, but just fails to be able to build them.

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Dr Russel Winder      t: +44 20 7585 2200
41 Buckmaster Road    m: +44 7770 465 077
London SW11 1EN, UK   w: www.russel.org.uk
Aug 15 2018
next sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
On 15/08/2018 9:31 PM, Russel Winder wrote:

 Maybe then Dub should be better able to handle shared objects. Whilst the
 whole static linking vs. shared objects is another topic that descends into
 flame wars of prejudice, the de facto standard tool should not really be
 taking sides in the war but should support both sides. Dub supports shared
 objects as dependencies, but just fails to be able to build them.


Until shared library support is actually designed and only then 
implemented, supporting it in dub any further is a lost cause.
Aug 15 2018
prev sibling parent reply Filipe =?UTF-8?B?TGHDrW5z?= <lains archlinux.org> writes:
On Wednesday, 15 August 2018 at 09:31:48 UTC, Russel Winder wrote:

 That's the battle isn't it. Should the system that builds be 
 the system that also manages the packages for the build (cf. 
 Dub, Cargo, Go, Mason), or should the two be kept separate (cf. 
 CMake, Meson, SCons, Make).

 The debate has been had on the D mailing lists a number of 
 times and at great length. But in the end D, Rust, and Go are 
 going with a single system for everything.


This is obviously bad. Your distro has a package manager, you 
should use it, not create a separated language-specific one. If 
you are doing this locally, either but using the user's home of 
by installing to /usr/local, I don't think it's much of a 
problem. If you are implementing something like this at least do 
it in a way that the package managing feature is optional. I 
don't know if I'm being biased by being an Archlinux TU but from 
my perspective, it's not something we should do, at the very 
least globally.


 I am not sure how "dub describe" tells you which compiler is 
 being used, it always says "compiler": "dmd" on my projects 
 even though I only use ldc2 for build.


Weirdly enough, I can reproduce. This was working when I wrote 
the patch. I've opened an issue in the upstream.
Aug 16 2018
parent Russel Winder <russel winder.org.uk> writes:
On Thu, 2018-08-16 at 22:44 +0000, Filipe La=C3=ADns via Digitalmars-d-anno=
unce
wrote:
[=E2=80=A6]

Apologies for the delay in replying to this one.


 This is obviously bad. Your distro has a package manager, you=20
 should use it, not create a separated language-specific one. If=20


I'm afraid you are onto a lost cause on this one. The whole JVM-based milie=
u,
Ruby, Python, Go, D, Rust, etc. all have language specific repositories.
Debian, Fedora, etc. pick and choose which bits they choose to package base=
d
on some algorithm almost, but not quite, totally unrelated to what is the
latest version. Operating system package managers are providing the operati=
ng
system, not the development tools and dependencies needed for software
development.=20

Go, Rust, and indeed D, are going the route of static compilation as much
because operating system dependencies can never be guaranteed, and are ofte=
n
wrong. It is not clear to me why Debian spend so much time packaging bits o=
f
the Go universe that no-one uses even if the dependencies are in fact used.

On the other hand, having GtkD (and GStreamerD) packaged is great since the=
re
are shared objects for use with D codes. There is nothing quite so depressi=
ng
as waiting for LDC or DMD to statically link to GtkD. So static linking is =
not
something I want. But waiting for Debian and Fedora to package things is of=
ten
like Waiting for Godot. Hence "build it yourself" becomes a bit of a must.

This is not a simple situation, and every individuals positions on it is
likely inconsistent and full of holes.=20


 you are doing this locally, either but using the user's home of=20
 by installing to /usr/local, I don't think it's much of a=20
 problem. If you are implementing something like this at least do=20
 it in a way that the package managing feature is optional. I=20
 don't know if I'm being biased by being an Archlinux TU but from=20
 my perspective, it's not something we should do, at the very=20
 least globally.


It may be that Arch stays more up to date than Debian and Fedora (because i=
t
is less centralised and more like Homebrew/Linuxbrew, but Debian (and Fedor=
a?)
is where the bulk of Linux programs get executed, and so is the obvious pla=
ce
to develop.
=20

 Weirdly enough, I can reproduce. This was working when I wrote=20
 the patch. I've opened an issue in the upstream.


Excellent.  I have a clone of Meson so can try stuff out on master/HEAD as
needed.

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Dr Russel Winder      t: +44 20 7585 2200
41 Buckmaster Road    m: +44 7770 465 077
London SW11 1EN, UK   w: www.russel.org.uk
Sep 06 2018