www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - dud: A dub replacement

reply Robert Schadek <rschadek symmetryinvestments.com> writes:
So dub has some problems, and personally I find its code base 
very hard to get
into.
At Symmetry we are a very heavy user of dub, resulting in many 
wasted hours.

So I started to write dud [1]. I kept some boring/nice parts from 
dub, but most
code so far is a complete rewrite.

The goal of dud is mostly do what dub does, but more 
understandable.
dud will/does aim for a tasteful subset of dub's features.
Meaning, most dub files should be good with dud.
If they are not, you will/should get an error message telling you 
whats wrong.
The bigger goal, at least personally, is to have a code base of 
pure functions
that is "trivial" to understand and debug.
The rules of thumb is: "When your line gets longer than 80 
characters,
restructure your code!", "Branch statements are code smells."

So what can dud do right now.

$ dud convert

works.
IMO that is an important step, as this involves ingesting 
dub.(json|sdl) into
a common representation.

I use dubproxy [2] to get all ~1.6k packages from code.dlang.org 
and create a
git working tree for all the versions of all the packages.
Currently, that results in ~60k (package|dub).(json|sdl) files.
Then, I run dud on those and see what breaks.
Only a few percent are not ingestable by dud, and those are in 
IHMO not valid
anyway (to be fair, there is some strange ūüí© out there).

Now that dud can parse dub files, the next step will be a 
semantic phase,
checking the input for errors.
After that, path expansion and dependency resolution.

PR's are always welcome.

Destroy!

[1] https://github.com/symmetryinvestments/dud
[2] https://github.com/symmetryinvestments/dubproxy
Nov 11 2019
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2019-11-11 14:44, Robert Schadek wrote:
 So dub has some problems, and personally I find its code base very hard 
 to get
 into.
 At Symmetry we are a very heavy user of dub, resulting in many wasted 
 hours.
 
 So I started to write dud [1]. I kept some boring/nice parts from dub, 
 but most
 code so far is a complete rewrite.
Hmm, that's unfortunate. It happens way too often in the D community, that a project is rewritten from scratch instead of improving the existing one. Especially since the D community is not that big. Although I can understand the choice. Even if you were to contribute there would be no one to merge the pull requests :(. -- /Jacob Carlborg
Nov 11 2019
parent jmh530 <john.michael.hall gmail.com> writes:
On Monday, 11 November 2019 at 19:17:54 UTC, Jacob Carlborg wrote:
 [snip]

 Hmm, that's unfortunate. It happens way too often in the D 
 community, that a project is rewritten from scratch instead of 
 improving the existing one. Especially since the D community is 
 not that big.

 Although I can understand the choice. Even if you were to 
 contribute there would be no one to merge the pull requests :(.
Yeah...it was wondering if there was a way to have like a dub-core that both dub and dud can depend on.
Nov 11 2019
prev sibling next sibling parent mipri <mipri minimaltype.com> writes:
On Monday, 11 November 2019 at 13:44:28 UTC, Robert Schadek wrote:
 So I started to write dud [1]. I kept some boring/nice parts 
 from dub, but most
 code so far is a complete rewrite.
Someone's eventually going to say this, and they probably really won't want to, so I'll spare them: this is a horrible name. Is it even deliberate? Is it only accidentally a meaningful English word? https://en.wiktionary.org/wiki/dud#Noun (informal) A device or machine that is useless because it does not work properly or has failed to work, such as a bomb, or explosive projectile. (informal) A failure of any kind. (informal) A loser; an unlucky person. A lottery ticket that does not give a payout. Maybe this is in the tradition of tool names that have bad connotations but are actually intended to be meaningless, like Fedora's dnf (Did Not Finish). Or even horrible, disgusting connotations, with the hope that this will serve as an attention-getter followed by the reader being unwilling to concede that emotions can interfere in technical decisions. ( https://duckduckgo.com/the%20open-source,%20cloud%20native%20SQL%20database )
Nov 11 2019
prev sibling next sibling parent GoaLitiuM <goalitium dforums.mail.kapsi.fi> writes:
On Monday, 11 November 2019 at 13:44:28 UTC, Robert Schadek wrote:
 So dub has some problems, and personally I find its code base 
 very hard to get into.
 At Symmetry we are a very heavy user of dub, resulting in many 
 wasted hours.
I'm interested in which areas does you and/or Symmetry find dub not delivering expected results? Is it package management in general? Build process? What's the actual need to rewrite dub again? I started working on a build tool to replace dub for my personal projects some time ago because I grew tired of long build times when only small portion of the project was touched. This last month I decided to rewrite it and work on it more actively, and I've already managed to get some huge libraries like vibe-d and serve-d building. I have no plans to rewrite the dub as a whole, but only focus on the build command, leaving the package management to dub which I have personally no problems with.
Nov 11 2019
prev sibling next sibling parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 11/11/19 8:44 AM, Robert Schadek wrote:
 
 The goal of dud is mostly do what dub does, but more understandable.
 dud will/does aim for a tasteful subset of dub's features.
 Meaning, most dub files should be good with dud.
 If they are not, you will/should get an error message telling you whats 
 wrong.
 The bigger goal, at least personally, is to have a code base of pure 
 functions
 that is "trivial" to understand and debug.
 The rules of thumb is: "When your line gets longer than 80 characters,
 restructure your code!", "Branch statements are code smells."
 
Sounds cool!
 Only a few percent are not ingestable by dud, and those are in IHMO not 
 valid
 anyway (to be fair, there is some strange ūüí© out there).
From what I've seen, dub tends to take the approach of "silently ignore anything in dub.sdl I don't recognize as valid". I'll admit that may be an important approach for a tool that has no concept of its own changing versions (which is somewhat strange for dub, considering half of dub is a package manager.) But the flipside is that this fosters and breeds errors and invalid files in the wild (see: 90's era web browsers). I'm curious, are there any particular anti-patterns you've noticed as being common in the uningestible repos?
Nov 14 2019
parent reply Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Thursday, 14 November 2019 at 23:33:06 UTC, Nick Sabalausky 
(Abscissa) wrote:

 From what I've seen, dub tends to take the approach of 
 "silently ignore anything in dub.sdl I don't recognize as 
 valid". I'll admit that may be an important approach for a tool 
 that has no concept of its own changing versions (which is 
 somewhat strange for dub, considering half of dub is a package 
 manager.) But the flipside is that this fosters and breeds 
 errors and invalid files in the wild (see: 90's era web 
 browsers).
Also, this, apparently, should lead to the fact that dud will have their own package description format. Almost inevitable. (This has already been discussed, perhaps.)
Nov 17 2019
parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Sunday, 17 November 2019 at 16:26:45 UTC, Denis Feklushkin 
wrote:
 On Thursday, 14 November 2019 at 23:33:06 UTC, Nick Sabalausky 
 (Abscissa) wrote:
 Also, this, apparently, should lead to the fact that dud will 
 have their own package description format. Almost inevitable. 
 (This has already been discussed, perhaps.)
SDL all the way please.
Nov 17 2019
next sibling parent Soulsbane <paul acheronsoft.com> writes:
On Sunday, 17 November 2019 at 19:10:05 UTC, Sebastiaan Koppe 
wrote:
 On Sunday, 17 November 2019 at 16:26:45 UTC, Denis Feklushkin 
 wrote:
 On Thursday, 14 November 2019 at 23:33:06 UTC, Nick Sabalausky 
 (Abscissa) wrote:
 Also, this, apparently, should lead to the fact that dud will 
 have their own package description format. Almost inevitable. 
 (This has already been discussed, perhaps.)
SDL all the way please.
Yes please
Nov 17 2019
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On Sunday, 17 November 2019 at 19:10:05 UTC, Sebastiaan Koppe 
wrote:

 SDL all the way please.
Perhaps this ship has already sail. But YAML would have been a better choice. It's a superset of JSON. All the existing JSON description files would have worked as is. -- /Jacob Carlborg
Nov 18 2019
parent "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 11/18/19 3:54 AM, Jacob Carlborg wrote:
 
 Perhaps this ship has already sail. But YAML would have been a better 
 choice. It's a superset of JSON. All the existing JSON description files 
 would have worked as is.
YAML's grammar is a convoluted read-only mess. Every time I have to write it I feel like I'm just guessing and hoping I'm magically writing it correctly. I've even had silent problems on travis-ci that turned out to be .travis.yml not being parsed the way I thought. That sort of thing shouldn't even be *possible* on a config file format. Especially irritating to me is even just the basic nesting - the whole "sometimes you HAVE to use a hyphen and sometimes you have to OMIT the hyphen (or it'll silently be interpreted wrong)". And yea, yea, "Object vs array", whatever, but when you're using YAML for hand-written config files, the whole distinction between what should be objects vs arrays is just a big pile of arbitrary pointlessness (a flaw it inherits from JSON). And the grammar being a superset of JSON doesn't really help much in practice, because real idiomatic YAML in the wild (including config file examples) doesn't use it.
Nov 18 2019
prev sibling parent reply Russel Winder <russel winder.org.uk> writes:
On Sun, 2019-11-17 at 19:10 +0000, Sebastiaan Koppe via Digitalmars-d-annou=
nce=20
wrote:
 On Sunday, 17 November 2019 at 16:26:45 UTC, Denis Feklushkin=20
 wrote:
 On Thursday, 14 November 2019 at 23:33:06 UTC, Nick Sabalausky=20
 (Abscissa) wrote:
 Also, this, apparently, should lead to the fact that dud will=20
 have their own package description format. Almost inevitable.=20
 (This has already been discussed, perhaps.)
=20 SDL all the way please.
Is SDL the right format? Cargo uses TOML to great effect. And TOML has, I suspect greater traction more widely than SDL. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk
Nov 18 2019
parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Monday, 18 November 2019 at 08:57:58 UTC, Russel Winder wrote:
 Is SDL the right format? Cargo uses TOML to great effect.

 And TOML has, I suspect greater traction more widely than SDL.
I personally prefer SDL when it comes to nested data, but yeah, that would work as well. The point I was making is to just have 1 format. With dud that should be possible.
Nov 18 2019
parent reply Russel Winder <russel winder.org.uk> writes:
On Mon, 2019-11-18 at 09:31 +0000, Sebastiaan Koppe via Digitalmars-d-annou=
nce=20
wrote:
 On Monday, 18 November 2019 at 08:57:58 UTC, Russel Winder wrote:
 Is SDL the right format? Cargo uses TOML to great effect.
=20
 And TOML has, I suspect greater traction more widely than SDL.
=20 I personally prefer SDL when it comes to nested data, but yeah,=20 that would work as well. =20 The point I was making is to just have 1 format. With dud that=20 should be possible.
For me the argument is that SDL and TOML are intended for human's to write configuration scripts, whilst XML and JSON are intended for computers to se= nd data to other computers. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk
Nov 18 2019
parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Monday, 18 November 2019 at 09:42:16 UTC, Russel Winder wrote:
 On Mon, 2019-11-18 at 09:31 +0000, Sebastiaan Koppe via 
 Digitalmars-d-announce wrote:
 On Monday, 18 November 2019 at 08:57:58 UTC, Russel Winder 
 wrote:
 Is SDL the right format? Cargo uses TOML to great effect.
 
 And TOML has, I suspect greater traction more widely than 
 SDL.
I personally prefer SDL when it comes to nested data, but yeah, that would work as well. The point I was making is to just have 1 format. With dud that should be possible.
For me the argument is that SDL and TOML are intended for human's to write configuration scripts, whilst XML and JSON are intended for computers to send data to other computers.
A win-win move would be to have dud emit the other formats automatically as part of the compilation procedure, so to have always all of them present and synced on the content. It shouldn't be too much difficult, and maybe it's a cleaver way to move on from discussions about different formats.
Nov 18 2019
next sibling parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Monday, 18 November 2019 at 09:53:56 UTC, Paolo Invernizzi 
wrote:
 A win-win move would be to have dud emit the other formats 
 automatically as part of the compilation procedure, so to have 
 always all of them present and synced on the content.
I already regret starting about this. Instead of rooting for the SDL format, I should have just recommended to deprecate the json one.
Nov 18 2019
parent reply Russel Winder <russel winder.org.uk> writes:
On Mon, 2019-11-18 at 10:26 +0000, Sebastiaan Koppe via Digitalmars-d-annou=
nce=20
wrote:
 On Monday, 18 November 2019 at 09:53:56 UTC, Paolo Invernizzi=20
 wrote:
 A win-win move would be to have dud emit the other formats=20
 automatically as part of the compilation procedure, so to have=20
 always all of them present and synced on the content.
=20 I already regret starting about this. Instead of rooting for the=20 SDL format, I should have just recommended to deprecate the json=20 one.
I do not think you should regret starting a discussion on this. Personally I only ever use SDL with Dub. Even contemplating using JSON for human written configuration files is, for me, totally the wrong thing to do= . --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk
Nov 18 2019
parent reply JN <666total wp.pl> writes:
On Monday, 18 November 2019 at 10:40:28 UTC, Russel Winder wrote:
 On Mon, 2019-11-18 at 10:26 +0000, Sebastiaan Koppe via 
 Digitalmars-d-announce wrote:
 On Monday, 18 November 2019 at 09:53:56 UTC, Paolo Invernizzi 
 wrote:
 A win-win move would be to have dud emit the other formats 
 automatically as part of the compilation procedure, so to 
 have always all of them present and synced on the content.
I already regret starting about this. Instead of rooting for the SDL format, I should have just recommended to deprecate the json one.
I do not think you should regret starting a discussion on this. Personally I only ever use SDL with Dub. Even contemplating using JSON for human written configuration files is, for me, totally the wrong thing to do.
I only use the JSON format. JSON is widespread together with XML. SDL I heard first time of in the context of Dub and never seen it used elsewhere. TOML also I know only from Cargo. YAML at least I know from several different projects. I guess the tool must be working very well though if the main argument is what kind of data format to use :)
Nov 18 2019
next sibling parent reply Russel Winder <russel winder.org.uk> writes:
On Mon, 2019-11-18 at 16:54 +0000, JN via Digitalmars-d-announce wrote:
 [=E2=80=A6]
=20
 I only use the JSON format. JSON is widespread together with XML.=20
 SDL I heard first time of in the context of Dub and never seen it=20
 used elsewhere. TOML also I know only from Cargo. YAML at least I=20
 know from several different projects.
It is so easy to make spelling errors in keys using hand-written JSON. SDL I have only ever used with Dub, but it works better for me than JSON. TOML is really an INI evolution so known to many. YAML is used by many but doesn't really allow for the ease of specifying li= sts as TOML and SDL have.
 I guess the tool must be working very well though if the main=20
 argument is what kind of data format to use :)
Probably yes. Though Cargo has taken many different decisions to Dub and mostly I think Cargo took better decisions. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk
Nov 18 2019
next sibling parent reply Tobias Pankrath <tobias pankrath.net> writes:
On Monday, 18 November 2019 at 19:54:38 UTC, Russel Winder wrote:

 Probably yes. Though Cargo has taken many different decisions 
 to Dub and mostly I think Cargo took better decisions.
Could you elaborate a bit, please? I am not familiar with Cargo though.
Nov 18 2019
next sibling parent Russel Winder <russel winder.org.uk> writes:
On Mon, 2019-11-18 at 20:19 +0000, Tobias Pankrath via Digitalmars-d-announ=
ce
wrote:
 On Monday, 18 November 2019 at 19:54:38 UTC, Russel Winder wrote:
=20
 Probably yes. Though Cargo has taken many different decisions=20
 to Dub and mostly I think Cargo took better decisions.
=20 Could you elaborate a bit, please? I am not familiar with Cargo=20 though.
The single biggest difference for me just now is management of compilation products. In an attempt to share compiled products between projects, Dub keeps all downloaded package source and compilations products of that source in a central location. This ends up with a very complicated, and more or less unusable except by Dub, store of compilation products =E2=80=93 especially = as the easy to get to compilation product has unknown compilation details. Also it assu= mes you have infinite amounts of disc space. Cargo keeps the downloaded package sources centrally, but leaves all compilation products in the project build area. This makes it easy to manag= e and the compilation details of the compilation products are easy to understand. OK you still have to manually manage the central repository but= as it is source only, you get a much smaller growth to infinite disc space requirement. As for smaller issues: I prefer TOML over SDL as project specification notation (JSON is not an option for me). The convention over configuration rules for projects with multiple compilat= ion products is nicer in Cargo compared to Dub. Cargo is far more accepted in Rust circles than Dub is in D circles. The is= sue is not that this is true, but why is it true: is it just that D folk are st= ill obsessed with Make, SCons, CMake, Meson, etc. where Rust folk have never ev= en considered them. D is still working with the ancient philosophy of the distributed standard library provides everything, cf. Python and "batteries included". Python ha= s long ago had to give up on this and open its arms to PyPI based package distribution. Go and Rust never went with the "batteries included" but put the infrastructure in place at the outset for supporting libraries from repositories =E2=80=93 Rust/Cargo handles this far better than Go. The upsh= ot is that the D package repository is still not really central to D code development. And using Git, Breezy, Mercurial, etc. repositories is just not as simple a= nd straightforward as it is with Go and Rust/Cargo. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk
Nov 19 2019
prev sibling parent David Gileadi <gileadisNOSPM gmail.com> writes:
On 11/18/19 1:19 PM, Tobias Pankrath wrote:
 On Monday, 18 November 2019 at 19:54:38 UTC, Russel Winder wrote:
 
 Probably yes. Though Cargo has taken many different decisions to Dub 
 and mostly I think Cargo took better decisions.
Could you elaborate a bit, please? I am not familiar with Cargo though.
See also Russel's previous post on this issue [1]:
 Go and Rust emphasised using Git, Mercurial, and Breezy repositories as
packages from the outset. Go chose not to add a central repository, Rust chose
to add one. Rust's choice was the correct one for supporting developers. In
hindsight, Go has had problems with packages from the outset based on the
initial workspace model. Slowly over the decade Go is finding ways forward.
Rust got it right from the beginning, once they had stripped down the standard
library and emphasised use of the central repository ‚Äď if only Phobos could
be stripped right back to the absolute necessary and everything else provided
via the central repository. Obviously not all is good in Rust-land, just as
with Python and PyPI, the central repository is not curated, and this leads to
horrible messes. Ceylon got this more right, but it is a language few have
heard of and even fewer use.
 
 Dub does not allow for use of Git, Mercurial, or Breezy repositories only the
uncurated (and therefore potentially problematic) central repository. OK so you
can do Git checkouts as a separate activity and use local filestore references,
but this is not feasible for packages in the central repository.
 
 Dub builds packages to a location outside the project being worked on. Cargo
pulls sources to such a central non-project place and then compiles into a
project specific location. Dub tries to store all compiled version out of
project in the same area as the sources. Does this matter? Isn't Dub making
things easier to share? Sort of, sort of, and no. Dub stores all compilations,
but hides them and presents only the last compilation easily. This makes things
hard to work with for non-Dub tooling. Cargo makes it a lot easier, at the
expense of lack of sharing but always compiling everything into the project
area.
 
 Cargo uses TOML, Dub uses SDL (or if you are masochistic JSON). 'nuff said.
 QED.
 
 Dub seems to have become the de facto, and indeed de jure, standard for D
build, and yet somehow Cargo is just assumed by all Rust programmers whereas
Dub is a source of contention and ill-will in the D community.
 
 Dub's biggest problem is that there are many in the D community who will not
use it ‚Äď and they are vocal about it. The solution is not censorship, the
solution is for Dub to evolve very rapidly so that these vocal people have the
rug pulled from under them, i.e. their complaints become invalid.
[1]: https://forum.dlang.org/post/mailman.543.1571300816.8294.digital ars-d puremagic.com (scroll toward the end of the post)
Nov 19 2019
prev sibling parent reply Laurent =?UTF-8?B?VHLDqWd1aWVy?= <laurent.treguier.sink gmail.com> writes:
On Monday, 18 November 2019 at 19:54:38 UTC, Russel Winder wrote:
 It is so easy to make spelling errors in keys using 
 hand-written JSON.
I don't understand why this would apply to JSON specifically. Whatever the language is, the config files will be hand-written; spelling errors are pretty universal, and anything we write is prone to mistakes to some extent
Nov 18 2019
parent reply Robert Schadek <rschadek symmetryinvestments.com> writes:
On Monday, 18 November 2019 at 23:08:13 UTC, Laurent Tréguier 
wrote:

 I don't understand why this would apply to JSON specifically. 
 Whatever the language is, the config files will be 
 hand-written; spelling errors are pretty universal, and 
 anything we write is prone to mistakes to some extent
dud already tells you if you mistyped a key. Adding new file formats is "trivial" with dud current code base. This https://github.com/symmetryinvestments/dud/blob/master/pkgdescription/source/dud/pkgd scription/package.d is the data structure used to mirror the file content. This https://github.com/symmetryinvestments/dud/blob/master/pkgdescription/source/dud/p gdescription/json.d is the json reader. Currently, I have no plans to add any other file format. But PR's are always welcome. The decision on json and sdl has been made a long time ago, for better or for worse. Please don't turn this thread into bike-shedding. Additionally, dub/dud is already on the way to add cli functions to manipulate the config file. A look at adding dependencies with "dub add" already shows that.
Nov 19 2019
next sibling parent Laurent =?UTF-8?B?VHLDqWd1aWVy?= <laurent.treguier.sink gmail.com> writes:
On Tuesday, 19 November 2019 at 08:15:20 UTC, Robert Schadek 
wrote:
 Currently, I have no plans to add any other file format.
 But PR's are always welcome.
 The decision on json and sdl has been made a long time ago, for 
 better
 or for worse.
 Please don't turn this thread into bike-shedding.
That wasn't my intention; sorry if I formulated my question badly. The reason I was asking is actually because I really don't see any one format having a huge advantage over the others; I'm fine with basically every format that has been listed in this thread so far.
Nov 19 2019
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 11/19/19 3:15 AM, Robert Schadek wrote:
 On Monday, 18 November 2019 at 23:08:13 UTC, Laurent Tréguier wrote:
 
 I don't understand why this would apply to JSON specifically. Whatever 
 the language is, the config files will be hand-written; spelling 
 errors are pretty universal, and anything we write is prone to 
 mistakes to some extent
dud already tells you if you mistyped a key. Adding new file formats is "trivial" with dud current code base. This https://github.com/symmetryinvestments/dud/blob/master/pkgdescription/source/dud/pkgd scription/package.d is the data structure used to mirror the file content. This https://github.com/symmetryinvestments/dud/blob/master/pkgdescription/source/dud/p gdescription/json.d is the json reader. Currently, I have no plans to add any other file format. But PR's are always welcome. The decision on json and sdl has been made a long time ago, for better or for worse. Please don't turn this thread into bike-shedding. Additionally, dub/dud is already on the way to add cli functions to manipulate the config file. A look at adding dependencies with "dub add" already shows that.
And I would complain that the fact json exists as a file format already screws up dub add -- using dub add removes ALL comments in an SDL file, and rewrites the file in the order it sees fit. result: I don't use dub add any more. -Steve
Nov 19 2019
next sibling parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 11/19/19 11:30 AM, Steven Schveighoffer wrote:
 
 And I would complain that the fact json exists as a file format already 
 screws up dub add -- using dub add removes ALL comments in an SDL file, 
 and rewrites the file in the order it sees fit.
 
 result: I don't use dub add any more.
Oops, that's probably actually my fault, not dub's. Those are current limitations of the sdl lib, SDLang-D. 1. The (pull) parser doesn't currently emit comments (and consequently the DOM doesn't support them either, not that it matters for dub which uses the pull parser directly). IIRC, the comments are currently being ignored in the lexer. Shouldn't be TOO terribly hard to fix, though. Modify the lexer to emit them as a new token type, and adjust the parser's grammar logic to accept them and to emit them as a new pull parser event. (PR's always welcome!) 2. IIRC, the lib's only method of generating sdl involves using the DOM, and TBH, I don't remember offhand just how friendly/unfriendly it is to maintaining a specific ordering. I'd have to look into that. Ideally, there should probably be a range-based reverse-pull-parser that just takes user-generated pull parser events and spits out an sdl doc.
Nov 19 2019
parent Robert Schadek <rschadek symmetryinvestments.com> writes:
On Tuesday, 19 November 2019 at 17:13:49 UTC, Nick Sabalausky 
(Abscissa) wrote:
 On 11/19/19 11:30 AM, Steven Schveighoffer wrote:
 
 And I would complain that the fact json exists as a file 
 format already screws up dub add -- using dub add removes ALL 
 comments in an SDL file, and rewrites the file in the order it 
 sees fit.
 
 result: I don't use dub add any more.
Oops, that's probably actually my fault, not dub's. Those are current limitations of the sdl lib, SDLang-D.
dud has its own sdlang parser, I wanted to have no dependencies other than phobos and SDLang-D was not pure nor safe, mostly because of static Key[5] keywords; in Lexer.lexIdentKeyword.
Nov 20 2019
prev sibling parent Robert Schadek <rschadek symmetryinvestments.com> writes:
On Tuesday, 19 November 2019 at 16:30:26 UTC, Steven 
Schveighoffer wrote:
 And I would complain that the fact json exists as a file format 
 already screws up dub add -- using dub add removes ALL comments 
 in an SDL file, and rewrites the file in the order it sees fit.

 result: I don't use dub add any more.

 -Steve
I haven't implemented dud add yet, but it should be as easy as parsing the file checking if the dep is already in. If it is not the case add a new line to the end of the file. dub.sdl files have no order after all.
Nov 20 2019
prev sibling parent reply Guillaume Piolat <first.last gmail.com> writes:
On Monday, 18 November 2019 at 16:54:21 UTC, JN wrote:
 Personally I only ever use SDL with Dub. Even contemplating 
 using JSON for human written configuration files is, for me, 
 totally the wrong thing to do.
I only use the JSON format. JSON is widespread together with XML. SDL I heard first time of in the context of Dub and never seen it used elsewhere. TOML also I know only from Cargo. YAML at least I know from several different projects. I guess the tool must be working very well though if the main argument is what kind of data format to use :)
I also only ever use JSON. When SDL was introduced, it was promised "JSON will never be deprecated". And here people are wanting to deprecate JSON as if SDL parsers were all that numerous. JSON is the answer, SDL is the failed experiment that just bring disagreement :) We DO parse dub.json a lot. I suspect any D organization probably does because dub describe is slow. I don't care how "inhuman" a 20-line file is.
Nov 22 2019
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 23/11/2019 4:37 AM, Guillaume Piolat wrote:
 On Monday, 18 November 2019 at 16:54:21 UTC, JN wrote:
 Personally I only ever use SDL with Dub. Even contemplating using 
 JSON for human written configuration files is, for me, totally the 
 wrong thing to do.
I only use the JSON format. JSON is widespread together with XML. SDL I heard first time of in the context of Dub and never seen it used elsewhere. TOML also I know only from Cargo. YAML at least I know from several different projects. I guess the tool must be working very well though if the main argument is what kind of data format to use :)
I also only ever use JSON. When SDL was introduced, it was promised "JSON will never be deprecated". And here people are wanting to deprecate JSON as if SDL parsers were all that numerous. JSON is the answer, SDL is the failed experiment that just bring disagreement :) We DO parse dub.json a lot. I suspect any D organization probably does because dub describe is slow. I don't care how "inhuman" a 20-line file is.
+1
Nov 22 2019
parent GoaLitiuM <goalitium dforums.mail.kapsi.fi> writes:
On Saturday, 23 November 2019 at 03:57:02 UTC, rikki cattermole 
wrote:
 On 23/11/2019 4:37 AM, Guillaume Piolat wrote:
 On Monday, 18 November 2019 at 16:54:21 UTC, JN wrote:
 Personally I only ever use SDL with Dub. Even contemplating 
 using JSON for human written configuration files is, for me, 
 totally the wrong thing to do.
I only use the JSON format. JSON is widespread together with XML. SDL I heard first time of in the context of Dub and never seen it used elsewhere. TOML also I know only from Cargo. YAML at least I know from several different projects. I guess the tool must be working very well though if the main argument is what kind of data format to use :)
I also only ever use JSON. When SDL was introduced, it was promised "JSON will never be deprecated". And here people are wanting to deprecate JSON as if SDL parsers were all that numerous. JSON is the answer, SDL is the failed experiment that just bring disagreement :) We DO parse dub.json a lot. I suspect any D organization probably does because dub describe is slow. I don't care how "inhuman" a 20-line file is.
+1
+1, and JSON as a main format would make sense too because querying information from DUB registry is already being done in JSON, so throwing in SDL would complicate things. There's also plenty of JSON parsers out there, some of them being extremely performant (like fast.json and asdf), but only one (AFAIK) SDL parser with no ideas how fast it is. The performance should not be ignored as every build operation requires parsing all of the dependencies and their dependencies too which can quickly accumulate with larger complicated projects. The easy of modifying SDL vs JSON is a non-issue for me, you don't spend that much time editing the configuration files while developing your project.
Nov 23 2019
prev sibling parent reply Russel Winder <russel winder.org.uk> writes:
On Mon, 2019-11-18 at 09:53 +0000, Paolo Invernizzi via Digitalmars-d-annou=
nce=20
wrote:
[=E2=80=A6]
 A win-win move would be to have dud emit the other formats=20
 automatically as part of the compilation procedure, so to have=20
 always all of them present and synced on the content.
Why? In fact, why even think of doing this at all? There should be one and only one human written configuration file for a bui= ld and it should be in a notation suitable for being written by humans.
 It shouldn't be too much difficult, and maybe it's a cleaver way=20
 to move on from discussions about different formats.
Again why? It seems like a pointless overhead that achieves nothing constructive. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk
Nov 18 2019
parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Monday, 18 November 2019 at 10:44:18 UTC, Russel Winder wrote:
 On Mon, 2019-11-18 at 09:53 +0000, Paolo Invernizzi via 
 Digitalmars-d-announce
 wrote:
 […]
 A win-win move would be to have dud emit the other formats 
 automatically as part of the compilation procedure, so to have 
 always all of them present and synced on the content.
Why? In fact, why even think of doing this at all? There should be one and only one human written configuration file for a build and it should be in a notation suitable for being written by humans.
 It shouldn't be too much difficult, and maybe it's a cleaver 
 way to move on from discussions about different formats.
Again why? It seems like a pointless overhead that achieves nothing constructive.
Closing this kind of discussions and letting anyone to choose "tabs or spaces" is a constructive solution, I think. That's the whole point of a win-win solution, anyone has a gain, the JSON party and the SDL party. But, hey, as someone has to implement that, feel free to simply ignore my opinion ...
Nov 18 2019
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Monday, 18 November 2019 at 13:19:12 UTC, Paolo Invernizzi 
wrote:
 Closing this kind of discussions and letting anyone to choose 
 "tabs or spaces" is a constructive solution, I think.
It is quite extraordinary how readily folks fall to arguing over what the config format should be, rather than what the app should actually be able to do. :-\
Nov 18 2019
next sibling parent reply Russel Winder <russel winder.org.uk> writes:
On Mon, 2019-11-18 at 15:35 +0000, Joseph Rushton Wakeling via Digitalmars-=
d-
announce wrote:
=20
[=E2=80=A6]
 It is quite extraordinary how readily folks fall to arguing over=20
 what the config format should be, rather than what the app should=20
 actually be able to do. :-\
Perhaps because writing the configuration files is a critical part of the usability of the tool. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk
Nov 18 2019
next sibling parent reply bachmeier <no spam.net> writes:
On Monday, 18 November 2019 at 19:44:46 UTC, Russel Winder wrote:
 On Mon, 2019-11-18 at 15:35 +0000, Joseph Rushton Wakeling via 
 Digitalmars-d- announce wrote:
 
[…]
 It is quite extraordinary how readily folks fall to arguing 
 over what the config format should be, rather than what the 
 app should actually be able to do. :-\
Perhaps because writing the configuration files is a critical part of the usability of the tool.
IMO this is one of the most important parts of the first five minutes with the language. Someone has installed the compiler, and now they want to test it out. If they have a bad experience with Dub, they will not continue with the language. A package manager, including the choice of format, is something you have to get right. Rust understands this.
Nov 18 2019
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Monday, 18 November 2019 at 20:48:53 UTC, bachmeier wrote:
 IMO this is one of the most important parts of the first five 
 minutes with the language. Someone has installed the compiler, 
 and now they want to test it out. If they have a bad experience 
 with Dub, they will not continue with the language. A package 
 manager, including the choice of format, is something you have 
 to get right. Rust understands this.
Fair point. But that isn't something that has to be decided at the _start_ of a rewrite: better to focus on wanted behaviour, and then derive the ideal config format from that.
Nov 19 2019
parent Russel Winder <russel winder.org.uk> writes:
On Tue, 2019-11-19 at 10:38 +0000, Joseph Rushton Wakeling via Digitalmars-=
d-
announce wrote:
 On Monday, 18 November 2019 at 20:48:53 UTC, bachmeier wrote:
 IMO this is one of the most important parts of the first five=20
 minutes with the language. Someone has installed the compiler,=20
 and now they want to test it out. If they have a bad experience=20
 with Dub, they will not continue with the language. A package=20
 manager, including the choice of format, is something you have=20
 to get right. Rust understands this.
=20 Fair point. But that isn't something that has to be decided at=20 the _start_ of a rewrite: better to focus on wanted behaviour,=20 and then derive the ideal config format from that.
I'd argue that from a socio-technical perspective it is an important factor that should be an integral part of developing any user facing tool. The functionality of the tool is important, but how that functionality is presented to a user new to the tool and a user proficient with the tool sho= uld be equally important. As an example of how not to do things, perhaps I can suggest earlier forms = of Git. It took nigh on a decade to simplify things to get to the overcomplica= ted CLI we have today with Git. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk
Nov 19 2019
prev sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Monday, 18 November 2019 at 19:44:46 UTC, Russel Winder wrote:
 On Mon, 2019-11-18 at 15:35 +0000, Joseph Rushton Wakeling via 
 Digitalmars-d- announce wrote:
 
[…]
 It is quite extraordinary how readily folks fall to arguing 
 over what the config format should be, rather than what the 
 app should actually be able to do. :-\
Perhaps because writing the configuration files is a critical part of the usability of the tool.
Fair. My remark was maybe a little too intemperate :-) I'm simply concerned that if we don't put enough scrutiny on the app features and behaviour, we run the risk of simply reproducing some of the problematic design decisions of the existing tool. As an example -- try running `dub test --build=release`. Intuition suggests that this is testing a release build. But actually it strips out your unittests, because `--build=release` determines not only the optimization flags, but the complete set of DFLAGS used -- and `-unittest` isn't among them. I ran `dub test && dub test --build=release` as a matter of habit for some time before discovering the latter wasn't actually testing anything. And while I now know that I _can_ define a custom build to get what I want, that's clumsy and hard(er) to discover, and annoying to have to repeat for every project I create. That's the kind of usability and design concern we really ought to be revisiting in any rewrite.
Nov 19 2019
parent Russel Winder <russel winder.org.uk> writes:
On Tue, 2019-11-19 at 10:32 +0000, Joseph Rushton Wakeling via Digitalmars-=
d-
announce wrote:
 [=E2=80=A6]
=20
 I'm simply concerned that if we don't put enough scrutiny on the=20
 app features and behaviour, we run the risk of simply reproducing=20
 some of the problematic design decisions of the existing tool.
=20
 As an example -- try running `dub test --build=3Drelease`. =20
 Intuition suggests that this is testing a release build.  But=20
 actually it strips out your unittests, because `--build=3Drelease`=20
 determines not only the optimization flags, but the complete set=20
 of DFLAGS used -- and `-unittest` isn't among them.
=20
 I ran `dub test && dub test --build=3Drelease` as a matter of habit=20
 for some time before discovering the latter wasn't actually=20
 testing anything.  And while I now know that I _can_ define a=20
 custom build to get what I want, that's clumsy and hard(er) to=20
 discover, and annoying to have to repeat for every project I=20
 create.
=20
 That's the kind of usability and design concern we really ought=20
 to be revisiting in any rewrite.
How can I not agree with this being a critical bug in Dub! :-) --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk
Nov 19 2019
prev sibling parent Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Monday, 18 November 2019 at 15:35:12 UTC, Joseph Rushton 
Wakeling wrote:
 On Monday, 18 November 2019 at 13:19:12 UTC, Paolo Invernizzi 
 wrote:
 Closing this kind of discussions and letting anyone to choose 
 "tabs or spaces" is a constructive solution, I think.
It is quite extraordinary how readily folks fall to arguing over what the config format should be, rather than what the app should actually be able to do. :-\
Exactly, that's why I love pragmatism: let's dud emit the ones that are outdated, automatically. Everyone can choose, work and update or the format he prefers, without impacting other people choices, and the discussion can move forward to something more interesting ... my 2 cents, at least ...
Nov 18 2019
prev sibling next sibling parent reply FeepingCreature <feepingcreature gmail.com> writes:
On Monday, 11 November 2019 at 13:44:28 UTC, Robert Schadek wrote:
 Now that dud can parse dub files, the next step will be a 
 semantic phase,
 checking the input for errors.
 After that, path expansion and dependency resolution.

 PR's are always welcome.

 Destroy!

 [1] https://github.com/symmetryinvestments/dud
 [2] https://github.com/symmetryinvestments/dubproxy
Are you looking to replace dub as the reference build tool for D? (Please say yes...) Any estimate what the schedule looks like until dud can be used in practice?
Nov 15 2019
parent Robert Schadek <rschadek symmetryinvestments.com> writes:
On Friday, 15 November 2019 at 10:29:07 UTC, FeepingCreature 
wrote:
 Are you looking to replace dub as the reference build tool for 
 D? (Please say yes...)
reference build tool, I don't know. We will see.
 Any estimate what the schedule looks like until dud can be used 
 in practice?
dub convert works already ;-) No, PR's welcome
Nov 17 2019
prev sibling next sibling parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Monday, 11 November 2019 at 13:44:28 UTC, Robert Schadek wrote:
 So dub has some problems, and personally I find its code base 
 very hard to get into.
I understand what you mean. What I found most disruptive to my internal model of the code is the mutable state. It prevented me from a solid understanding of all the parts.
 The bigger goal, at least personally, is to have a code base of 
 pure functions
 that is "trivial" to understand and debug.
 The rules of thumb is: "When your line gets longer than 80 
 characters,
 restructure your code!", "Branch statements are code smells."
I certainly applaud your goal and I think many parts can be improved upon. However, not everything is accidental complexity; sometimes problems are hard and code is non trivial. Although in almost all cases it can be made nicer, it is just a matter of how much time you want to spend on it.
Nov 17 2019
prev sibling next sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Monday, 11 November 2019 at 13:44:28 UTC, Robert Schadek wrote:
 So dub has some problems, and personally I find its code base 
 very hard to get
 into.
 At Symmetry we are a very heavy user of dub, resulting in many 
 wasted hours.

 So I started to write dud [1]. I kept some boring/nice parts 
 from dub, but most
 code so far is a complete rewrite.

 The goal of dud is mostly do what dub does, but more 
 understandable.
Cool :-) Since I have also been experiencing a fair bit of production-use DUB pain in the last year, I really appreciate your taking action on this. A few things that would be good to understand up front: * what are the particular issues with DUB that you want to address? - making the codebase cleaner and more functional is obviously nice but is at most a means to an end -- what's the real end you have in mind? - I would imagine getting dependency resolution really right would be top of the list -- it would be good to aim to fix issues like https://github.com/dlang/dub/issues/1732 - I would personally really appreciate it if you would make it a design goal to better separate concerns in terms of what DFLAGS are used and why (for example, the fact that right now `dub test --build=release` will not actually run unittests, as `--build=release` drops the `-unittest` flag) * are there any particular known issues with DUB that this definitely will NOT address? * are there any key _breaking_ changes you have in mind? * where does this stand w.r.t. some of the proposals to break DUB apart into more cleanly separated components, e.g. determining compatible dependencies, downloading dependencies, building or running tests ... ? Some concrete feedback on the project as it stands: * the tickboxes of compatible commands are nice, but it would be good to have a more contextualized roadmap, in particular outlining the design concerns for core data structures and functionality - this should probably be in issues rather than the README, but it needs to be somewhere, otherwise it's hard for anyone outside to identify things they can do - it might be nice to use a GitHub project to manage things so that us outside folks can identify better what's being worked on and what's blocked by what * I don't mind breaking changes in either the config format or the command line API if it gets us to a nicer tool, so please embrace the opportunity where appropriate - I can imagine this might be the reason why the aim is to support a "tasteful subset" of DUB's features: it means that others can be re-implemented in an incompatible but better way - auto-conversion mechanisms may be preferable to outright support for old formats and commands * I really recommend trying to start writing clean, clear commit messages from the very start -- think of this as another form of code documentation that communicates to all interested readers the intent and considerations behind any particular change to the codebase. This makes it much easier for outsiders to get a clear understanding of the project and get into the habit of contribution - right now, pretty much all the commit messages read like spontaneous notes where even YOU won't remember the whys or wherefores in a few months' time - long term it saves a LOT of time when trying to look back and understand "Why did we do things that way?" -- particularly useful when trying to fix some subtle design bug * for the same reasons, really try to provide good documentation and comments for all code from the start -- this really makes it much easier for anyone interested to grasp the major design concerns and get contributing * these concerns are going to be strongest for the key data structures and algorithms -- please make sure these are REALLY documented well, from the very start Hope all of that's helpful, and best wishes for taking this forward -- I will try to help as I can, but time right now is very constrained ... ;-) Thanks & best wishes, -- Joe
Nov 18 2019
next sibling parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 11/18/19 7:59 AM, Joseph Rushton Wakeling wrote:
        - I would imagine getting dependency resolution really right
          would be top of the list -- it would be good to aim to fix
          issues like https://github.com/dlang/dub/issues/1732
As has been discussed elsewhere a few months ago, dependency resolution should be outsourced to an established SAT <https://en.wikipedia.org/wiki/Boolean_satisfiability_problem> solving lib, to avoid re-inventing a notoriously difficult wheel. This is what all the serious package managers have been moving towards, after invariably hitting problems (much like dub) trying to roll-their-own.
Nov 18 2019
next sibling parent reply Robert Schadek <rschadek symmetryinvestments.com> writes:
On Monday, 18 November 2019 at 16:31:09 UTC, Nick Sabalausky 
(Abscissa) wrote:
 On 11/18/19 7:59 AM, Joseph Rushton Wakeling wrote:
        - I would imagine getting dependency resolution really 
 right
          would be top of the list -- it would be good to aim 
 to fix
          issues like https://github.com/dlang/dub/issues/1732
As has been discussed elsewhere a few months ago, dependency resolution should be outsourced to an established SAT <https://en.wikipedia.org/wiki/Boolean_satisfiability_problem> solving lib, to avoid re-inventing a notoriously difficult wheel. This is what all the serious package managers have been moving towards, after invariably hitting problems (much like dub) trying to roll-their-own.
OT: By saying "all the __serious__" you effectively ended this part of the thread. You basically say, that whatever I do, solve P=NP for instance, dud will never be a __serious__ package manager because it does not use an existing SAT solver. That's just bad arguing. The thing I want from dud, is to tell me what dependency chain let to conflicts and this I find hard to extract from current SAT solvers. Most I have worked with just told me: "This solution works" "NO" Also the debug experience is not really great most of the time. And I like a good debug experience. I'm not saying I will or want to reinvent the wheel on dependency resolution, but if there is a choice of Understandability + debugability vs. performance. Performance will suffer.
Nov 19 2019
parent "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 11/19/19 3:29 AM, Robert Schadek wrote:
 On Monday, 18 November 2019 at 16:31:09 UTC, Nick Sabalausky (Abscissa) 
 wrote:
 As has been discussed elsewhere a few months ago, dependency 
 resolution should be outsourced to an established SAT 
 <https://en.wikipedia.org/wiki/Boolean_satisfiability_problem> solving 
 lib, to avoid re-inventing a notoriously difficult wheel. This is what 
 all the serious package managers have been moving towards, after 
 invariably hitting problems (much like dub) trying to roll-their-own.
OT: By saying "all the __serious__" you effectively ended this part of the thread. You basically say, that whatever I do, solve P=NP for instance, dud will never be a __serious__ package manager because it does not use an existing SAT solver. That's just bad arguing.
Dude, it's just casual speech, not formal logic. A colloquialism, not a formal claim of "For all X, X always necessarily implies Y".
 The thing I want from dud, is to tell me what dependency chain let to 
 conflicts
 and this I find hard to extract from current SAT solvers.
 Most I have worked with just told me: "This solution works" "NO"
 Also the debug experience is not really great most of the time.
Ok, now that I wasn't aware of.
Nov 19 2019
prev sibling next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Monday, 18 November 2019 at 16:31:09 UTC, Nick Sabalausky 
(Abscissa) wrote:
 As has been discussed elsewhere a few months ago, dependency 
 resolution should be outsourced to an established SAT
I have no objections to that in principle, but I am not sure that will resolve the issue I posted: IIRC DUB's key problem is not so much how it resolves dependencies _per se_ but the fact that it includes dependency constraints that actually don't apply to a given build (e.g. constraints that derive from the testing requirements of dependencies).
Nov 19 2019
prev sibling parent drug <drug2004 bk.ru> writes:
On 11/18/19 7:31 PM, Nick Sabalausky (Abscissa) wrote:
 On 11/18/19 7:59 AM, Joseph Rushton Wakeling wrote:
        - I would imagine getting dependency resolution really right
          would be top of the list -- it would be good to aim to fix
          issues like https://github.com/dlang/dub/issues/1732
As has been discussed elsewhere a few months ago, dependency resolution should be outsourced to an established SAT <https://en.wikipedia.org/wiki/Boolean_satisfiability_problem> solving lib, to avoid re-inventing a notoriously difficult wheel. This is what all the serious package managers have been moving towards, after invariably hitting problems (much like dub) trying to roll-their-own.
does linear constraints solver fit here too?
Nov 19 2019
prev sibling parent reply Robert Schadek <rschadek symmetryinvestments.com> writes:
On Monday, 18 November 2019 at 12:59:25 UTC, Joseph Rushton 
Wakeling wrote:
 Cool :-)  Since I have also been experiencing a fair bit of 
 production-use DUB pain in the last year, I really appreciate 
 your taking action on this.

 A few things that would be good to understand up front:

   * what are the particular issues with DUB that you want to 
 address?

       - making the codebase cleaner and more functional is 
 obviously
         nice but is at most a means to an end -- what's the 
 real end
         you have in mind?
My reason for making it cleaner is, because I assume this will give me a build tool I can fix when broken. And hopefully it has less bugs to begin with because its cleaner.
       - I would imagine getting dependency resolution really 
 right
         would be top of the list -- it would be good to aim to 
 fix
         issues like https://github.com/dlang/dub/issues/1732
That is one thing yes.
       - I would personally really appreciate it if you would 
 make it
         a design goal to better separate concerns in terms of 
 what
         DFLAGS are used and why (for example, the fact that 
 right now
         `dub test --build=release` will not actually run 
 unittests,
         as `--build=release` drops the `-unittest` flag)

   * are there any particular known issues with DUB that this 
 definitely
     will NOT address?
I'm not sure yet.
   * are there any key _breaking_ changes you have in mind?

   * where does this stand w.r.t. some of the proposals to break 
 DUB apart
     into more cleanly separated components, e.g. determining 
 compatible
     dependencies, downloading dependencies, building or running 
 tests ... ?
It is build as a library first. The CLI is just using the library constructs.
 Some concrete feedback on the project as it stands:

   * the tickboxes of compatible commands are nice, but it would 
 be good to
     have a more contextualized roadmap, in particular outlining 
 the design
     concerns for core data structures and functionality

       - this should probably be in issues rather than the 
 README, but
         it needs to be somewhere, otherwise it's hard for 
 anyone outside
         to identify things they can do
True, I'll work on that.
       - it might be nice to use a GitHub project to manage 
 things so that
         us outside folks can identify better what's being 
 worked on and
         what's blocked by what
I already started that, somewhat.
   * I don't mind breaking changes in either the config format 
 or the command
     line API if it gets us to a nicer tool, so please embrace 
 the opportunity
     where appropriate

       - I can imagine this might be the reason why the aim is 
 to support
         a "tasteful subset" of DUB's features: it means that 
 others can
         be re-implemented in an incompatible but better way

       - auto-conversion mechanisms may be preferable to 
 outright support
         for old formats and commands

   * I really recommend trying to start writing clean, clear 
 commit messages
     from the very start -- think of this as another form of 
 code documentation
     that communicates to all interested readers the intent and 
 considerations
     behind any particular change to the codebase.  This makes 
 it much easier
     for outsiders to get a clear understanding of the project 
 and get into the
     habit of contribution

       - right now, pretty much all the commit messages read 
 like spontaneous
         notes where even YOU won't remember the whys or 
 wherefores in a few
         months' time
I know, I'll try to do better
       - long term it saves a LOT of time when trying to look 
 back and understand
         "Why did we do things that way?" -- particularly useful 
 when trying to
         fix some subtle design bug

   * for the same reasons, really try to provide good 
 documentation and comments
     for all code from the start -- this really makes it much 
 easier for anyone
     interested to grasp the major design concerns and get 
 contributing
Here is disagree, to a degree I consider comments a code smell. If I have to write them, I failed to convey the information needed to understand the code in the code.
   * these concerns are going to be strongest for the key data 
 structures and
     algorithms -- please make sure these are REALLY documented 
 well, from the
     very start

 Hope all of that's helpful, and best wishes for taking this 
 forward -- I will try to help as I can, but time right now is 
 very constrained ... ;-)

 Thanks & best wishes,

      -- Joe
Thank you
Nov 20 2019
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On Wednesday, 20 November 2019 at 11:40:19 UTC, Robert Schadek 
wrote:

 Here is disagree, to a degree I consider comments a code smell.
 If I have to write them, I failed to convey the information
 needed to understand the code in the code.
You think this is a code smell: https://dlang.org/phobos/std_array.html ? -- /Jacob Carlborg
Nov 20 2019
next sibling parent Robert Schadek <rschadek symmetryinvestments.com> writes:
I assume you don't mean the documentation for std.array 
specifically,
but the act of having documentation of the module.

Then, yes I do think documentation should not be needed.

I think it would be far better if I only needed the signatures of
the functions and the members of the structs to use them.
That I need to look at an example usage of a function to grasp its
meaning or read some text about it, is not a good sign IMHO.

Please keep in mind, I'm striving for an ideal, not reality.
dud will be documented, no question about it.
But I think, the longer documentation is not needed to understand
dud, the better.

p.s. I find it hard to explain this in writing.
It makes perfect sense in my head. Maybe I can convince you in 
person
at the next dconf.
Nov 20 2019
prev sibling parent reply Rumbu <rumbu rumbu.ro> writes:
On Wednesday, 20 November 2019 at 13:37:39 UTC, Jacob Carlborg 
wrote:
 On Wednesday, 20 November 2019 at 11:40:19 UTC, Robert Schadek 
 wrote:

 Here is disagree, to a degree I consider comments a code smell.
 If I have to write them, I failed to convey the information
 needed to understand the code in the code.
You think this is a code smell: https://dlang.org/phobos/std_array.html ? -- /Jacob Carlborg
When a function signature looks like this ElementEncodingType!(ElementType!RoR)[] join(RoR, R)(RoR ror, scope R sep) if (isInputRange!RoR && isInputRange!(Unqual!(ElementType!RoR)) && isInputRange!R && is(Unqual!(ElementType!(ElementType!RoR)) == Unqual!(ElementType!R))) or like this: E[] replaceFirst(E, R1, R2)(E[] subject, R1 from, R2 to) if (isDynamicArray!(E[]) && isForwardRange!R1 && is(typeof(appender!(E[])().put(from[0..1]))) && isForwardRange!R2 && is(typeof(appender!(E[])().put(to[0..1])))); it's understandable why documentation is mandatory.
Nov 20 2019
parent Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Wednesday, 20 November 2019 at 16:29:20 UTC, Rumbu wrote:
 When a function signature looks like this

 ElementEncodingType!(ElementType!RoR)[] join(RoR, R)(RoR ror, 
 scope R sep)
 if (isInputRange!RoR && isInputRange!(Unqual!(ElementType!RoR)) 
 && isInputRange!R && is(Unqual!(ElementType!(ElementType!RoR)) 
 == Unqual!(ElementType!R)))

 or like this:

 E[] replaceFirst(E, R1, R2)(E[] subject, R1 from, R2 to)
 if (isDynamicArray!(E[]) && isForwardRange!R1 && 
 is(typeof(appender!(E[])().put(from[0..1]))) && 
 isForwardRange!R2 && 
 is(typeof(appender!(E[])().put(to[0..1]))));

 it's understandable why documentation is mandatory.
That's true, Rumbu! And despite that, it's always marvel me the fact that I can simply read the above and actually "understand it"! It's some kind of magic, but maybe it's simply why I'm forced to read too much C++ recently... :-P PS ... the most difficult part for a beginner maybe is the historical "is(typeof( ... bla bla ...)"
Nov 21 2019
prev sibling next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Wednesday, 20 November 2019 at 11:40:19 UTC, Robert Schadek 
wrote:
 Here is disagree, to a degree I consider comments a code smell.
 If I have to write them, I failed to convey the information
 needed to understand the code in the code.
That depends on what you're using documentation and comments for. It's obviously great to try to write code that is as clean and clear and comprehensible as possible. But in general even the cleanest and clearest code rarely communicates the WHY behind design decisions. "This design was chosen because ..." "These are the assumptions made ..." etc. That WHY is what you want to clearly document, because that's what saves the most time for anyone looking to understand and modify the codebase.
Nov 21 2019
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 11/20/19 6:40 AM, Robert Schadek wrote:
 * for the same reasons, really try to provide good documentation and 
 comments
     for all code from the start -- this really makes it much easier 
 for anyone
     interested to grasp the major design concerns and get contributing
Here is disagree, to a degree I consider comments a code smell. If I have to write them, I failed to convey the information needed to understand the code in the code.
Assuming the code you wrote does what you wanted it to do... Often times, comments convey what you're thinking, and it's much easier to understand a description than mentally compiling and running the code to figure out what you were thinking. -Steve
Nov 21 2019
parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Thursday, 21 November 2019 at 14:50:59 UTC, Steven 
Schveighoffer wrote:
 Assuming the code you wrote does what you wanted it to do...

 Often times, comments convey what you're thinking, and it's 
 much easier to understand a description than mentally compiling 
 and running the code to figure out what you were thinking.
Exactly. And of course this isn't just for the benefit of others -- future you often needs the exact same help to understand what past you was thinking ...
Nov 21 2019
prev sibling next sibling parent Robert Schadek <rschadek symmetryinvestments.com> writes:
Regarding dependency resolution:

Did anybody here had a look at what the Dart people are
doing with pubgrab?

https://github.com/dart-lang/pub/blob/master/doc/solver.md
https://medium.com/ nex3/pubgrub-2fb6470504f
https://www.youtube.com/watch?v=Fifni75xYeQ

Especially the error reporting looks promising to me.
Nov 25 2019
prev sibling next sibling parent reply Andre Pany <andre s-e-a-p.de> writes:
On Monday, 11 November 2019 at 13:44:28 UTC, Robert Schadek wrote:
 So dub has some problems, and personally I find its code base 
 very hard to get
 into.
 At Symmetry we are a very heavy user of dub, resulting in many 
 wasted hours.

 So I started to write dud [1]. I kept some boring/nice parts 
 from dub, but most
 code so far is a complete rewrite.

 The goal of dud is mostly do what dub does, but more 
 understandable.
 dud will/does aim for a tasteful subset of dub's features.
 Meaning, most dub files should be good with dud.
 If they are not, you will/should get an error message telling 
 you whats wrong.
 The bigger goal, at least personally, is to have a code base of 
 pure functions
 that is "trivial" to understand and debug.
 The rules of thumb is: "When your line gets longer than 80 
 characters,
 restructure your code!", "Branch statements are code smells."

 So what can dud do right now.

 $ dud convert

 works.
 IMO that is an important step, as this involves ingesting 
 dub.(json|sdl) into
 a common representation.

 I use dubproxy [2] to get all ~1.6k packages from 
 code.dlang.org and create a
 git working tree for all the versions of all the packages.
 Currently, that results in ~60k (package|dub).(json|sdl) files.
 Then, I run dud on those and see what breaks.
 Only a few percent are not ingestable by dud, and those are in 
 IHMO not valid
 anyway (to be fair, there is some strange ūüí© out there).

 Now that dud can parse dub files, the next step will be a 
 semantic phase,
 checking the input for errors.
 After that, path expansion and dependency resolution.

 PR's are always welcome.

 Destroy!

 [1] https://github.com/symmetryinvestments/dud
 [2] https://github.com/symmetryinvestments/dubproxy
Is there any chance you can be convinced to join our force to improve Dub? A lot of developers invested their time to either improve Dub in general or to get their needed scenarios running. My gut feeling is, it would take years to rebuild the same set of functionality starting from the green field. The code base of Dub isn't that bad. I also invested time and there are some parts you can easily get into. Of course there are also some parts which are quite complex but I would say this is caused by their nature (Dependency resolution is a quite complex topic). But really the other parts are straightforward. The biggest issue with Dub is to find out whether some specific parts a bugs or intended feature. But this issue you will also face if you rebuild the functionality of Dub. You never know whether it was intended or it is a bug, or you should it work. Every effort which is put into Dub is immediately available for the whole D community, free developers and also companies from small size to the biggest size. (In no way I want to say you should not work on Dud, it is just my hope we can win you to help working on Dub). Kind regards André
Nov 25 2019
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Monday, 25 November 2019 at 11:59:11 UTC, Andre Pany wrote:
 Is there any chance you can be convinced to join our force to 
 improve Dub?
 A lot of developers invested their time to either improve Dub 
 in general
 or to get their needed scenarios running.

 My gut feeling is, it would take years to rebuild the same set 
 of functionality
 starting from the green field.
The distinction seems academic at this stage, because nothing stops anyone blessing dud as dub 2.0 when it's in a good enough state to be used in production, and nothing stops dub devs and contributors from offering feedback and code to help out with dud. And even if Robert were to work inside the dub repo, the basic tasks -- going bottom-up to rewrite core data structures and algorithms, etc. -- would still be mostly the same. That's why I strongly recommend that at this stage, the focus should be on the behaviours we want from our build system, and how they should interact and overlap. What's currently broken or impossible in DUB? What parts of that can be fixed without changing the config or CLI? And what improvements are most efficiently made via breaking changes? Please, let's bring our focus on that.
Nov 25 2019
next sibling parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Monday, 25 November 2019 at 12:15:42 UTC, Joseph Rushton 
Wakeling wrote:
 What's currently broken or impossible in DUB?  What parts of 
 that can be fixed without changing the config or CLI?  And what 
 improvements are most efficiently made via breaking changes?

 Please, let's bring our focus on that.
The biggest thing for me would be incremental compilation. As well as a dub build and test 'watch' mode to avoid scanning the dependencies every time.
Nov 25 2019
next sibling parent reply Robert Schadek <rschadek symmetryinvestments.com> writes:
On Monday, 25 November 2019 at 13:14:09 UTC, Sebastiaan Koppe 
wrote:
 The biggest thing for me would be incremental compilation. As 
 well as a dub build and test 'watch' mode to avoid scanning the 
 dependencies every time.
I think there are two levels to incremental compilation (IC). 1. File level IC. Meaning, if you have one file change, you only recompile all files that depend on that directly or transitively. Finally, you relink. 2. Symbol level IC. Meaning, the compiler becomes a daemon, and you track dependency on a symbol basis. For 1. my goal with dud is to do that. My first target is to emit ninja files. So IC is going to be the default, at least at first. For 2. dud is build as a library first, meaning should the compiler become a daemon at some point, libdud can be used to pull in packages from code.dlang.org and resolve dub packages dependencies.
Nov 25 2019
parent reply Atila Neves <atila.neves gmail.com> writes:
On Monday, 25 November 2019 at 15:27:10 UTC, Robert Schadek wrote:
 On Monday, 25 November 2019 at 13:14:09 UTC, Sebastiaan Koppe 
 wrote:
 The biggest thing for me would be incremental compilation. As 
 well as a dub build and test 'watch' mode to avoid scanning 
 the dependencies every time.
I think there are two levels to incremental compilation (IC). 1. File level IC. Meaning, if you have one file change, you only recompile all files that depend on that directly or transitively. Finally, you relink.
This is the done already by reggae. Unfortunately, since every D module is effectively a header, the number of files that need to be recompiled is usually large, despite the fact that for most changes the recompilation isn't actually necessary.
 For 1. my goal with dud is to do that. My first target is to 
 emit ninja files.
That isn't going to work: https://issues.dlang.org/show_bug.cgi?id=14188 reggae wraps dmd to act in a way that ninja can use. I'd also suggest that writing more ninja emitting code when reggae already does the job might be unnecessary. The reason I stopped working on my own dub replacement is because I needed to attach to "a real build system", and since I want to rewrite reggae from scratch by building on the "Build Systems à la carte" paper...
Nov 28 2019
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 28 November 2019 at 13:10:44 UTC, Atila Neves wrote:
 This is the done already by reggae. Unfortunately, since every 
 D module is effectively a header, the number of files that need 
 to be recompiled is usually large, despite the fact that for 
 most changes the recompilation isn't actually necessary.
Do you think it might work if it did dmd -H and make the auto-generated .di file and then did a content-based change detection on them for recompiling dependencies? It'd prolly still do more work than it has to, but seeing .d changed, rebuild .di, if .di changed, rebuild other stuff might just avoid full rebuilds upon just simple function body changes.
Nov 28 2019
parent Atila Neves <atila.neves gmail.com> writes:
On Thursday, 28 November 2019 at 14:06:39 UTC, Adam D. Ruppe 
wrote:
 On Thursday, 28 November 2019 at 13:10:44 UTC, Atila Neves 
 wrote:
 This is the done already by reggae. Unfortunately, since every 
 D module is effectively a header, the number of files that 
 need to be recompiled is usually large, despite the fact that 
 for most changes the recompilation isn't actually necessary.
Do you think it might work if it did dmd -H and make the auto-generated .di file and then did a content-based change detection on them for recompiling dependencies?
Probably. In fact, I had a plan to do exactly that and measure to see what the difference was. I just haven't gotten around to doing it yet.
 It'd prolly still do more work than it has to, but seeing .d 
 changed, rebuild .di, if .di changed, rebuild other stuff might 
 just avoid full rebuilds upon just simple function body changes.
That was my idea as well.
Nov 29 2019
prev sibling parent Atila Neves <atila.neves gmail.com> writes:
On Monday, 25 November 2019 at 13:14:09 UTC, Sebastiaan Koppe 
wrote:
 On Monday, 25 November 2019 at 12:15:42 UTC, Joseph Rushton 
 Wakeling wrote:
 What's currently broken or impossible in DUB?  What parts of 
 that can be fixed without changing the config or CLI?  And 
 what improvements are most efficiently made via breaking 
 changes?

 Please, let's bring our focus on that.
The biggest thing for me would be incremental compilation.
https://github.com/atilaneves/reggae Works right now with any (*) dub project. * If it doesn't, please file a bug
Nov 28 2019
prev sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Nov 25, 2019 at 12:15:42PM +0000, Joseph Rushton Wakeling via
Digitalmars-d-announce wrote:
[...]
 What's currently broken or impossible in DUB?
I'm probably not the intended audience here, but just so it's out there, here's a list of dub showstoppers for me: - lack of support for multi-language projects. E.g., a mixed C++/D codebase, or a Java/D codebase. (I didn't check, but supposedly this can be done as a pre/post action, but it's hacky and not well-integrated into the dependency resolution system). - lack of support for build-time code generation (i.e., build a subset of files into an executable, run the executable to generate .d files, compile output .d files plus other existing .d files into final product). - lack of support for cross-compilation (e.g., cross-compile to Android from a Linux x86 host, or cross-compile from Linux host to Windows executable via wine / cygwin). - lack of support for multiple targets (i.e., from given subsets of source files (with overlapping core files), build one executable for Linux/X11, build another executable in APK form for Android target). - unfriendly towards integration with other existing build systems (e.g., build a sub-project with cmake, then integrate products into current D project with dub). Probably some (all?) of these will require breaking changes because of the way dub is designed, so I'm not expecting this list will be met anytime in the near future. Though I'd love to be pleasantly surprised! --T
Nov 25 2019
next sibling parent reply GreatSam4sure <greatsam4sure gmail.com> writes:
On Monday, 25 November 2019 at 18:28:55 UTC, H. S. Teoh wrote:
 On Mon, Nov 25, 2019 at 12:15:42PM +0000, Joseph Rushton 
 Wakeling via Digitalmars-d-announce wrote: [...]
 [...]
I'm probably not the intended audience here, but just so it's out there, here's a list of dub showstoppers for me: [...]
I am interested in D/java project. Can you help me with material or link on that. I Will be happy to use javafx as front end and D as back end for a desktop App. I have looking on GraalVm but no luck yet. Just need a good tutorial for a start-up
Nov 25 2019
next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Nov 25, 2019 at 07:48:46PM +0000, GreatSam4sure via
Digitalmars-d-announce wrote:
[...]
 I am interested in D/java project. Can you help me with material or
 link on that. I Will be happy to use javafx as front end and D as back
 end for a desktop App.
 
 I have looking on GraalVm but no luck yet. Just need a good tutorial
 for a start-up
So far, my only experience with interfacing Java with D is my Android project, that involves Java code talking to Android's Java GUI APIs, communicating with D code via JNI. If you wish to go this route, I'd recommend reading up on JNI. The docs are mostly geared for interfacing with C/C++ code, but it should be relatively easy to interface with D code the same way. You just need to declare JNI methods in a D module so that D code can call it. Also, need to be aware of JNI quirks to avoid runtime crashes. With D's compile-time introspection capabilities, I was able to come up with a quite-nice wrapper for the JNI API, that saves a lot of typing (JNI is notoriously verbose to setup per call across the JNI boundary). In a nutshell, you declare native methods in your Java class(es), then write a D module with appropriately-named functions according to JNI naming convention, and compile that into an .so (or .lib), then use System.loadLibrary() to load the methods at runtime, then call away. This assumes the main program is started in Java, of course. I haven't tried doing it the other way (having main() in D, and calling Java from there), but I imagine it'd be pretty similar: link with a JVM, and in D's main() call whatever methods it takes to create the VM, then use JNI to call Java methods in that VM instance. Compiling a mixed-language project is a separate issue. Dub is, sad to say, of no help here. You're better off using something like CMake, or something of that sort. I use SCons, but it's not as well-known and does require a bit of an initial learning curve. T -- If the comments and the code disagree, it's likely that *both* are wrong. -- Christopher
Nov 25 2019
prev sibling parent =?UTF-8?B?UsOpbXkgTW91w6t6YQ==?= <remy.moueza gmail.com> writes:
On Monday, 25 November 2019 at 19:48:46 UTC, GreatSam4sure wrote:
 On Monday, 25 November 2019 at 18:28:55 UTC, H. S. Teoh wrote:
 On Mon, Nov 25, 2019 at 12:15:42PM +0000, Joseph Rushton 
 Wakeling via Digitalmars-d-announce wrote: [...]
 [...]
I'm probably not the intended audience here, but just so it's out there, here's a list of dub showstoppers for me: [...]
I am interested in D/java project. Can you help me with material or link on that. I Will be happy to use javafx as front end and D as back end for a desktop App. I have looking on GraalVm but no luck yet. Just need a good tutorial for a start-up
I once got a php/D bridge proof of concept, that should have worked with Java as well: - create some D utility code, - make an extern (C) API, - write the C headers of the API, - use swig (http://www.swig.org/) to wrap the C API to the language of choice. - Once loaded, the module has to initialize the D runtime, by a call to core.runtime.rt_init() or core.runtime.Runtime.initialize() (https://dlang.org/phobos/core_runtime.html). Swig generates shared objects. This worked well on Linux. I am not sure how the dll support is on Windows though: it could get more difficult. Back then, the extern (C++) wasn't as extensive as it is now, so today it might be more pertinent to create an extern (C++) API to conveniently wrap extern (C++) D classes/interfaces to Java classes/interfaces. These pages of the documentation detail how to interface with C and C++: https://dlang.org/spec/interfaceToC.html https://dlang.org/spec/cpp_interface.html Maybe the extern (C/C++) API approach can be used with GraalVM, I haven't researched that yet. When I tried GraalVM (for ahead of time native compilation) I got into difficulties with libraries using the java reflection API. One need to get the "pro" GraalVM distribution if I recall correctly. It seemed too much of a hurdle for what I was evaluating. I did eventually did fine without GraalVM. The C/C++ API from D process is quite involving. If I were to get Java and D together I would explore a D server communicating with a Java client over a TCP socket or HTTP, maybe using json: - Adam D. Ruppe wrote a D socket tutorial not long ago: https://forum.dlang.org/post/ciiiskgfeyhqfkfuxdai forum.dlang.org - I used asdf (http://code.dlang.org/packages/asdf) for JSON serialization and deserialization with success. There is also std.json already available in phobos. - Maybe a small http server with vibe.d could be started with even less code. For a desktop app, I would rather use dlangui (http://code.dlang.org/packages/dlangui). There are other maintained GUI libraries for D. Ron Tarrant has written many tutorials an GtkD on his blog page: - https://gtkdcoding.com/ - http://code.dlang.org/packages/gtk-d There is also DWT, a D port of the SWT Java GUI library: http://code.dlang.org/packages/dwt TKD is a binding over the Tcl/Tk toolkit: - http://code.dlang.org/packages/tkd
Nov 25 2019
prev sibling next sibling parent reply kinke <noone nowhere.com> writes:
On Monday, 25 November 2019 at 18:28:55 UTC, H. S. Teoh wrote:
 - lack of support for cross-compilation (e.g., cross-compile to 
 Android
   from a Linux x86 host, or cross-compile from Linux host to 
 Windows
   executable via wine / cygwin).
You're not up to speed, this has been solved with latest dub and LDC. No need for any Wine/Cygwin/MinGW/... either.
Nov 25 2019
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Nov 25, 2019 at 09:51:31PM +0000, kinke via Digitalmars-d-announce
wrote:
 On Monday, 25 November 2019 at 18:28:55 UTC, H. S. Teoh wrote:
 - lack of support for cross-compilation (e.g., cross-compile to
   Android from a Linux x86 host, or cross-compile from Linux host to
   Windows executable via wine / cygwin).
You're not up to speed, this has been solved with latest dub and LDC. No need for any Wine/Cygwin/MinGW/... either.
Oooh very nice!! That's wonderful to hear. So you're saying LDC out-of-the-box can cross-compile from Linux to Windows directly? How to do this? I'm *very* interested! T -- What did the alien say to Schubert? "Take me to your lieder."
Nov 25 2019
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 25 November 2019 at 23:46:31 UTC, H. S. Teoh wrote:
 Oooh very nice!!  That's wonderful to hear.  So you're saying 
 LDC out-of-the-box can cross-compile from Linux to Windows 
 directly?  How to do this?  I'm *very* interested!
ldc2 -mtriple=x86_64-pc-windows-msvc though you will prolly have to grab the druntime libs from the windows build. this same basic thing works for android and mac os too.
Nov 25 2019
next sibling parent tchaloupka <chalucha gmail.com> writes:
On Tuesday, 26 November 2019 at 00:32:54 UTC, Adam D. Ruppe wrote:
 On Monday, 25 November 2019 at 23:46:31 UTC, H. S. Teoh wrote:
 Oooh very nice!!  That's wonderful to hear.  So you're saying 
 LDC out-of-the-box can cross-compile from Linux to Windows 
 directly?  How to do this?  I'm *very* interested!
ldc2 -mtriple=x86_64-pc-windows-msvc though you will prolly have to grab the druntime libs from the windows build. this same basic thing works for android and mac os too.
Or with dub: dub build --arch=x86_64-pc-windows-msvc You only need to provide windows libs from windows ldc package to your linux installation and tweak the config. See https://wiki.dlang.org/Cross-compiling_with_LDC for details. Using it every day now and it works like a charm.
Nov 25 2019
prev sibling next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Nov 26, 2019 at 12:32:54AM +0000, Adam D. Ruppe via
Digitalmars-d-announce wrote:
 On Monday, 25 November 2019 at 23:46:31 UTC, H. S. Teoh wrote:
 Oooh very nice!!  That's wonderful to hear.  So you're saying LDC
 out-of-the-box can cross-compile from Linux to Windows directly?
 How to do this?  I'm *very* interested!
ldc2 -mtriple=x86_64-pc-windows-msvc though you will prolly have to grab the druntime libs from the windows build. this same basic thing works for android and mac os too.
Hmm. I tried this (test.d is a hello world program): ldc2 -mtriple=x86_64-pc-windows-msvc test.d and got this: Error: failed to locate link.exe Where am I supposed to get link.exe? T -- That's not a bug; that's a feature!
Nov 26 2019
parent reply kinke <noone nowhere.com> writes:
On Tuesday, 26 November 2019 at 21:19:58 UTC, H. S. Teoh wrote:
 	Error: failed to locate link.exe

 Where am I supposed to get link.exe?
You're supposed to consult the dedicated Wiki page linked by tchaloupka above (also linked in DMD 2.089 and LDC 1.18 release notes) in such a case, where you can read up on the used cross-linker, the -link-internally switch, and that it's supported in *official* prebuilt packages (i.e., the GitHub downloads, we don't officially support any distros). If you don't want to or cannot use the official package, then you can also install an external lld and use it via -linker=lld-link. On Monday, 25 November 2019 at 23:46:31 UTC, H. S. Teoh wrote:
 Oooh very nice!!  That's wonderful to hear.  So you're saying 
 LDC out-of-the-box can cross-compile from Linux to Windows 
 directly?
Almost out of the box as mentioned; Windows targets are particularly simple (no need for a cross-gcc/clang toolchain). This has been possible for over 2 years now (LDC v1.3); the MS libs aren't required anymore starting with LDC v1.13. Glancing over the release notes does pay off sometimes. ;)
Nov 26 2019
next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Nov 27, 2019 at 12:24:11AM +0000, kinke via Digitalmars-d-announce
wrote:
 On Tuesday, 26 November 2019 at 21:19:58 UTC, H. S. Teoh wrote:
 	Error: failed to locate link.exe
 
 Where am I supposed to get link.exe?
You're supposed to consult the dedicated Wiki page linked by tchaloupka above (also linked in DMD 2.089 and LDC 1.18 release notes) in such a case,
[...] I did consult the LDC cross-compiling page, and did modify ldc2.conf appropriately. But somewhere along the line I must have done something wrong, because it's still not cross-compiling properly. It's using the built-in LLD now, so the link.exe error is gone, but now it's coming back with a bunch of undefined druntime symbols even though I did specify the path to the prebuilt Windows druntime-ldc.lib in the ldc2.conf. I gave up and resorted to installing the Windows build of ldc2 under Wine, and now I have a build script that uses wineconsole to compile Windows executables. :-P So far, it appears to work. T -- Don't throw out the baby with the bathwater. Use your hands...
Nov 26 2019
prev sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Nov 26, 2019 at 05:10:41PM -0800, H. S. Teoh via Digitalmars-d-announce
wrote:
[...]
 I did consult the LDC cross-compiling page, and did modify ldc2.conf
 appropriately. But somewhere along the line I must have done something
 wrong, because it's still not cross-compiling properly. It's using the
 built-in LLD now, so the link.exe error is gone, but now it's coming
 back with a bunch of undefined druntime symbols even though I did
 specify the path to the prebuilt Windows druntime-ldc.lib in the
 ldc2.conf.
[...] Just for the record, I finally figured out that it was a mistake on my part -- I overlooked some source files when testing the cross-compilation so that's why it had link errors. After copying over the Windows libs and setting up ldc2.conf properly, cross-compilation to Windows does actually work flawlessly. Absolutely awesome. Kudos to the LDC team for making this happen!! T -- A bend in the road is not the end of the road unless you fail to make the turn. -- Brian White
Nov 27 2019
prev sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Nov 26, 2019 at 01:19:58PM -0800, H. S. Teoh via Digitalmars-d-announce
wrote:
 On Tue, Nov 26, 2019 at 12:32:54AM +0000, Adam D. Ruppe via
Digitalmars-d-announce wrote:
[...]
 ldc2 -mtriple=x86_64-pc-windows-msvc
 
 though you will prolly have to grab the druntime libs from the windows
 build.
[...] Tried installing the libs from the LDC windows build and editing ldc2.conf to setup the paths, etc., but still no go. It finally does find the linker and the Windows .lib's, but then it spews a whole bunch of undefined symbol errors. Decided instead to just run the native Windows version of ldc2 under Wine (via wineconsole); works like a charm. For now, at least for building Windows executables, I'm not gonna bother with cross-compiling. Wine appears to do the Right Thing(tm). T -- Those who've learned LaTeX swear by it. Those who are learning LaTeX swear at it. -- Pete Bleackley
Nov 26 2019
prev sibling parent Bastiaan Veelo <Bastiaan Veelo.net> writes:
On Monday, 25 November 2019 at 18:28:55 UTC, H. S. Teoh wrote:
 - lack of support for build-time code generation (i.e., build a 
 subset
   of files into an executable, run the executable to generate 
 .d files,
   compile output .d files plus other existing .d files into 
 final
   product).
I am doing exactly that. It can be done through preGenerateCommands, calling either a secondary build script or nested dub project, optionally using excludedSourceFiles. The only reason it doesn’t work well is because of https://github.com/dlang/dub/issues/1474. But there are workarounds, and a poc fix. Bastiaan.
Dec 01 2019
prev sibling parent reply Robert Schadek <rschadek symmetryinvestments.com> writes:
dud needs your help.

I'm starting work on the dependency resolution and for that I had
to implement proper handling for Semantic Versions, Version 
Ranges,
and Version Unions(VersionUnion is basically a VersionRange[]).
The dependency resolution algorithm I want to implement (based on
the algorithm used by Dart) needs a few checks and operations on
those types.

```D
SemVer parseSemVer(string input);

Nullable!VersionRange parseVersionRange(string input);

alias Types = AliasSeq!((SemVer,VersionRange,VersionUnion);
static foreach(T, Types) {
     auto inv = invert(T);
     static foreach(S, Types) {
         bool allowsAll(T, T);
         bool allowsAll(T, S);
         bool allowsAll(S, T);

         bool allowsAny(T, T);
         bool allowsAny(T, S);
         bool allowsAny(S, T);

         auto unionOf(T, T);
         auto unionOf(T, S);
         auto unionOf(S, T);

         auto intersectionOf(T, T);
         auto intersectionOf(T, S);
         auto intersectionOf(S, T);

         auto differenceOf(T, T);
         auto differenceOf(T, S);
         auto differenceOf(S, T);
     }
}
```

I think I did okay work and the tests cover all/most cases.
But that's properly being a bit overconfident.

Therefore, it would be awesome if you could try to break
these functions and create PRs that break the functions.

The code can be found in the git here 
https://github.com/symmetryinvestments/dud
the relevant folder is semver.
The tests are located in the files:

allowsAny: semver/source/dud/semver/checkstest.d
allowsAll: semver/source/dud/semver/checkstest1.d
allowsAll: semver/source/dud/semver/setoperationtest.d
intersectionOf: semver/source/dud/semver/setoperationtest1.d
invert, differenceOf: semver/source/dud/semver/setoperationtest2.d
semver/source/dud/semver/versionrangetest.d

Building dud, and semver should be as easy as cloning and typing 
dud, dub test.

```sh
git clone https://github.com/symmetryinvestments/dud.git
cd dud/semver
dub test
```
should get you going.

DESTROY!
Jan 23
parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Thursday, 23 January 2020 at 17:05:10 UTC, Robert Schadek 
wrote:
 DESTROY!
Tried: --- unittest { immutable SemVer v1 = SemVer(1,1,1); immutable SemVer v2 = SemVer(2,2,2); immutable SemVer v3 = SemVer(3,3,3); immutable SemVer v4 = SemVer(4,4,4); immutable VersionRange a = VersionRange(v1, Inclusive.yes, v2, Inclusive.yes); immutable VersionRange b = VersionRange(v3, Inclusive.yes, v4, Inclusive.yes); immutable VersionRange c = VersionRange(v2, Inclusive.yes, v3, Inclusive.yes); assert(intersectionOf(a, invert(a)).ranges.length == 0); assert(intersectionOf(a, a) == a); assert(intersectionOf(a, b) != a); assert(intersectionOf(a, b) != b); assert(intersectionOf(b, a) != a); assert(intersectionOf(b, a) != b); assert(intersectionOf(invert(a), invert(a)) == invert(a)); assert(invert(intersectionOf(a, invert(b))) == invert(intersectionOf(a,a))); auto add(T, P)(T a, P b) { return invert(intersectionOf(invert(a), invert(b))); } auto remove(T, P)(T a, P b) { return intersectionOf(a, invert(b)); } assert(add(a,b) == add(b,a)); assert(invert(invert(a)) == add(intersectionOf(a,c), remove(a,c))); assert(differenceOf(a, a).ranges.length == 0); assert(invert(intersectionOf(invert(a), invert(a))) == invert(invert(a))); assert(invert(invert(differenceOf(a, invert(a)))) == invert(invert(a))); assert(invert(invert(intersectionOf(a, c))) == invert(invert(remove(a, remove(a, c))))); } --- Everything green. Nice work. Haven't tried Inclusive.no yet. I'll leave that to someone else.
Jan 23
next sibling parent Robert Schadek <rschadek symmetryinvestments.com> writes:
On Thursday, 23 January 2020 at 20:50:09 UTC, Sebastiaan Koppe 
wrote:
 Haven't tried Inclusive.no yet. I'll leave that to someone else.
Indirectly you already did, invert turns Inclusive.yes bound into Inclusive.no bounds.
Jan 24
prev sibling parent Robert Schadek <rschadek symmetryinvestments.com> writes:
Thank you, very nice test
Jan 24