www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Compile all-of-dub?

reply "qznc" <qznc web.de> writes:
The Rust people have this Crater [0,1] tool, which essentially 
builds all Rust libraries with two compiler versions and compares 
for regressions.

Since D has a central library repository as well, it would make 
sense to do this broad testing as well. We don't have nightly 
builds (or do we?), but release candidates. Is something like 
this already done?

For example, dfmt broke between 2.068 and 2.068.1 [2]. It is 
easily possible to detect regressions like this automatically. 
The biggest problem is probably that someone has to provide 
servers or pay for AWS instances.

[0] https://internals.rust-lang.org/t/crater-plans/2206
[1] https://github.com/brson/taskcluster-crater
[2] 
https://github.com/Hackerpilot/dfmt/pull/184#issuecomment-138713802
Sep 09 2015
next sibling parent reply Rikki Cattermole <alphaglosined gmail.com> writes:
On 09/09/15 8:26 PM, qznc wrote:

 The Rust people have this Crater [0,1] tool, which essentially builds
 all Rust libraries with two compiler versions and compares for regressions.

 Since D has a central library repository as well, it would make sense to
 do this broad testing as well. We don't have nightly builds (or do we?),
 but release candidates. Is something like this already done?

 For example, dfmt broke between 2.068 and 2.068.1 [2]. It is easily
 possible to detect regressions like this automatically. The biggest
 problem is probably that someone has to provide servers or pay for AWS
 instances.

 [0] https://internals.rust-lang.org/t/crater-plans/2206
 [1] https://github.com/brson/taskcluster-crater
 [2] https://github.com/Hackerpilot/dfmt/pull/184#issuecomment-138713802


Or use travis-ci.
Sep 09 2015
next sibling parent reply "Edwin van Leeuwen" <edder tkwsping.nl> writes:
On Wednesday, 9 September 2015 at 08:56:56 UTC, Rikki Cattermole 
wrote:

 On 09/09/15 8:26 PM, qznc wrote:

 The Rust people have this Crater [0,1] tool, which essentially 
 builds
 all Rust libraries with two compiler versions and compares for 
 regressions.

 Since D has a central library repository as well, it would 
 make sense to
 do this broad testing as well. We don't have nightly builds 
 (or do we?),
 but release candidates. Is something like this already done?

 For example, dfmt broke between 2.068 and 2.068.1 [2]. It is 
 easily
 possible to detect regressions like this automatically. The 
 biggest
 problem is probably that someone has to provide servers or pay 
 for AWS
 instances.


 Or use travis-ci.


One problem with relying on travis is that it doesn't rerun tests 
for the libraries when compiler is updated. Crater seems to 
automatically compile/test all libraries whenever there is a 
compiler upgrade. Maybe it could be integrated with the 
autotester?

https://auto-tester.puremagic.com/
Sep 09 2015
parent Rikki Cattermole <alphaglosined gmail.com> writes:
On 09/09/15 9:21 PM, Edwin van Leeuwen wrote:

 On Wednesday, 9 September 2015 at 08:56:56 UTC, Rikki Cattermole wrote:

 On 09/09/15 8:26 PM, qznc wrote:

 The Rust people have this Crater [0,1] tool, which essentially builds
 all Rust libraries with two compiler versions and compares for
 regressions.

 Since D has a central library repository as well, it would make sense to
 do this broad testing as well. We don't have nightly builds (or do we?),
 but release candidates. Is something like this already done?

 For example, dfmt broke between 2.068 and 2.068.1 [2]. It is easily
 possible to detect regressions like this automatically. The biggest
 problem is probably that someone has to provide servers or pay for AWS
 instances.


 Or use travis-ci.


 One problem with relying on travis is that it doesn't rerun tests for
 the libraries when compiler is updated. Crater seems to automatically
 compile/test all libraries whenever there is a compiler upgrade. Maybe
 it could be integrated with the autotester?

 https://auto-tester.puremagic.com/


You would need to update the travis configuration file for the new 
compiler. That would trigger a new build.

Where is this problem?
Sep 09 2015
prev sibling parent "qznc" <qznc web.de> writes:
On Wednesday, 9 September 2015 at 08:56:56 UTC, Rikki Cattermole 
wrote:

 Or use travis-ci.


They have that as well: http://rust-ci.org/
Sep 09 2015
prev sibling next sibling parent "Sebastiaan Koppe" <mail skoppe.eu> writes:
On Wednesday, 9 September 2015 at 08:26:59 UTC, qznc wrote:

 The Rust people have this Crater [0,1] tool, which essentially 
 builds all Rust libraries with two compiler versions and 
 compares for regressions.

 Since D has a central library repository as well, it would make 
 sense to do this broad testing as well. We don't have nightly 
 builds (or do we?), but release candidates. Is something like 
 this already done?


I think this is a splendid idea. Limiting to just 
release-candidates allows the instances to be killed in between, 
thus saving costs.
Sep 09 2015
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-09-09 10:26, qznc wrote:

 The Rust people have this Crater [0,1] tool, which essentially builds
 all Rust libraries with two compiler versions and compares for regressions.

 Since D has a central library repository as well, it would make sense to
 do this broad testing as well. We don't have nightly builds (or do we?),
 but release candidates. Is something like this already done?

 For example, dfmt broke between 2.068 and 2.068.1 [2]. It is easily
 possible to detect regressions like this automatically. The biggest
 problem is probably that someone has to provide servers or pay for AWS
 instances.

 [0] https://internals.rust-lang.org/t/crater-plans/2206
 [1] https://github.com/brson/taskcluster-crater
 [2] https://github.com/Hackerpilot/dfmt/pull/184#issuecomment-138713802


I think it's a great idea. This has been suggested before. The 
objections were that:

* If you do find a problem who should be responsible for figuring out if 
it's a regression or an intended change?

* Not all packages are maintained enough to keep up with all compiler 
changes

-- 
/Jacob Carlborg
Sep 09 2015
next sibling parent "Jack Stouffer" <jack jackstouffer.com> writes:
On Wednesday, 9 September 2015 at 12:02:55 UTC, Jacob Carlborg 
wrote:

 I think it's a great idea. This has been suggested before. The 
 objections were that:

 * If you do find a problem who should be responsible for 
 figuring out if it's a regression or an intended change?

 * Not all packages are maintained enough to keep up with all 
 compiler changes


A good compromise is to whitelist libraries created by language 
contributors, e.g. vibed, cerealed, dfmt, etc.
Sep 09 2015
prev sibling parent reply "qznc" <qznc web.de> writes:
On Wednesday, 9 September 2015 at 12:02:55 UTC, Jacob Carlborg 
wrote:

 I think it's a great idea. This has been suggested before. The 
 objections were that:

 * If you do find a problem who should be responsible for 
 figuring out if it's a regression or an intended change?


It does raise the bar for language changes. Most changes should 
be backwards compatible. For intended changes it forces us to 
come up with a way to automatically detect and ideally fix it 
(dfix?).

In an ideal world, I would imagine it like this: Tester finds a 
package breaks. Package is on Github, so Tester can file an issue 
there. Tester checks out the source from repo, runs dfix, sends 
pull request referencing the issue.


 * Not all packages are maintained enough to keep up with all 
 compiler changes


Then it would be quite interesting to know about this and provide 
this information at code.dlang.org, like "broken for 2.068.0 and 
above".
Sep 09 2015
parent "ponce" <contact gam3sfrommars.fr> writes:
On Wednesday, 9 September 2015 at 13:48:16 UTC, qznc wrote:

 On Wednesday, 9 September 2015 at 12:02:55 UTC, Jacob Carlborg 
 wrote:

 I think it's a great idea. This has been suggested before. The 
 objections were that:

 * If you do find a problem who should be responsible for 
 figuring out if it's a regression or an intended change?


 It does raise the bar for language changes. Most changes should 
 be backwards compatible. For intended changes it forces us to 
 come up with a way to automatically detect and ideally fix it 
 (dfix?).

 In an ideal world, I would imagine it like this: Tester finds a 
 package breaks. Package is on Github, so Tester can file an 
 issue there. Tester checks out the source from repo, runs dfix, 
 sends pull request referencing the issue.


 * Not all packages are maintained enough to keep up with all 
 compiler changes


 Then it would be quite interesting to know about this and 
 provide this information at code.dlang.org, like "broken for 
 2.068.0 and above".


There was a thread in announce in the last 12 months that 
announced a site (was it ddocs.org?) that had documentation for 
every package in the DUB registry, for each tag. I think it also 
reported build problems. Don't know why it went offline.
Sep 09 2015
prev sibling next sibling parent reply "ZombineDev" <valid_email he.re> writes:
On Wednesday, 9 September 2015 at 08:26:59 UTC, qznc wrote:

...


https://github.com/MartinNowak/project_tester
Sep 11 2015
parent Martin Nowak <code+news.digitalmars dawg.eu> writes:
On 09/11/2015 01:37 PM, ZombineDev wrote:

 On Wednesday, 9 September 2015 at 08:26:59 UTC, qznc wrote:

 ...


 
 https://github.com/MartinNowak/project_tester


Right, please contribute.
Sep 11 2015
prev sibling parent Martin Nowak <code+news.digitalmars dawg.eu> writes:
On 09/09/2015 10:26 AM, qznc wrote:

 The Rust people have this Crater [0,1] tool, which essentially builds
 all Rust libraries with two compiler versions and compares for regressions.


I recently created a Jenkins project so that I can easily test many
projects with a particular DPL version or against PRs.

https://github.com/MartinNowak/project_tester
It's still much better when the actual author of a library tests the
latest beta, and beta are always available via Travis-CI.
Sep 11 2015