www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Continuous Integration

reply Jason House <jason.james.house gmail.com> writes:
Does anyone have a setup for doing continuous integration with d?
Mar 25 2008
next sibling parent reply Christopher Wright <dhasenan gmail.com> writes:
Jason House wrote:
 Does anyone have a setup for doing continuous integration with d?

I'm going to start on that now, since DUnit (latest svn) can output vaguely junit-compatible xml. It'll be using CruiseControl. By tomorrow, I'll be able to tell you when you should start bothering me about it in earnest. Of course, this will do you little good if you don't use DUnit or some other unittest runner besides the default one, since you just get a binary value for whether your tests pass, along with, if you're lucky, a stack trace. If you're not lucky, you just get the words "Segmentation fault". Well, okay, you could have a static constructor that outputs the first portion of the junit xml file, a couple of templates and functions that output more xml for each unittest, and your unittest main would output the end of the junit xml, or something like that. But at that point, you might as well just use DUnit.
Mar 25 2008
next sibling parent reply Jason House <jason.james.house gmail.com> writes:
Christopher Wright Wrote:

 Jason House wrote:
 Does anyone have a setup for doing continuous integration with d?

I'm going to start on that now, since DUnit (latest svn) can output vaguely junit-compatible xml. It'll be using CruiseControl. By tomorrow, I'll be able to tell you when you should start bothering me about it in earnest. Of course, this will do you little good if you don't use DUnit or some other unittest runner besides the default one, since you just get a binary value for whether your tests pass, along with, if you're lucky, a stack trace. If you're not lucky, you just get the words "Segmentation fault". Well, okay, you could have a static constructor that outputs the first portion of the junit xml file, a couple of templates and functions that output more xml for each unittest, and your unittest main would output the end of the junit xml, or something like that. But at that point, you might as well just use DUnit.

I'm sensitive to per-unittest code-writing overhead, but already have some in my code. If DUnit offers comparable overhead, and enhanced functionality, I'll switch.
Mar 26 2008
parent reply Christopher Wright <dhasenan gmail.com> writes:
Jason House wrote:
 I'm sensitive to per-unittest code-writing overhead, but already have some in
my code. If DUnit offers comparable overhead, and enhanced functionality, I'll
switch.

DUnit uses test fixture classes, so it probably won't suit you at present. I'll look into a method with less overhead.
Mar 26 2008
parent reply Jason House <jason.james.house gmail.com> writes:
Christopher Wright wrote:

 Jason House wrote:
 I'm sensitive to per-unittest code-writing overhead, but already have
 some in my code. If DUnit offers comparable overhead, and enhanced
 functionality, I'll switch.

DUnit uses test fixture classes, so it probably won't suit you at present. I'll look into a method with less overhead.

Hmmm... maybe :( Would this be possible? unittest{ mixin DUnitTest!("My test name", { ...// my code }); } My limit is probably something near: unittest{ mixin startTest!("My test name"); try{ ... // my code catch{ mixin catchTest; } } I'd like to stay with the D style of unit tests. Maybe I should check what enhancement requests are in there. It'd be nice to be able to hook in a unit test handler...
Mar 26 2008
next sibling parent reply Sean Kelly <sean invisibleduck.org> writes:
== Quote from Jason House (jason.james.house gmail.com)'s article
 I'd like to stay with the D style of unit tests.  Maybe I should check what
 enhancement requests are in there.  It'd be nice to be able to hook in a
 unit test handler...

Tango has one. Look at "moduleUnitTester" here: http://www.dsource.org/projects/tango/docs/current/tango.core.Runtime.html Sean
Mar 26 2008
parent reply Jason House <jason.james.house gmail.com> writes:
Sean Kelly wrote:

 == Quote from Jason House (jason.james.house gmail.com)'s article
 I'd like to stay with the D style of unit tests.  Maybe I should check
 what
 enhancement requests are in there.  It'd be nice to be able to hook in a
 unit test handler...

Tango has one. Look at "moduleUnitTester" here: http://www.dsource.org/projects/tango/docs/current/tango.core.Runtime.html Sean

See http://d.puremagic.com/issues/show_bug.cgi?id=1952 for the enhancement request. Per unit test customization (such as a unit test name) is very useful to have. I also don't see a way to tell the program to run all unit tests (and just report the individual passes/failures)
Mar 26 2008
next sibling parent reply Lars Ivar Igesund <larsivar igesund.net> writes:
Jason House Wrote:

 Sean Kelly wrote:
 
 == Quote from Jason House (jason.james.house gmail.com)'s article
 I'd like to stay with the D style of unit tests.  Maybe I should check
 what
 enhancement requests are in there.  It'd be nice to be able to hook in a
 unit test handler...

Tango has one. Look at "moduleUnitTester" here: http://www.dsource.org/projects/tango/docs/current/tango.core.Runtime.html Sean

See http://d.puremagic.com/issues/show_bug.cgi?id=1952 for the enhancement request. Per unit test customization (such as a unit test name) is very useful to have. I also don't see a way to tell the program to run all unit tests (and just report the individual passes/failures)

Here is a simple program that uses the above hook - note that you need to link one of the stacktrace libs (jive on linux) to actually get stacktraces printed on exceptions, not just assert line numbers. Also, I think it will be unable to continue after a segfault.: module myunittestrunner; import tango.io.Stdout; import tango.core.Runtime; bool tangoUnitTester() { uint count = 0; Stdout ("NOTE: This is still fairly rudimentary, and will only report the").newline; Stdout (" first error per module.").newline; foreach ( m; ModuleInfo ) // moduleinfo array { if ( m.unitTest) { Stdout.format ("{}. Executing unittests in '{}' ", count, m.name); try { m.unitTest(); } catch (Exception e) { Stdout(" - Unittest failed.").newline; Stdout.format(" File '{}', line '{}'.", e.file, e.line).newline; Stdout.format(" Message is : '{}'", e.msg).newline; if (e.info) Stdout.format(" TraceInfo: {}", e.info.toString).newline; continue; } Stdout(" - Success.").newline; count++; } } return true; } static this() { Runtime.moduleUnitTester( &tangoUnitTester ); } void main() {}
Mar 27 2008
parent reply Jason House <jason.james.house gmail.com> writes:
Lars Ivar Igesund Wrote:

 Jason House Wrote:
 
 Sean Kelly wrote:
 
 == Quote from Jason House (jason.james.house gmail.com)'s article
 I'd like to stay with the D style of unit tests.  Maybe I should check
 what
 enhancement requests are in there.  It'd be nice to be able to hook in a
 unit test handler...

Tango has one. Look at "moduleUnitTester" here: http://www.dsource.org/projects/tango/docs/current/tango.core.Runtime.html Sean

See http://d.puremagic.com/issues/show_bug.cgi?id=1952 for the enhancement request. Per unit test customization (such as a unit test name) is very useful to have. I also don't see a way to tell the program to run all unit tests (and just report the individual passes/failures)

Here is a simple program that uses the above hook - note that you need to link one of the stacktrace libs (jive on linux) to actually get stacktraces printed on exceptions, not just assert line numbers. Also, I think it will be unable to continue after a segfault.:

Hmmm... There's some black magic in your sample. What's "ModuleInfo"? Where is it documented? The code looks like it does not run unit tests one at a time, but I'll wait for real analysis of the code until I know what's going on.
Mar 27 2008
parent reply Sean Kelly <sean invisibleduck.org> writes:
== Quote from Jason House (jason.james.house gmail.com)'s article
 Lars Ivar Igesund Wrote:
 Jason House Wrote:

 Sean Kelly wrote:

 == Quote from Jason House (jason.james.house gmail.com)'s article
 I'd like to stay with the D style of unit tests.  Maybe I should check
 what
 enhancement requests are in there.  It'd be nice to be able to hook in a
 unit test handler...

Tango has one. Look at "moduleUnitTester" here: http://www.dsource.org/projects/tango/docs/current/tango.core.Runtime.html Sean

See http://d.puremagic.com/issues/show_bug.cgi?id=1952 for the enhancement request. Per unit test customization (such as a unit test name) is very useful to have. I also don't see a way to tell the program to run all unit tests (and just report the individual passes/failures)

Here is a simple program that uses the above hook - note that you need to link one of the stacktrace libs (jive on linux) to actually


 Hmmm...  There's some black magic in your sample.  What's "ModuleInfo"?  Where
is it documented?

ModuleInfo is declared in the global object.di in Tango, and it isn't documented anywhere. Sorry about that. I need to look into documenting what's in object.di.
 The code looks like it does not run unit tests one at a time, but I'll wait
for real analysis of the code until I know what's going on.

Unfortunately, the compiler lumps all the unit tests in a particular module together. I would prefer if they were split out into a per- module list however, and this would have to happen for your enhancement request to go through. Sean
Mar 27 2008
parent reply Jason House <jason.james.house gmail.com> writes:
Sean Kelly wrote:

 == Quote from Jason House (jason.james.house gmail.com)'s article
 Lars Ivar Igesund Wrote:
 Jason House Wrote:

 Sean Kelly wrote:

 == Quote from Jason House (jason.james.house gmail.com)'s article
 I'd like to stay with the D style of unit tests.  Maybe I should
 check what
 enhancement requests are in there.  It'd be nice to be able to
 hook in a unit test handler...

Tango has one. Look at "moduleUnitTester" here:





 Sean

See http://d.puremagic.com/issues/show_bug.cgi?id=1952 for the enhancement request. Per unit test customization (such as a unit test name) is very useful to have. I also don't see a way to tell the program to run all unit tests (and just report the individual passes/failures)

Here is a simple program that uses the above hook - note that you need to link one of the stacktrace libs (jive on linux) to actually


I think it will be unable to continue after a segfault.:
 Hmmm...  There's some black magic in your sample.  What's "ModuleInfo"? 
 Where is it documented?

ModuleInfo is declared in the global object.di in Tango, and it isn't documented anywhere. Sorry about that. I need to look into documenting what's in object.di.
 The code looks like it does not run unit tests one at a time, but I'll
 wait for real analysis of the code until I know what's going on.

Unfortunately, the compiler lumps all the unit tests in a particular module together. I would prefer if they were split out into a per- module list however, and this would have to happen for your enhancement request to go through. Sean

Please, please add the extra detail to the request! How should the compiler do it?
Mar 27 2008
parent reply Sean Kelly <sean invisibleduck.org> writes:
== Quote from Jason House (jason.james.house gmail.com)'s article
 Sean Kelly wrote:
 == Quote from Jason House (jason.james.house gmail.com)'s article
 Lars Ivar Igesund Wrote:
 Jason House Wrote:

 Sean Kelly wrote:

 == Quote from Jason House (jason.james.house gmail.com)'s article
 I'd like to stay with the D style of unit tests.  Maybe I should
 check what
 enhancement requests are in there.  It'd be nice to be able to
 hook in a unit test handler...

Tango has one. Look at "moduleUnitTester" here:





 Sean

See http://d.puremagic.com/issues/show_bug.cgi?id=1952 for the enhancement request. Per unit test customization (such as a unit test name) is very useful to have. I also don't see a way to tell the program to run all unit tests (and just report the individual passes/failures)

Here is a simple program that uses the above hook - note that you need to link one of the stacktrace libs (jive on linux) to actually


I think it will be unable to continue after a segfault.:
 Hmmm...  There's some black magic in your sample.  What's "ModuleInfo"?
 Where is it documented?

ModuleInfo is declared in the global object.di in Tango, and it isn't documented anywhere. Sorry about that. I need to look into documenting what's in object.di.
 The code looks like it does not run unit tests one at a time, but I'll
 wait for real analysis of the code until I know what's going on.

Unfortunately, the compiler lumps all the unit tests in a particular module together. I would prefer if they were split out into a per- module list however, and this would have to happen for your enhancement request to go through. Sean

do it?

I'd like ModuleInfo to get something vaguely like this: alias void function() UnitTest; struct NamedPair { char[] name; UnitTest test; } NamedPair[] unitTests; Then executing the unit tests for a module could be done on a per-test basis, with the names you suggested visible as well. Of course, this means even more meta-data per module, but it would make unittest more flexible so... Sean
Mar 27 2008
parent Jason House <jason.james.house gmail.com> writes:
Sean Kelly wrote:

 == Quote from Jason House (jason.james.house gmail.com)'s article
 Sean Kelly wrote:
 == Quote from Jason House (jason.james.house gmail.com)'s article
 Lars Ivar Igesund Wrote:
 Jason House Wrote:

 Sean Kelly wrote:

 == Quote from Jason House (jason.james.house gmail.com)'s
 article
 I'd like to stay with the D style of unit tests.  Maybe I
 should check what
 enhancement requests are in there.  It'd be nice to be able to
 hook in a unit test handler...

Tango has one. Look at "moduleUnitTester" here:







 Sean

See http://d.puremagic.com/issues/show_bug.cgi?id=1952 for the enhancement request. Per unit test customization (such as a unit test name) is very useful to have. I also don't see a way to tell the program to run all unit tests (and just report the individual passes/failures)

Here is a simple program that uses the above hook - note that you need to link one of the stacktrace libs (jive on linux) to actually


Also, I think it will be unable to continue after a segfault.:
 Hmmm...  There's some black magic in your sample.  What's
 "ModuleInfo"? Where is it documented?

ModuleInfo is declared in the global object.di in Tango, and it isn't documented anywhere. Sorry about that. I need to look into documenting what's in object.di.
 The code looks like it does not run unit tests one at a time, but I'll
 wait for real analysis of the code until I know what's going on.

Unfortunately, the compiler lumps all the unit tests in a particular module together. I would prefer if they were split out into a per- module list however, and this would have to happen for your enhancement request to go through. Sean

compiler do it?

I'd like ModuleInfo to get something vaguely like this: alias void function() UnitTest; struct NamedPair { char[] name; UnitTest test; } NamedPair[] unitTests; Then executing the unit tests for a module could be done on a per-test basis, with the names you suggested visible as well. Of course, this means even more meta-data per module, but it would make unittest more flexible so... Sean

I was hoping those kind of thoughts would go into the feature request ;)
Mar 27 2008
prev sibling parent Sean Kelly <sean invisibleduck.org> writes:
== Quote from Jason House (jason.james.house gmail.com)'s article
 Sean Kelly wrote:
 == Quote from Jason House (jason.james.house gmail.com)'s article
 I'd like to stay with the D style of unit tests.  Maybe I should check
 what
 enhancement requests are in there.  It'd be nice to be able to hook in a
 unit test handler...

Tango has one. Look at "moduleUnitTester" here: http://www.dsource.org/projects/tango/docs/current/tango.core.Runtime.html

request. Per unit test customization (such as a unit test name) is very useful to have. I also don't see a way to tell the program to run all unit tests (and just report the individual passes/failures)

Yes, there is no way to name the built-in unit tests. That would be a nice feature to have, so long as the name were optional. I've been asking for unittest blocks to get a more function-like syntax anyway, to match invariant in D 2.0, so this provides a good reason for the syntax. As for running all unit tests, simply have your unit tester perform each unit test in a try block/catch block. If an exception is thrown then the test failed, otherwise it succeeded. Sean
Mar 27 2008
prev sibling parent reply Christopher Wright <dhasenan gmail.com> writes:
Jason House wrote:
 Christopher Wright wrote:
 
 Jason House wrote:
 I'm sensitive to per-unittest code-writing overhead, but already have
 some in my code. If DUnit offers comparable overhead, and enhanced
 functionality, I'll switch.

present. I'll look into a method with less overhead.

Hmmm... maybe :( Would this be possible? unittest{ mixin DUnitTest!("My test name", { ...// my code }); }

Yes. I initially had some issues with alias parameters and delegates, but those seem to have disappeared. You wouldn't want to put that in a unittest block, though, since it's redundant and would give me, and by extension you, less control. You'd also need to either version out your main function or call dunit_main from yours when running tests.
Mar 26 2008
parent reply Jason House <jason.james.house gmail.com> writes:
Christopher Wright wrote:

 Jason House wrote:
 Christopher Wright wrote:
 
 Jason House wrote:
 I'm sensitive to per-unittest code-writing overhead, but already have
 some in my code. If DUnit offers comparable overhead, and enhanced
 functionality, I'll switch.

present. I'll look into a method with less overhead.

Hmmm... maybe :( Would this be possible? unittest{ mixin DUnitTest!("My test name", { ...// my code }); }

Yes.

Cool :)
 I initially had some issues with alias parameters and delegates, 
 but those seem to have disappeared. You wouldn't want to put that in a
 unittest block, though, since it's redundant and would give me, and by
 extension you, less control.

Huh? Can you explain? Would I have less control than DUnit currently provides or less control thant just a hand made unittest{...}? In either case, how severe of a gap are you envisioning? (So far, D unittest blocks are my only real unit testing experience. Work will eventually have me using NUnit, but that hasn't happened yet)
 You'd also need to either version out your main function or call
 dunit_main from yours when running tests.

I'm totally ok with that! One-time overhead at a program level isn't too bad. Most programs that really use unit tests are moderate or large in size anyway. It's the per-test overhead that would really irk me as I code away.
Mar 27 2008
parent Christopher Wright <dhasenan gmail.com> writes:
Jason House wrote:
 Christopher Wright wrote:
 
 Jason House wrote:
 Christopher Wright wrote:

 Jason House wrote:
 I'm sensitive to per-unittest code-writing overhead, but already have
 some in my code. If DUnit offers comparable overhead, and enhanced
 functionality, I'll switch.

present. I'll look into a method with less overhead.

Would this be possible? unittest{ mixin DUnitTest!("My test name", { ...// my code }); }


Cool :)
 I initially had some issues with alias parameters and delegates, 
 but those seem to have disappeared. You wouldn't want to put that in a
 unittest block, though, since it's redundant and would give me, and by
 extension you, less control.

Huh? Can you explain? Would I have less control than DUnit currently provides or less control thant just a hand made unittest{...}? In either case, how severe of a gap are you envisioning? (So far, D unittest blocks are my only real unit testing experience. Work will eventually have me using NUnit, but that hasn't happened yet)

Currently DUnit does a lot of work with static constructors, and I'm not sure how they interact with unittest blocks. If DUnit used unittest blocks instead of delegates and static constructors, that would be less control, since you'd only get two benefits: named tests and continuing testing after one failure. Well, three, actually; I could also do some aggregate reports and such, if you used a main() that I supplied. If I add some feature later that lets you specify which tests to run via commandline options, those wouldn't be available with a unittest{} wrapper. Anything that affects the actual running of the tests and not the accounting.
 You'd also need to either version out your main function or call
 dunit_main from yours when running tests.

I'm totally ok with that! One-time overhead at a program level isn't too bad. Most programs that really use unit tests are moderate or large in size anyway. It's the per-test overhead that would really irk me as I code away.

Of course, you can make editor macros to take most of the pain away.
Mar 27 2008
prev sibling parent Christopher Wright <dhasenan gmail.com> writes:
Christopher Wright wrote:
 Jason House wrote:
 Does anyone have a setup for doing continuous integration with d?

I'm going to start on that now, since DUnit (latest svn) can output vaguely junit-compatible xml. It'll be using CruiseControl. By tomorrow, I'll be able to tell you when you should start bothering me about it in earnest.

Okay, assuming you are using dsss, you can simply use: <schedule interval="60"> <exec workingdir="projects/${project.name}" command="/usr/local/bin/dsss" args="build" timeout="5000" /> </schedule> If you have a binary that includes your tests, you can change that to: <schedule interval="60"> <composite> <exec workingdir="projects/${project.name}" command="/usr/local/bin/dsss" args="build" timeout="5000" /> <exec workingdir="projects/${project.name}" command="./binary" args="some-args" /> </composite> </schedule> If you don't use <composite>, well, that's entirely allowed, but only the first element will be executed. I don't have xml output for dunit working correctly with cruisecontrol yet.
Mar 26 2008
prev sibling parent Sergey Pashin <spashin viewtier.com> writes:
Jason House Wrote:

 Does anyone have a setup for doing continuous integration with d?

Jason, You might check out our Parabuild. It is free for open source development. And it will run any build or test that you can run from the command line. Sergey Pashin
Mar 26 2008