digitalmars.D.announce - dunit 0.7.0 released
- linkrope (26/26) Sep 29 2013 https://github.com/linkrope/dunit/tree/v0.7.0
- ilya-stromberg (3/6) Oct 21 2013 Guys, we have at least 5 (!) different unit test projects!
- Dicebot (2/4) Oct 21 2013 ...and add it to Phobos review queue ;)
- Jonathan M Davis (8/13) Oct 21 2013 I confess that I don't understand why anyone is creating any unit test
- Dicebot (12/23) Oct 21 2013 Sorry but it feels like you didn't really investigate the topic.
- Jonathan M Davis (17/43) Oct 21 2013 I know that you can extend the built-in facilities by overriding how ass...
- Dicebot (13/30) Oct 21 2013 Overriding assert is dangerous because changes behavior of
- ilya-stromberg (9/28) Oct 21 2013 Yes, you are right if we talk about SIMPLE unit tests.
- Gary Willoughby (33/52) Oct 21 2013 "640K ought to be enough for anybody"
- ilya-stromberg (2/57) Oct 21 2013 +1
- Marco Leise (11/75) Nov 05 2013 We had a discussion about verbose contract assertion failures
- Meta (8/20) Oct 22 2013 I have to almost completely disagree with you here. I often find
- Russel Winder (21/27) Oct 21 2013 The focus on unit testing is a problem for me, unit testing is but ⅓ o...
- qznc (6/10) Oct 21 2013 Somewhat off topic, but out of curiosity: How do you distinguish
- Russel Winder (18/23) Oct 21 2013 Integration testing is when you test part or all of the system but not
- Dicebot (7/10) Oct 21 2013 There is a certain terminology issue here as system testing may
https://github.com/linkrope/dunit/tree/v0.7.0 The xUnit testing framework for D is used in production for one year now. The latest changes are: - added XML test reporting in JUnitReport format - dub support - changed Ignore to Ignore("reason to skip the test") - added assertOp together with corresponding aliases - added assertEmpty and assertNotEmpty - extended assertArrayEquals to associative arrays The XML reporting is currently used for a good integration with the CI tool Jenkins. While the dunit/framework is best suited for "testing in the large", dunit/assertion can be used on its own in simple unittest blocks for improved failure messages: unittest { import dunit.assertion; assertArrayEquals([1: "foo", 2: "bar"], [1: "foo", 2: "baz"]); } You will get the following message: array mismatch at key 2; expected: <ba<r>> but was: <ba<z>> Even the difference between the string values "bar" and "baz" is highlighted within <...>. (Still, I would wish for something like Python's ndiff for multi-line string values.)
Sep 29 2013
On Sunday, 29 September 2013 at 21:06:16 UTC, linkrope wrote:https://github.com/linkrope/dunit/tree/v0.7.0 The xUnit testing framework for D is used in production for one year now.Guys, we have at least 5 (!) different unit test projects! Can you cooperate your efforts to create one, but wonderful?
Oct 21 2013
On Monday, 21 October 2013 at 11:09:25 UTC, ilya-stromberg wrote:Guys, we have at least 5 (!) different unit test projects! Can you cooperate your efforts to create one, but wonderful?...and add it to Phobos review queue ;)
Oct 21 2013
On Monday, October 21, 2013 13:48:16 Dicebot wrote:On Monday, 21 October 2013 at 11:09:25 UTC, ilya-stromberg wrote:I confess that I don't understand why anyone is creating any unit test projects for D, and I'd likely vote against any attempt to add such a thing to Phobos. D has built in unit testing functionality, and it works great. Maybe some additional assert-like functions could be useful (similar to assertThrown or assertNotThrown), but we really don't need much beyond what the language provides. - Jonathan m DavisGuys, we have at least 5 (!) different unit test projects! Can you cooperate your efforts to create one, but wonderful?...and add it to Phobos review queue ;)
Oct 21 2013
On Monday, 21 October 2013 at 11:58:14 UTC, Jonathan M Davis wrote:I confess that I don't understand why anyone is creating any unit test projects for D, and I'd likely vote against any attempt to add such a thing to Phobos. D has built in unit testing functionality, and it works great. Maybe some additional assert-like functions could be useful (similar to assertThrown or assertNotThrown), but we really don't need much beyond what the language provides.Sorry but it feels like you didn't really investigate the topic. D built-in unit-test facilities are a basic and extendable framework. Many notable unittest libraries work on top of it (using custom assert helpers and/or __traits(getUnittests)) being at the same time perfectly compatible with raw `dmd -unittest -main -run file.d`. At the very same time built-in reporting facilities are very limited and almost any bigger project will want some extension. Assuming full compatibility with built-in ones and zero external dependencies I don't see why this can't go to Phobos.
Oct 21 2013
On Monday, October 21, 2013 14:10:13 Dicebot wrote:On Monday, 21 October 2013 at 11:58:14 UTC, Jonathan M Davis wrote:I know that you can extend the built-in facilities by overriding how assert works and the like. I also see no reason to do so. IIRC, such facilities were even removed at one point, because no one was using them. They were readded after someone complained about wanting them, but every time I see anyone talking about doing anything with those, it seems like overkill to me. And since it's frequently for nonsense like making it so that the tests continue after an assertion fails (which is outright bad practice IMHO), I have a very low opinion of attempts to override the built-in assert for unit tests. If someone wants to present something to the review queue that's related to unit tests, that's fine. But I really don't see any problem with the built-in unit tests facilities and would expect to be against any such submission, particularly since every time I've delved into any discussions on them, what folks are doing with them has seemed like a really bad idea to me. But I guess that we'll just have to wait and see what gets submitted for review, if anything. - Jonathan M DavisI confess that I don't understand why anyone is creating any unit test projects for D, and I'd likely vote against any attempt to add such a thing to Phobos. D has built in unit testing functionality, and it works great. Maybe some additional assert-like functions could be useful (similar to assertThrown or assertNotThrown), but we really don't need much beyond what the language provides.Sorry but it feels like you didn't really investigate the topic. D built-in unit-test facilities are a basic and extendable framework. Many notable unittest libraries work on top of it (using custom assert helpers and/or __traits(getUnittests)) being at the same time perfectly compatible with raw `dmd -unittest -main -run file.d`. At the very same time built-in reporting facilities are very limited and almost any bigger project will want some extension. Assuming full compatibility with built-in ones and zero external dependencies I don't see why this can't go to Phobos.
Oct 21 2013
"Dicebot" <public dicebot.lv> On Monday, 21 October 2013 at 12:24:18 UTC, Jonathan M Davis wrote:I know that you can extend the built-in facilities by overriding how assert works and the like.Overriding assert is dangerous because changes behavior of program itself and lacks context data. Own test runner implemented using __traits(getUnittests) is perfectly safe.And since it's frequently for nonsense like making it so that the tests continue after an assertion fails (which is outright bad practice IMHO),It is absolutely necessary feature in any big project of you want to keep reasonable edit-build cycle times. Tests are always hierarchical naturally, there is no reason to stop the world if completely unrelated ones fail.I have a very low opinion of attempts to override the built-in assert for unit tests.You are right here but it is not needed anymore.But I really don't see any problem with the built-in unit tests facilities and would expect to be against any such submission,There is nothing wrong with built-in ones, just some higher-level tools on top of them are lacking.But I guess that we'll just have to wait and see what gets submitted for review, if anything.Exactly, lets argue in time ;)
Oct 21 2013
On Monday, 21 October 2013 at 11:58:14 UTC, Jonathan M Davis wrote:On Monday, October 21, 2013 13:48:16 Dicebot wrote:Yes, you are right if we talk about SIMPLE unit tests. For more complex cases like integration tests you have A LOT of problems. It isn't only my opinion - for example, look at the "Higgs, an Experimental JIT Compiler written in D" Slides from last dconf (page 57): http://dconf.org/2013/talks/chevalier_boisvert.htmlOn Monday, 21 October 2013 at 11:09:25 UTC, ilya-stromberg wrote:I confess that I don't understand why anyone is creating any unit test projects for D, and I'd likely vote against any attempt to add such a thing to Phobos. D has built in unit testing functionality, and it works great. Maybe some additional assert-like functions could be useful (similar to assertThrown or assertNotThrown), but we really don't need much beyond what the language provides. - Jonathan m DavisGuys, we have at least 5 (!) different unit test projects! Can you cooperate your efforts to create one, but wonderful?...and add it to Phobos review queue ;)
Oct 21 2013
On Monday, 21 October 2013 at 11:58:14 UTC, Jonathan M Davis wrote:On Monday, October 21, 2013 13:48:16 Dicebot wrote:"640K ought to be enough for anybody" Seriously though, unit testing is a major tool that all languages should have access to. The built-in stuff is adequate for very simple testing on simple types such as asserting something is true, etc. But when you're writing tests for a class that need dependencies then the built-in stuff won't cut it. The framework i'm currently working allows for mocking of dependencies which in itself is a big deal. Replacing dependencies with 'fake' stubs is something which makes unit tests a lot clearer and less complicated. It also means i can override the return value of any method at runtime to pretend returned data is real there too. I've also extended the assert mechanism in the D runtime to create better errors. Instead of just getting: core.exception.AssertError file(57): Assertion failure You now get: +---------------------------------------------------------------------- | Failed asserting equal +---------------------------------------------------------------------- | File: file.d | Line: 57 +---------------------------------------------------------------------- | ✓ Expected value: (int[]) [1, 2, 3, 4, 5] | ✗ Actual value: (int[]) [1, 2, 4, 5] This is way more clear exactly what and where failed. This is essential when unit testing using structs, arrays, classes, etc.. as you need this information to truly narrow down exactly what failed. Extending stuff like this also helps with overall project reporting, writing to a file or drawing pretty graphs etc..On Monday, 21 October 2013 at 11:09:25 UTC, ilya-stromberg wrote:I confess that I don't understand why anyone is creating any unit test projects for D, and I'd likely vote against any attempt to add such a thing to Phobos. D has built in unit testing functionality, and it works great. Maybe some additional assert-like functions could be useful (similar to assertThrown or assertNotThrown), but we really don't need much beyond what the language provides. - Jonathan m DavisGuys, we have at least 5 (!) different unit test projects! Can you cooperate your efforts to create one, but wonderful?...and add it to Phobos review queue ;)
Oct 21 2013
On Monday, 21 October 2013 at 19:02:45 UTC, Gary Willoughby wrote:On Monday, 21 October 2013 at 11:58:14 UTC, Jonathan M Davis wrote:+1On Monday, October 21, 2013 13:48:16 Dicebot wrote:Seriously though, unit testing is a major tool that all languages should have access to. The built-in stuff is adequate for very simple testing on simple types such as asserting something is true, etc. But when you're writing tests for a class that need dependencies then the built-in stuff won't cut it. The framework i'm currently working allows for mocking of dependencies which in itself is a big deal. Replacing dependencies with 'fake' stubs is something which makes unit tests a lot clearer and less complicated. It also means i can override the return value of any method at runtime to pretend returned data is real there too. I've also extended the assert mechanism in the D runtime to create better errors. Instead of just getting: core.exception.AssertError file(57): Assertion failure You now get: +---------------------------------------------------------------------- | Failed asserting equal +---------------------------------------------------------------------- | File: file.d | Line: 57 +---------------------------------------------------------------------- | ✓ Expected value: (int[]) [1, 2, 3, 4, 5] | ✗ Actual value: (int[]) [1, 2, 4, 5] This is way more clear exactly what and where failed. This is essential when unit testing using structs, arrays, classes, etc.. as you need this information to truly narrow down exactly what failed. Extending stuff like this also helps with overall project reporting, writing to a file or drawing pretty graphs etc..On Monday, 21 October 2013 at 11:09:25 UTC, ilya-stromberg wrote:I confess that I don't understand why anyone is creating any unit test projects for D, and I'd likely vote against any attempt to add such a thing to Phobos. D has built in unit testing functionality, and it works great. Maybe some additional assert-like functions could be useful (similar to assertThrown or assertNotThrown), but we really don't need much beyond what the language provides. - Jonathan m DavisGuys, we have at least 5 (!) different unit test projects! Can you cooperate your efforts to create one, but wonderful?...and add it to Phobos review queue ;)
Oct 21 2013
Am Mon, 21 Oct 2013 21:47:22 +0200 schrieb "ilya-stromberg" <ilya-stromberg-2009 yandex.ru>:On Monday, 21 October 2013 at 19:02:45 UTC, Gary Willoughby wrote:We had a discussion about verbose contract assertion failures a while ago. I did something similar, with an API like that: ensure!"=3D=3D"(x, [1, 2, 3, 4, 5], "x isn't the ordered list of integers f= rom 1 to 5"); It has its shortcomings though, especially when you need x to be one of a set of values or a power of 2 etc... Btw: Is =E2=9C=93 and =E2=9C=97 supported on Windows Vista and XP ? --=20 MarcoOn Monday, 21 October 2013 at 11:58:14 UTC, Jonathan M Davis=20 wrote:=20 +1On Monday, October 21, 2013 13:48:16 Dicebot wrote:Seriously though, unit testing is a major tool that all=20 languages should have access to. The built-in stuff is adequate=20 for very simple testing on simple types such as asserting=20 something is true, etc. But when you're writing tests for a=20 class that need dependencies then the built-in stuff won't cut=20 it. The framework i'm currently working allows for mocking of=20 dependencies which in itself is a big deal. Replacing=20 dependencies with 'fake' stubs is something which makes unit=20 tests a lot clearer and less complicated. It also means i can=20 override the return value of any method at runtime to pretend=20 returned data is real there too. I've also extended the assert mechanism in the D runtime to=20 create better errors. Instead of just getting: core.exception.AssertError file(57): Assertion failure You now get: =20 +---------------------------------------------------------------------- | Failed asserting equal =20 +---------------------------------------------------------------------- | File: file.d | Line: 57 =20 +---------------------------------------------------------------------- | =E2=9C=93 Expected value: (int[]) [1, 2, 3, 4, 5] | =E2=9C=97 Actual value: (int[]) [1, 2, 4, 5] This is way more clear exactly what and where failed. This is=20 essential when unit testing using structs, arrays, classes,=20 etc.. as you need this information to truly narrow down exactly=20 what failed. Extending stuff like this also helps with overall=20 project reporting, writing to a file or drawing pretty graphs=20 etc..On Monday, 21 October 2013 at 11:09:25 UTC, ilya-stromberg=20 wrote:I confess that I don't understand why anyone is creating any=20 unit test projects for D, and I'd likely vote against any attempt to add=20 such a thing to Phobos. D has built in unit testing functionality, and it=20 works great. Maybe some additional assert-like functions could be useful (similar=20 to assertThrown or assertNotThrown), but we really don't need much beyond what=20 the language provides. - Jonathan m DavisGuys, we have at least 5 (!) different unit test projects! Can you cooperate your efforts to create one, but wonderful?=20 ...and add it to Phobos review queue ;)
Nov 05 2013
On Monday, 21 October 2013 at 11:58:14 UTC, Jonathan M Davis wrote:I confess that I don't understand why anyone is creating any unit test projects for D, and I'd likely vote against any attempt to add such a thing to Phobos. D has built in unit testing functionality, and it works great. Maybe some additional assert-like functions could be useful (similar to assertThrown or assertNotThrown), but we really don't need much beyond what the language provides. - Jonathan m DavisI have to almost completely disagree with you here. I often find myself rolling my own small testing extensions for the built in unittests, and a standard solution is sorely needed. The only thing I agree with you on is that any such solution would probably be better off not overriding assertion behaviour, but as Dicebot said, that's no longer necessary.
Oct 22 2013
On Mon, 2013-10-21 at 04:58 -0700, Jonathan M Davis wrote: […]I confess that I don't understand why anyone is creating any unit test projects for D, and I'd likely vote against any attempt to add such a thing to Phobos. D has built in unit testing functionality, and it works great. Maybe some additional assert-like functions could be useful (similar to assertThrown or assertNotThrown), but we really don't need much beyond what the language provides.The focus on unit testing is a problem for me, unit testing is but ⅓ of the testing needed. There is integration testing and system testing to consider as well. Not to mention the different features needed of BDD rather than TDD. I appreciate that many people on this list think of anything to do with the JVM beneath contempt ;-) but the JUnit → TestNG → Spock and Cucumber journey is worth considering as a lesson in why not to get too focused on programmer generation of units. Python has the same lessons with the way unittest (aka PyUnit) and py.test are used. It is all about a testing framework supporting unit test, integration test, and system test usage. See Catch for the current C++ front-runner. https://github.com/philsquared/Catch -- Russel. ============================================================================= Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.net 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Oct 21 2013
On Monday, 21 October 2013 at 13:45:06 UTC, Russel Winder wrote:It is all about a testing framework supporting unit test, integration test, and system test usage.Somewhat off topic, but out of curiosity: How do you distinguish between integration and system testing? The descriptions I found usually sound like system testing is a special case of integration testing, where you simply integrate all components.
Oct 21 2013
On Mon, 2013-10-21 at 16:22 +0200, qznc wrote: […]Somewhat off topic, but out of curiosity: How do you distinguish between integration and system testing?Integration testing is when you test part or all of the system but not in a full deployment context (so possibly still using mocks for the database, network, user interface, etc.) whilst system testing is testing the whole system in a real, albeit test, context.The descriptions I found usually sound like system testing is a special case of integration testing, where you simply integrate all components.No, the difference is mostly in the context of execution of the system. Integration testing is about the interworking of the components and so you can mock out bits as needed to speed things up and drive things more deterministically. System testing allows no use of mocks and must occur in a full deployment setting, with the only compromise being that the deployment is in a sandbox isolated from the rest of the world. -- Russel. ============================================================================= Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.net 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Oct 21 2013
On Monday, 21 October 2013 at 14:22:31 UTC, qznc wrote:The descriptions I found usually sound like system testing is a special case of integration testing, where you simply integrate all components.There is a certain terminology issue here as system testing may apply to both certain program on its own (and thus come before integration one) and software platform as a whole (being essentially final integration test of all components). I personally favor hierarchical separation into unit -> functional -> integration.
Oct 21 2013