www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - phobos unit tests

reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
This document: https://wiki.dlang.org/Contributing_to_Phobos

States: "Avoid unittest in templates (it will generate a new unittest
for each instance) - put your tests outside"

Sounds reasonable, but then I realised that most of my unit tests are
documenting unittests... this recommendation is in conflict with the
documentation standards... who wins?
Sep 02 2016
next sibling parent Seb <seb wilzba.ch> writes:
On Saturday, 3 September 2016 at 03:36:06 UTC, Manu wrote:
 This document: https://wiki.dlang.org/Contributing_to_Phobos

 States: "Avoid unittest in templates (it will generate a new 
 unittest for each instance) - put your tests outside"

 Sounds reasonable, but then I realised that most of my unit 
 tests are documenting unittests... this recommendation is in 
 conflict with the documentation standards... who wins?
For the rationale see e.g. this thread: https://forum.dlang.org/post/wnydmelyxprvhsxewums forum.dlang.org A trick around this is sth. like: version(D_Ddoc) { enum doUnittest = true; } else { version(unittest) { enum doUnittest = true; } else { enum doUnittest = false; } } and then `static if (doUnittest)` the tests inside. For example ndslice uses a similar pattern.
Sep 03 2016
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 9/3/16 5:36 AM, Manu via Digitalmars-d wrote:
 This document: https://wiki.dlang.org/Contributing_to_Phobos

 States: "Avoid unittest in templates (it will generate a new unittest
 for each instance) - put your tests outside"
Actually that's a good thing, sometimes you do want to run a unittest for each instantiation.
 Sounds reasonable, but then I realised that most of my unit tests are
 documenting unittests... this recommendation is in conflict with the
 documentation standards... who wins?
Just version it only for the documentation. Andrei
Sep 03 2016
parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Saturday, September 03, 2016 15:06:33 Andrei Alexandrescu via Digitalmars-d 
wrote:
 On 9/3/16 5:36 AM, Manu via Digitalmars-d wrote:
 This document: https://wiki.dlang.org/Contributing_to_Phobos

 States: "Avoid unittest in templates (it will generate a new unittest
 for each instance) - put your tests outside"
Actually that's a good thing, sometimes you do want to run a unittest for each instantiation.
Sometimes, sure. But usually not.
 Sounds reasonable, but then I realised that most of my unit tests are
 documenting unittests... this recommendation is in conflict with the
 documentation standards... who wins?
Just version it only for the documentation.
Then the tests won't be run unless your documentation build is also your unittest build, and they frequently can't be the same - e.g. when something differs between operating systems, and you have a separate D_Ddoc or StdDdoc block so that you can have proper documentation without requiring that the documentation be built on all systems and potentially being able to document what exists on each system. For instance, the supported clock types vary quite a bit across operating systems, so the core.time.ClockType enum has different values on different systems and version(CoreDdoc) is used so that the documentation can list them all and list the consistently across systems. You could add some sort of version identifier that's specific to your build that you use for your unittest builds (similar to how Phobos has StdDdoc separate from D_Ddoc), but even if you do that, you still have the problem of the non-generic unit tests within a template being compiled into every instantiation of the template that's part of your unittest build, and all of those tests will get run, increasing both the build times and the time it takes to run the unit tests. - Jonathan M Davis
Sep 03 2016
parent reply ZombineDev <petar.p.kirov gmail.com> writes:
On Saturday, 3 September 2016 at 15:54:31 UTC, Jonathan M Davis 
wrote:
 On Saturday, September 03, 2016 15:06:33 Andrei Alexandrescu 
 via Digitalmars-d wrote:
 On 9/3/16 5:36 AM, Manu via Digitalmars-d wrote:
 This document: https://wiki.dlang.org/Contributing_to_Phobos

 States: "Avoid unittest in templates (it will generate a new 
 unittest for each instance) - put your tests outside"
Actually that's a good thing, sometimes you do want to run a unittest for each instantiation.
Sometimes, sure. But usually not.
 Sounds reasonable, but then I realised that most of my unit 
 tests are documenting unittests... this recommendation is in 
 conflict with the documentation standards... who wins?
Just version it only for the documentation.
Then the tests won't be run unless your documentation build is also your unittest build, and they frequently can't be the same - e.g. when something differs between operating systems, and you have a separate D_Ddoc or StdDdoc block so that you can have proper documentation without requiring that the documentation be built on all systems and potentially being able to document what exists on each system. For instance, the supported clock types vary quite a bit across operating systems, so the core.time.ClockType enum has different values on different systems and version(CoreDdoc) is used so that the documentation can list them all and list the consistently across systems. You could add some sort of version identifier that's specific to your build that you use for your unittest builds (similar to how Phobos has StdDdoc separate from D_Ddoc), but even if you do that, you still have the problem of the non-generic unit tests within a template being compiled into every instantiation of the template that's part of your unittest build, and all of those tests will get run, increasing both the build times and the time it takes to run the unit tests. - Jonathan M Davis
You can just enable the unittests for a single instance that you know for sure that it will be used. For example: 1) https://github.com/dlang/phobos/blob/v2.071.2-b3/std/experimental/ndslice/slice.d#L808 2) https://github.com/dlang/phobos/blob/v2.071.2-b3/std/experimental/ndslice/slice.d#L947
Sep 03 2016
parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Saturday, September 03, 2016 16:56:08 ZombineDev via Digitalmars-d wrote:
 You can just enable the unittests for a single instance that you
 know for sure that it will be used. For example:
 1)
 https://github.com/dlang/phobos/blob/v2.071.2-b3/std/experimental/ndslice/sl
 ice.d#L808

 2)
 https://github.com/dlang/phobos/blob/v2.071.2-b3/std/experimental/ndslice/sl
 ice.d#L947
So, in order to avoid having the unit tests compiled into the code of anyone who uses your template, you have to create a special version identifier that you use with your unit test build and your documentation build, and then you version the tests within the template with that version. And in order to avoid having the unit test compiled into every instantiation of that template during your test, you have to also put it in a static if for a specific instantiation? Sure, that's feasible, but that's getting _ugly_. - Jonathan M Davis
Sep 03 2016
parent ZombineDev <petar.p.kirov gmail.com> writes:
On Saturday, 3 September 2016 at 17:16:24 UTC, Jonathan M Davis 
wrote:
 On Saturday, September 03, 2016 16:56:08 ZombineDev via 
 Digitalmars-d wrote:
 You can just enable the unittests for a single instance that 
 you
 know for sure that it will be used. For example:
 1)
 https://github.com/dlang/phobos/blob/v2.071.2-b3/std/experimental/ndslice/sl
 ice.d#L808

 2) 
 https://github.com/dlang/phobos/blob/v2.071.2-b3/std/experimental/ndslice/sl
ice.d#L947
So, in order to avoid having the unit tests compiled into the code of anyone who uses your template, you have to create a special version identifier that you use with your unit test build and your documentation build, and then you version the tests within the template with that version. And in order to avoid having the unit test compiled into every instantiation of that template during your test, you have to also put it in a static if for a specific instantiation? Sure, that's feasible, but that's getting _ugly_. - Jonathan M Davis
I like and support your DIP about static unittests, I was just pointing out that there are workarounds, mainly for people like Manu, who need ways to get their job done now. Personally, I think that putting a single `static if (doUnittests)` before each unittest is not too much, considering how flexible the doUnittests condition can be (e.g. check if ddoc or unittest build, check template parameters, etc.).
Sep 03 2016