www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - [Issue 4747] New: Make each unittest block a separate function that runs independently of the others

reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=4747

           Summary: Make each unittest block a separate function that runs
                    independently of the others
           Product: D
           Version: D2
          Platform: Other
        OS/Version: Linux
            Status: NEW
          Severity: enhancement
          Priority: P2
         Component: DMD
        AssignedTo: nobody puremagic.com
        ReportedBy: jmdavisProg gmail.com


--- Comment #0 from Jonathan M Davis <jmdavisProg gmail.com> 2010-08-28
17:14:31 PDT ---
At the moment, each module has a single unittest function made up of all of the
unittest blocks in the module. This causes 2 problems:

1. A failure in one unittest block results in none of the others getting run,
even though they are independent in the code.

2. Stack traces from unit tests are useless because you have no way of knowing
_which_ unittest block failed.

By making each unittest block its own function, they can be run independently.
It also will help with stack traces, because each one would have its own name -
even it differs only by a number (since presumably, that's how unittest blocks
at the same scope would be given names by the compiler). At least then you
could count the unittest blocks to find it. Right now, there's no way to know.

Now, if you add that with named unit tests, e.g.

unittest (mytest)
{
}


then you have a function name for the unit test and the stack trace is properly
specific (though presumably unnamed unit tests would still use a numbered
naming scheme of some kind).

I know that an enhancement request which includes named unit tests for D1 was
reported a while back ( bug 2749 ) - though it also includes stuff like nesting
unit tests (which I think is pointless) - and I know that it has been
previously discussed that unit test blocks should be independent, but I'm
unaware of a bug report on it.

So, here's an enhancement request to make unittest blocks independent, and as
part of that, I would _strongly_ encourage that named unit tests be added as
well. That way stack traces are clear and useful. Right now, if you want to do
that, you have to create a function that contains your unit test and have the
unittest block call it, which is certainly cumbersome, and the potential
benefits of named unit tests go far beyond stack traces, so we really should
have them at some point. Both would help a lot with unit tests.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Aug 28 2010
next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=4747


jens.k.mueller gmx.de changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jens.k.mueller gmx.de


--- Comment #1 from jens.k.mueller gmx.de 2012-09-25 03:25:20 PDT ---
I'm not sure but isn't your second point more related to stack traces? I mean
if the stack trace would provide proper line numbers it was easy to find out
which unittest failed. Further this only applies if all you have is a stack
trace. That means if something was thrown you get the line number information.
If there is a segfault or similar (division by zero) you currently have to run
it under a debugger. I think this should be improved by providing better stack
traces.

Anyway the first point remains valid. unittests are often independent and it
should be possible to execute them independently.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 25 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=4747



--- Comment #2 from Jonathan M Davis <jmdavisProg gmx.com> 2012-09-25 03:51:40
PDT ---
It has to do with naming unittest blacks, which is related both to stack traces
and to calling unittest blocks independently.

Recently, it _was_ fixed so that the functions generated for unittest blocks
are named after the line number (and possibly file? - I don't know the exact
naming scheme), which reduces the problem with regards to stack traces, but if
you wanted to be able to call specific unittest blocks (say from a unit testing
tool), then having actual names for each unittest block makes that work much
better.

The main thing though is to fix the issue where it's all or nothing for running
a module's unit tests. When I created this request, I was under the incorrect
understanding that there was one function for all of a module's unittest
blocks, which isn't true, but it _is_ true that they can't be run independently
at present, and if I understand correctly, some dmd changes are required to fix
that.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 25 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=4747



--- Comment #3 from jens.k.mueller gmx.de 2012-09-25 04:04:14 PDT ---
But since unittests now have the line number in its name this issue is only
concerned having the ability to call individual unittests. Right? Because now
you can tell from the stack trace which unittest failed.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 25 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=4747



--- Comment #4 from Jonathan M Davis <jmdavisProg gmx.com> 2012-09-25 09:37:58
PDT ---
It makes it so that having named unittest blocks is less critical for stack
traces, but having actual names for them would still me quite a bit better (it
would also help for code organization, because it could document what the test
was for).

Regardless, the main point of this issue was always that the tests be
indivdually callable. Names for them just makes that a _lot_ cleaner, but
they'd have to be optional names regardless, because anything else would break
tons and tons of code.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 25 2012
prev sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=4747


Johannes Pfau <johannespfau gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |pull
                 CC|                            |johannespfau gmail.com


--- Comment #5 from Johannes Pfau <johannespfau gmail.com> 2012-09-25 09:47:13
PDT ---
Pull request: https://github.com/D-Programming-Language/dmd/pull/1131
(Allows to run unit tests individually. Also prepares name support, but full
support needs changes in lexer/parser)

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 25 2012