www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - [Issue 5091] New: main runs after unittests

reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=5091

           Summary: main runs after unittests
           Product: D
           Version: D1 & D2
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: enhancement
          Priority: P2
         Component: DMD
        AssignedTo: nobody puremagic.com
        ReportedBy: simen.kjaras gmail.com


--- Comment #0 from Simen Kjaeraas <simen.kjaras gmail.com> 2010-10-20 17:22:25
PDT ---
When compiling a program with dmd -unittest, after the unittests are run,
main() is called. This is rarely wanted behavior, so should not be the default.

Currently, it is possible to customize main to sidestep the problem:

void main( ) {
    version( unittest ) {
    } else {
        // Your program
    }
}

However, this is cluttering and mixes your application code with versioning
code.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Oct 20 2010
next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=5091


Peter Alexander <peter.alexander.au gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |peter.alexander.au gmail.co
                   |                            |m


--- Comment #1 from Peter Alexander <peter.alexander.au gmail.com> 2010-10-21
01:07:14 PDT ---
(In reply to comment #0)
 When compiling a program with dmd -unittest, after the unittests are run,
 main() is called. This is rarely wanted behavior, so should not be the default.
Why is this rarely wanted? If your way was the default then your work flow would be: - Make change - Run with unit test - Run without unit test - Make change - Run with unit test - Run without unit test - etc. The whole point of unit tests is that they are run regularly, preferably after ever change. If you only run them after lots of changes then it makes it much more difficult to find out what broke the build. If you have a lot of time-consuming unit tests in your code then these should be put inside a slowtest version block so that you can leave those for your nightly testing (or whatever you want to do). In general however, unittest should always be on. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Oct 21 2010
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=5091



--- Comment #2 from Peter Alexander <peter.alexander.au gmail.com> 2010-10-21
01:11:22 PDT ---
Even if you don't buy that argument, consider this: with the current unittest,
you only have to add a few lines of code to turn it off (as you described).
This is something that you'll only ever have to do once, and really isn't that
much of a burden.

On the other hand, if running main was off by default then there would be no
way to turn it back on! You would have to resort to scripting your build to get
it working again.

Therefore, the current unittest is unarguably more flexible to people's needs,
and arguably more desirable anyway.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Oct 21 2010
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=5091


bearophile_hugs eml.cc changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |bearophile_hugs eml.cc


--- Comment #3 from bearophile_hugs eml.cc 2010-10-21 03:19:14 PDT ---
(In reply to comment #1)
 Why is this rarely wanted?
D unittesting needs a global redesign, not small changes. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Oct 21 2010
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=5091



--- Comment #4 from Simen Kjaeraas <simen.kjaras gmail.com> 2010-10-21 10:32:36
PDT ---
(In reply to comment #1)
 (In reply to comment #0)
 When compiling a program with dmd -unittest, after the unittests are run,
 main() is called. This is rarely wanted behavior, so should not be the default.
Why is this rarely wanted?
Because I never want my program to run after I run my unittests? If the unittests pass, my program should be working, apart from things that are impossible/hard to unittest.
 If your way was the default then your work flow would be:
 
 - Make change
 - Run with unit test
 - Run without unit test
 - Make change
 - Run with unit test
 - Run without unit test
 - etc.
That's one possibility. When I program, I aim for such coverage with my unittests that running the program to see if things work is not necessary. Hence, a more likely scenario: 1 Make change 2 Run unittests (quick) 3 If unittests fail, goto 1 4 Run normally (slow) 5 Goto 1 I have not once wanted to run both the full application and the unittests at the same time. when I want to run the unittests, running the application would be a waste of time. When I want to run the program, I don't care about the unittests. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Oct 21 2010
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=5091



--- Comment #5 from Peter Alexander <peter.alexander.au gmail.com> 2010-10-22
00:43:56 PDT ---
(In reply to comment #4)
 I have not once wanted to run both the full application and the unittests at
 the same time. when I want to run the unittests, running the application would
 be a waste of time. When I want to run the program, I don't care about the
 unittests.
Well I have to say that this is totally alien to me. Putting our disagreement on exactly when unit tests should be run, we can at least agree on 3 things: 1. Some people want to run the program and unit tests at the same time, and others don't. 2. Leaving things as they are, you have the choice between the two (although one choice requires a few extra lines of code). 3. Changing things makes only one option possible. Unless I've missed anything, I think this is a clear argument for leaving things as they are, regardless of which choice we may be biased towards. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Oct 22 2010
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=5091


Andrej Mitrovic <andrej.mitrovich gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |andrej.mitrovich gmail.com


--- Comment #6 from Andrej Mitrovic <andrej.mitrovich gmail.com> 2013-01-07
17:14:36 PST ---
You can override this behavior by defining your own unittest function:

import core.runtime;
import std.stdio;
import std.string;
import std.path;

bool testRunner()
{
    foreach (m; ModuleInfo)
    {
        if (m is null)
            continue;

        if (auto fp = m.unitTest)
        {
            try
            {
                fp();
            }
            catch (Throwable e)
            {
                writeln(e);
            }
        }
    }

    // stop execution after tests are done
    return false;
}

shared static this()
{
    Runtime.moduleUnitTester(&testRunner);
}

unittest
{
    writeln("in unittest");
    assert(1);
}

void main()
{
    writeln("not in main");
}

--------------

I've written a similar test system where I can also set these at runtime via
getopt switches:

- specify which modules to run the tests on (--testMods=foo.bar)
- ditto but set modules which should *not* have their tests ran on
(--ignoreMods=foo.bar.doo)
- specify whether to run the unittests, and whether to run main() if the tests
were successful

I might write a DWiki entry later on how to do this if there's any interest.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jan 07 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=5091


Martin Nowak <code dawg.eu> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |code dawg.eu


--- Comment #7 from Martin Nowak <code dawg.eu> 2013-03-13 07:10:31 PDT ---
It's a little hacky though, because returning false is interpreted as failed
tests.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Mar 13 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=5091



--- Comment #8 from Andrej Mitrovic <andrej.mitrovich gmail.com> 2013-03-13
12:17:24 PDT ---
(In reply to comment #7)
 It's a little hacky though, because returning false is interpreted as failed
 tests.
Yeah, I've realized that after posting. Currently I use getOpt and a --runMain switch in my own code. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Mar 13 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=5091



--- Comment #9 from Andrej Mitrovic <andrej.mitrovich gmail.com> 2013-03-13
16:27:36 PDT ---
(In reply to comment #7)
 It's a little hacky though, because returning false is interpreted as failed
 tests.
Perhaps we could change ModuleUnitTester to return an int instead of a bool. Then 0 would mean tests failed, 1 would mean continue execution to main, and a new return (say 2) would mean don't execute main(). runModuleUnitTests would also have to return this status code, and then in dmain2.d we'd change: if (runModuleUnitTests()) to: if (runModuleUnitTests() == 1) Of course it might be best to use an enumeration for this: TestStatus { Fail, Ok, SkipMain } -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Mar 13 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=5091



--- Comment #10 from Martin Nowak <code dawg.eu> 2013-03-13 20:42:41 PDT ---
This always confused me a little because the documentation is inconsistent.
http://dlang.org/phobos/core_runtime.html#moduleUnitTester
http://dlang.org/phobos/core_runtime.html#runModuleUnitTests
One could also require that test failure is indicated by an exception.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Mar 13 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=5091



--- Comment #11 from Andrej Mitrovic <andrej.mitrovich gmail.com> 2013-03-13
20:49:45 PDT ---
(In reply to comment #10)
 This always confused me a little because the documentation is inconsistent.
 http://dlang.org/phobos/core_runtime.html#moduleUnitTester
 http://dlang.org/phobos/core_runtime.html#runModuleUnitTests
Yes me too.
 One could also require that test failure is indicated by an exception.
Speaking of which I just noticed that `runAll` doesn't wrap `runModuleUnitTests()` in a try/catch. That seems like a bug to me. rt_init does however. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Mar 13 2013
prev sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=5091



--- Comment #12 from Martin Nowak <code dawg.eu> 2013-03-13 21:58:50 PDT ---
(In reply to comment #11)
 Speaking of which I just noticed that `runAll` doesn't wrap
 `runModuleUnitTests()` in a try/catch. That seems like a bug to me.
 
It does https://github.com/D-Programming-Language/druntime/blob/master/src/rt/dmain2.d#L613. tryExec(&runAll); -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Mar 13 2013