• Mark Isaacson (27/27) May 30 2014 I'm having fun running some unittests. I set up a simple homemade
• Jonathan M Davis via Digitalmars-d-learn (19/46) Jun 01 2014 On Fri, 30 May 2014 20:13:19 +0000

"Mark Isaacson" <turck11 hotmail.com> writes:

I'm having fun running some unittests. I set up a simple homemade
mock of std.net.curl's functions that essentially just consists
of a global queue that I can add strings to and get back in a
predictable order when calling std.net.curl.get/post/etc.

I use this mock in a couple of different modules for unit
testing, namely the module I have the mock in and one other one.
When I run my unit tests, it seems to enqueue all of the
responses from both of my unit tests (from different modules)
before finishing those tests and removing things from the global
queue. This is problematic in that I cannot anticipate the state
of the global queue for my tests. Does this sound more like a bug
or a "feature". My understanding is that, at least for now, tests
are not run concurrently on the latest official dmd release;
since my queue is not qualified with shared, things should be
thread local anyway.

TLDR:
Executation of tests A and B is as follows:
A pushes to global queue
B pushes to global queue
B pops on global queue -- program crashes

Expected order:
A pushes to global queue
A pops from global queue
B pushes to global queue
B pops from global queue

Or switch the order in which A and B execute, doesn't really
matter.

Jonathan M Davis via Digitalmars-d-learn writes:

On Fri, 30 May 2014 20:13:19 +0000
Mark Isaacson via Digitalmars-d-learn
<digitalmars-d-learn puremagic.com> wrote:

 I'm having fun running some unittests. I set up a simple homemade
 mock of std.net.curl's functions that essentially just consists
 of a global queue that I can add strings to and get back in a
 predictable order when calling std.net.curl.get/post/etc.

 I use this mock in a couple of different modules for unit
 testing, namely the module I have the mock in and one other one.
 When I run my unit tests, it seems to enqueue all of the
 responses from both of my unit tests (from different modules)
 before finishing those tests and removing things from the global
 queue. This is problematic in that I cannot anticipate the state
 of the global queue for my tests. Does this sound more like a bug
 or a "feature". My understanding is that, at least for now, tests
 are not run concurrently on the latest official dmd release;
 since my queue is not qualified with shared, things should be
 thread local anyway.

 TLDR:
 Executation of tests A and B is as follows:
 A pushes to global queue
 B pushes to global queue
 B pops on global queue -- program crashes

 Expected order:
 A pushes to global queue
 A pops from global queue
 B pushes to global queue
 B pops from global queue

 Or switch the order in which A and B execute, doesn't really
 matter.

Well, the behavior you're seeing would make sense if you're using static
destructors for the popping part, since they aren't going to be run until the
program is shut down, but if you're using the unittest blocks to do the
popping, that's definitely a bit odd. I would have expected each module's unit
tests to be run sequentially. Certainly, within a module, the tests are
currently run sequentially. However, there has been recent discussion of
changing it so that unittest blocks will be run in parallel (at lest by
default), so in the future, they may very well run in parallel (probably
requiring an attribute of some kind to make them run sequentially).

It's generally considered good practice to make it so that your unittest
blocks don't rely on each other and that they don't change the global state,
in which case, it doesn't matter what order they're in (though that still
allows for setting stuff up in static constructors and shutting it down in
static destructors so long as that state doesn't change after a unittest block
is run).

- Jonathan M Davis