www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Unittesting in static libraries

reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
I'd like to turn some attention to unittests, which don't seem to work
with static libraries. Consider:

.\main.d:
import mylib.test;
void main()
{
    auto x = foo();
}

.\mylib\test.d
module mylib.test;
int foo() { return 1; }
unittest { assert(0); }

$ dmd -unittest main.d mylib\test.d && main.exe
core.exception.AssertError mylib.test(5): unittest failure

So far so good. It even works if I don't reference foo() from within
main(), even though the "unittest.d" file in Phobos has this comment
about it:
"// Bring in unit test for module by referencing function in it"

I think that comment might be outdated.

But now try it with a static library:

$ cd mylib
$ dmd -unittest -lib test.d
$ cd..
$ dmd -unittest main.d mylib\test.lib && main.exe

The unittests from the library don't run.

There's a bug about this in bugzilla somewhere (I just can't seem to
find it now), but it doesn't seem to be getting much attention. Since
unittests are first-class citizens in D, I really hope we can get this
working soon. Cheers.
Sep 16 2011
next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 16 Sep 2011 14:18:30 -0400, Andrej Mitrovic  
<andrej.mitrovich gmail.com> wrote:


 I'd like to turn some attention to unittests, which don't seem to work
 with static libraries.



[snip]


 There's a bug about this in bugzilla somewhere (I just can't seem to
 find it now), but it doesn't seem to be getting much attention. Since
 unittests are first-class citizens in D, I really hope we can get this
 working soon. Cheers.



http://d.puremagic.com/issues/show_bug.cgi?id=4669

-Steve
Sep 16 2011
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 9/16/11, Steven Schveighoffer <schveiguy yahoo.com> wrote:

 http://d.puremagic.com/issues/show_bug.cgi?id=4669



Thanks. I even commented on that one yesterday but couldn't find it
today because I search for "unittest" instead of "unit test", heh!

Anyway, my build command becomes significantly more complicated if I
want to unittest libraries. I essentially have to copy the build
command for the library, and then mix that in with the build command
for the app itself. Pain in the arse! :]
Sep 16 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 16 Sep 2011 14:39:01 -0400, Andrej Mitrovic  
<andrej.mitrovich gmail.com> wrote:


 On 9/16/11, Steven Schveighoffer <schveiguy yahoo.com> wrote:

 http://d.puremagic.com/issues/show_bug.cgi?id=4669



 Thanks. I even commented on that one yesterday but couldn't find it
 today because I search for "unittest" instead of "unit test", heh!

 Anyway, my build command becomes significantly more complicated if I
 want to unittest libraries. I essentially have to copy the build
 command for the library, and then mix that in with the build command
 for the app itself. Pain in the arse! :]



I agree, it needs to be fixed.  I think the reason it hasn't gotten much  
attention is because most people using a library do not want to unit test  
the library, and the library authors typically do not use the -lib switch  
when unit testing the library.

I know for dcollections, I have a separate command line for unit testing  
which doesn't use -lib.

-Steve
Sep 16 2011
prev sibling next sibling parent reply =?UTF-8?B?IkrDqXLDtG1lIE0uIEJlcmdlciI=?= <jeberger free.fr> writes:
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Andrej Mitrovic wrote:

 I'd like to turn some attention to unittests, which don't seem to work
 with static libraries. Consider:
=20
 .\main.d:
 import mylib.test;
 void main()
 {
     auto x =3D foo();
 }
=20
 .\mylib\test.d
 module mylib.test;
 int foo() { return 1; }
 unittest { assert(0); }
=20
 $ dmd -unittest main.d mylib\test.d && main.exe
 core.exception.AssertError mylib.test(5): unittest failure
=20
 So far so good. It even works if I don't reference foo() from within
 main(), even though the "unittest.d" file in Phobos has this comment
 about it:
 "// Bring in unit test for module by referencing function in it"
=20
 I think that comment might be outdated.
=20
 But now try it with a static library:
=20
 $ cd mylib
 $ dmd -unittest -lib test.d
 $ cd..
 $ dmd -unittest main.d mylib\test.lib && main.exe
=20
 The unittests from the library don't run.
=20
 There's a bug about this in bugzilla somewhere (I just can't seem to
 find it now), but it doesn't seem to be getting much attention. Since
 unittests are first-class citizens in D, I really hope we can get this
 working soon. Cheers.



	I think that it is not a bug, it is a feature (sort of). Could you
look at the assembly generated for your main.d file? My guess is
that it does not reference foo at all, either because the call to
foo was inlined or because it was discarded (since you do not use x
after initializing it).

	The reason it works with the first form is that all object files
that are specifically put on the command line are included in the
executable when linking even if they are not used.

	The reason it does not work with the library is that library
objects are only included if they are referenced (to save executable
size).

	Of course, that is an issue with unit tests. Not sure how to fix it
though, since it is as much a linker issue as a compiler issue,
unless there is a way to trick the linker into thinking that any
module with unittest is referenced from any other module importing
it (when compiling with -unittest) even when there is no other
reference (in which case, it will stop being necessary to call a
function from the tested module to include the unit tests).

		Jerome
--=20
mailto:jeberger free.fr
http://jeberger.free.fr
Jabber: jeberger jabber.fr
Sep 16 2011
parent =?UTF-8?B?IkrDqXLDtG1lIE0uIEJlcmdlciI=?= <jeberger free.fr> writes:
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Steven Schveighoffer wrote:

 On Fri, 16 Sep 2011 15:14:50 -0400, J=C3=A9r=C3=B4me M. Berger <jeberge=




 wrote:
=20

 Andrej Mitrovic wrote:

 I'd like to turn some attention to unittests, which don't seem to wor=










 with static libraries. Consider:

 .\main.d:
 import mylib.test;
 void main()
 {
     auto x =3D foo();
 }

 .\mylib\test.d
 module mylib.test;
 int foo() { return 1; }
 unittest { assert(0); }

 $ dmd -unittest main.d mylib\test.d && main.exe
 core.exception.AssertError mylib.test(5): unittest failure










     I think that it is not a bug, it is a feature (sort of). Could you=







 look at the assembly generated for your main.d file? My guess is
 that it does not reference foo at all, either because the call to
 foo was inlined or because it was discarded (since you do not use x
 after initializing it).

     The reason it works with the first form is that all object files
 that are specifically put on the command line are included in the
 executable when linking even if they are not used.

     The reason it does not work with the library is that library
 objects are only included if they are referenced (to save executable
 size).



 That isn't true, the library is not passed to the linker as a library,
 it's passed as an archive of object files.
=20
 Forgive my ignorance of OPTLINK syntax, I'll make my point with linux
 linker:
=20
 dmd main.d mylib/libtest.a -> compile in all object files from libtest.=




 dmd main.d -L-Lmylib -L-ltest -> only compile in parts of libtest.a tha=




 are referenced
=20



v2.21.1, but AFAIR it never was true). Both forms only link in the
parts of libtest.a that are referenced.


 In spite of all this, such inlining or optimizations are only made if
 you use -O or -inline.  And I think just importing the module includes =




=20



using x (for example printing it) to see what happens?

		Jerome
--=20
mailto:jeberger free.fr
http://jeberger.free.fr
Jabber: jeberger jabber.fr
Sep 17 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 16 Sep 2011 15:14:50 -0400, J=C3=A9r=C3=B4me M. Berger <jeberger=
 free.fr>  =

wrote:


 Andrej Mitrovic wrote:

 I'd like to turn some attention to unittests, which don't seem to wor=







 with static libraries. Consider:

 .\main.d:
 import mylib.test;
 void main()
 {
     auto x =3D foo();
 }

 .\mylib\test.d
 module mylib.test;
 int foo() { return 1; }
 unittest { assert(0); }

 $ dmd -unittest main.d mylib\test.d && main.exe
 core.exception.AssertError mylib.test(5): unittest failure

 So far so good. It even works if I don't reference foo() from within
 main(), even though the "unittest.d" file in Phobos has this comment
 about it:
 "// Bring in unit test for module by referencing function in it"

 I think that comment might be outdated.

 But now try it with a static library:

 $ cd mylib
 $ dmd -unittest -lib test.d
 $ cd..
 $ dmd -unittest main.d mylib\test.lib && main.exe

 The unittests from the library don't run.

 There's a bug about this in bugzilla somewhere (I just can't seem to
 find it now), but it doesn't seem to be getting much attention. Since=







 unittests are first-class citizens in D, I really hope we can get thi=







 working soon. Cheers.



 	I think that it is not a bug, it is a feature (sort of). Could you
 look at the assembly generated for your main.d file? My guess is
 that it does not reference foo at all, either because the call to
 foo was inlined or because it was discarded (since you do not use x
 after initializing it).

 	The reason it works with the first form is that all object files
 that are specifically put on the command line are included in the
 executable when linking even if they are not used.

 	The reason it does not work with the library is that library
 objects are only included if they are referenced (to save executable
 size).



That isn't true, the library is not passed to the linker as a library,  =

it's passed as an archive of object files.

Forgive my ignorance of OPTLINK syntax, I'll make my point with linux  =

linker:

dmd main.d mylib/libtest.a -> compile in all object files from libtest.a=

dmd main.d -L-Lmylib -L-ltest -> only compile in parts of libtest.a that=
  =

are referenced

In spite of all this, such inlining or optimizations are only made if yo=
u  =

use -O or -inline.  And I think just importing the module includes it.

-Steve
Sep 16 2011
prev sibling next sibling parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 9/16/2011 11:18 AM, Andrej Mitrovic wrote:

 I'd like to turn some attention to unittests, which don't seem to work
 with static libraries. Consider:

 ..\main.d:
 import mylib.test;
 void main()
 {
      auto x = foo();
 }

 ..\mylib\test.d
 module mylib.test;
 int foo() { return 1; }
 unittest { assert(0); }

 $ dmd -unittest main.d mylib\test.d&&  main.exe
 core.exception.AssertError mylib.test(5): unittest failure

 So far so good. It even works if I don't reference foo() from within
 main(), even though the "unittest.d" file in Phobos has this comment
 about it:
 "// Bring in unit test for module by referencing function in it"

 I think that comment might be outdated.

 But now try it with a static library:

 $ cd mylib
 $ dmd -unittest -lib test.d
 $ cd..
 $ dmd -unittest main.d mylib\test.lib&&  main.exe

 The unittests from the library don't run.

 There's a bug about this in bugzilla somewhere (I just can't seem to
 find it now), but it doesn't seem to be getting much attention. Since
 unittests are first-class citizens in D, I really hope we can get this
 working soon. Cheers.




I think the main issue here is that a module that is compiled to a 
library, is split into a lot of small "object files" (one for each 
function or global variable) before being combined to the library. This 
allows the linker to just take the actually referenced parts and leave 
out anything that is never called.

The unit tests are only referenced from the module info, so it might 
work if you have a reference to it in your main executable.
Another workaround would be to build the library in two steps, compiling 
to normal object files first, then binding these to a library (shameless 
ad: "separate compile and link" in Visual D):

dmd -unittest -c -od. test1.d test2.d
dmd -lib -oftest.lib test1.obj test1.obj

so it avoids breaking up the modules. the -od. is needed to not just 
build a single object file.
Sep 16 2011
next sibling parent Rainer Schuetze <r.sagitario gmx.de> writes:
On 9/16/2011 2:18 PM, Andrej Mitrovic wrote:

 On 9/17/11, Rainer Schuetze<r.sagitario gmx.de>  wrote:

 I think the main issue here is that a module that is compiled to a
 library, is split into a lot of small "object files" (one for each
 function or global variable) before being combined to the library. This
 allows the linker to just take the actually referenced parts and leave
 out anything that is never called.



 Well maybe DMD could tell Mr. Optlink to not do that if I pass
 -unittest and -lib. :)


 The unit tests are only referenced from the module info, so it might
 work if you have a reference to it in your main executable.



 How do I reference this module info?



You can do that by using its mangled name without the preceding 
underscore, but extern(C) linkage. Unfortunately, I just noticed itdoes 
not work after all, because every unit test gets its own module info 
with an unpredictable name (contrary to what's descrined in the ABI 
http://d-programming-language.org/abi.html).


 Another workaround would be to build the library in two steps, compiling
 to normal object files first, then binding these to a library (shameless
 ad: "separate compile and link" in Visual D):

 dmd -unittest -c -od. test1.d test2.d
 dmd -lib -oftest.lib test1.obj test1.obj



 That works as long as in main I reference a function from the library
 (I guess that's where that comment in unittest.d came from, and the
 complicated makefile). I'm so-so for this solution.



There is no point in creating a library if you want to link in 
everything anyway. You can just build the library module files into a 
single object file and add that to your executables' command line.
Sep 17 2011
prev sibling parent Rainer Schuetze <r.sagitario gmx.de> writes:
On 9/19/2011 3:55 AM, Steven Schveighoffer wrote:

 On Sat, 17 Sep 2011 01:34:35 -0400, Rainer Schuetze <r.sagitario gmx.de>
 wrote:


 I think the main issue here is that a module that is compiled to a
 library, is split into a lot of small "object files" (one for each
 function or global variable) before being combined to the library.
 This allows the linker to just take the actually referenced parts and
 leave out anything that is never called.

 The unit tests are only referenced from the module info, so it might
 work if you have a reference to it in your main executable.
 Another workaround would be to build the library in two steps,
 compiling to normal object files first, then binding these to a
 library (shameless ad: "separate compile and link" in Visual D):

 dmd -unittest -c -od. test1.d test2.d
 dmd -lib -oftest.lib test1.obj test1.obj

 so it avoids breaking up the modules. the -od. is needed to not just
 build a single object file.



 The output from these lines should be identical to:

 dmd -unittest -oftest.lib -lib test1.d test2.d

 If it's not, that's a bug.



It's different, and it is meant as a feature. As described above, a 
module is split into a number of object files, reducing dependencies 
between the modules in the library.

I'm not sure it is a good feature, though, because it has some side 
effects: the static constructors/destructors in a module are magically 
tied together, but the unittests are not. Also, the debug info is 
missing for classes/structs if the init member is never referenced (see 
http://d.puremagic.com/issues/show_bug.cgi?id=4014 ). Last time I tried, 
disabling the splitting caused other troubles as described in the bug 
report.
Sep 19 2011
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 9/17/11, Rainer Schuetze <r.sagitario gmx.de> wrote:

 I think the main issue here is that a module that is compiled to a
 library, is split into a lot of small "object files" (one for each
 function or global variable) before being combined to the library. This
 allows the linker to just take the actually referenced parts and leave
 out anything that is never called.



Well maybe DMD could tell Mr. Optlink to not do that if I pass
-unittest and -lib. :)


 The unit tests are only referenced from the module info, so it might
 work if you have a reference to it in your main executable.



How do I reference this module info?


 Another workaround would be to build the library in two steps, compiling
 to normal object files first, then binding these to a library (shameless
 ad: "separate compile and link" in Visual D):

 dmd -unittest -c -od. test1.d test2.d
 dmd -lib -oftest.lib test1.obj test1.obj



That works as long as in main I reference a function from the library
(I guess that's where that comment in unittest.d came from, and the
complicated makefile). I'm so-so for this solution.
Sep 16 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Sat, 17 Sep 2011 11:11:35 -0400, J=C3=A9r=C3=B4me M. Berger <jeberger=
 free.fr>  =

wrote:


 Steven Schveighoffer wrote:

 On Fri, 16 Sep 2011 15:14:50 -0400, J=C3=A9r=C3=B4me M. Berger <jeber=







 wrote:


 Andrej Mitrovic wrote:

 I'd like to turn some attention to unittests, which don't seem to w=













 with static libraries. Consider:

 .\main.d:
 import mylib.test;
 void main()
 {
     auto x =3D foo();
 }

 .\mylib\test.d
 module mylib.test;
 int foo() { return 1; }
 unittest { assert(0); }

 $ dmd -unittest main.d mylib\test.d && main.exe
 core.exception.AssertError mylib.test(5): unittest failure










     I think that it is not a bug, it is a feature (sort of). Could y=










 look at the assembly generated for your main.d file? My guess is
 that it does not reference foo at all, either because the call to
 foo was inlined or because it was discarded (since you do not use x
 after initializing it).

     The reason it works with the first form is that all object files=










 that are specifically put on the command line are included in the
 executable when linking even if they are not used.

     The reason it does not work with the library is that library
 objects are only included if they are referenced (to save executable=










 size).



 That isn't true, the library is not passed to the linker as a library=







 it's passed as an archive of object files.

 Forgive my ignorance of OPTLINK syntax, I'll make my point with linux=







 linker:

 dmd main.d mylib/libtest.a -> compile in all object files from libtes=







 dmd main.d -L-Lmylib -L-ltest -> only compile in parts of libtest.a t=







 are referenced



 v2.21.1, but AFAIR it never was true). Both forms only link in the
 parts of libtest.a that are referenced.



I could have sworn that passing an archive was like passing all the .o  =

files.


 In spite of all this, such inlining or optimizations are only made if=







 you use -O or -inline.  And I think just importing the module include=








 it.



 using x (for example printing it) to see what happens?



I know inlining does not happen unless -inline is passed.  No need to  =

check the output.

An easier check is to compile main.d without passing mylib/test.  If it =
 =

doesn't need anything from the object file, it will link, right?

-Steve
Sep 19 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Sat, 17 Sep 2011 01:34:35 -0400, Rainer Schuetze <r.sagitario gmx.de>  
wrote:


 I think the main issue here is that a module that is compiled to a  
 library, is split into a lot of small "object files" (one for each  
 function or global variable) before being combined to the library. This  
 allows the linker to just take the actually referenced parts and leave  
 out anything that is never called.

 The unit tests are only referenced from the module info, so it might  
 work if you have a reference to it in your main executable.
 Another workaround would be to build the library in two steps, compiling  
 to normal object files first, then binding these to a library (shameless  
 ad: "separate compile and link" in Visual D):

 dmd -unittest -c -od. test1.d test2.d
 dmd -lib -oftest.lib test1.obj test1.obj

 so it avoids breaking up the modules. the -od. is needed to not just  
 build a single object file.



The output from these lines should be identical to:

dmd -unittest -oftest.lib -lib test1.d test2.d

If it's not, that's a bug.

-Steve
Sep 19 2011
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 19 Sep 2011 22:40:06 -0400, Rainer Schuetze <r.sagitario gmx.de>  
wrote:


 On 9/19/2011 3:55 AM, Steven Schveighoffer wrote:

 On Sat, 17 Sep 2011 01:34:35 -0400, Rainer Schuetze <r.sagitario gmx.de>
 wrote:


 I think the main issue here is that a module that is compiled to a
 library, is split into a lot of small "object files" (one for each
 function or global variable) before being combined to the library.
 This allows the linker to just take the actually referenced parts and
 leave out anything that is never called.

 The unit tests are only referenced from the module info, so it might
 work if you have a reference to it in your main executable.
 Another workaround would be to build the library in two steps,
 compiling to normal object files first, then binding these to a
 library (shameless ad: "separate compile and link" in Visual D):

 dmd -unittest -c -od. test1.d test2.d
 dmd -lib -oftest.lib test1.obj test1.obj

 so it avoids breaking up the modules. the -od. is needed to not just
 build a single object file.



 The output from these lines should be identical to:

 dmd -unittest -oftest.lib -lib test1.d test2.d

 If it's not, that's a bug.



 It's different, and it is meant as a feature. As described above, a  
 module is split into a number of object files, reducing dependencies  
 between the modules in the library.

 I'm not sure it is a good feature, though, because it has some side  
 effects: the static constructors/destructors in a module are magically  
 tied together, but the unittests are not. Also, the debug info is  
 missing for classes/structs if the init member is never referenced (see  
 http://d.puremagic.com/issues/show_bug.cgi?id=4014 ). Last time I tried,  
 disabling the splitting caused other troubles as described in the bug  
 report.



Oh yes, I actually remember looking at the archive generated by -lib for  
another reason.  Adding a unit test caused a larger library to be output  
even when -unittest was not passed!  See here:  
http://d.puremagic.com/issues/show_bug.cgi?id=5560

So I can see why this happens now, but regardless of the cause, if there  
is no way to access unit tests compiled into the library, that is *not* a  
feature, it's a bug.

-Steve
Sep 19 2011