www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Why is mangling different for separate compilation?

reply Atila Neves <atila.neves gmail.com> writes:
foo.d:
-----
void main() {
     import bar;
     foreach(ut; __traits(getUnitTests, bar)) ut();
}
-----

bar.d:
-----
unittest { assert(1 == 2); }
-----


# compile all at once
dmd -unittest foo.d bar.d # fine


# compile separately
dmd -c -unittest foo.d
dmd -c -unittest bar.d
dmd foo.o bar.o
foo.o:foo.d:function _Dmain: error: undefined reference to 
'_D3bar16__unittestL2_531FZv'
collect2: error: ld returned 1 exit status
--- errorlevel 1


objdump shows me that bar.o has a '_D3bar14__unittestL2_1FZv' 
symbol, which is nearly, but not quite, what foo.o is trying to 
call.

Huh?

Atila
Feb 26 2016
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2016-02-26 13:45, Atila Neves wrote:
 foo.d:
 -----
 void main() {
      import bar;
      foreach(ut; __traits(getUnitTests, bar)) ut();
 }
 -----

 bar.d:
 -----
 unittest { assert(1 == 2); }
 -----


 # compile all at once
 dmd -unittest foo.d bar.d # fine


 # compile separately
 dmd -c -unittest foo.d
 dmd -c -unittest bar.d
 dmd foo.o bar.o
 foo.o:foo.d:function _Dmain: error: undefined reference to
 '_D3bar16__unittestL2_531FZv'
 collect2: error: ld returned 1 exit status
 --- errorlevel 1


 objdump shows me that bar.o has a '_D3bar14__unittestL2_1FZv' symbol,
 which is nearly, but not quite, what foo.o is trying to call.
What is the mangling when it succeeds. -- /Jacob Carlborg
Feb 26 2016
parent Atila Neves <atila.neves gmail.com> writes:
On Friday, 26 February 2016 at 14:56:04 UTC, Jacob Carlborg wrote:
 On 2016-02-26 13:45, Atila Neves wrote:
 foo.d:
 -----
 void main() {
      import bar;
      foreach(ut; __traits(getUnitTests, bar)) ut();
 }
 -----

 bar.d:
 -----
 unittest { assert(1 == 2); }
 -----


 # compile all at once
 dmd -unittest foo.d bar.d # fine


 # compile separately
 dmd -c -unittest foo.d
 dmd -c -unittest bar.d
 dmd foo.o bar.o
 foo.o:foo.d:function _Dmain: error: undefined reference to
 '_D3bar16__unittestL2_531FZv'
 collect2: error: ld returned 1 exit status
 --- errorlevel 1


 objdump shows me that bar.o has a '_D3bar14__unittestL2_1FZv' 
 symbol,
 which is nearly, but not quite, what foo.o is trying to call.
What is the mangling when it succeeds.
_D3bar14__unittestL1_1FZv Atila
Feb 26 2016
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/26/2016 4:45 AM, Atila Neves wrote:
 '_D3bar16__unittestL2_531FZv'
 '_D3bar14__unittestL2_1FZv'
It uses a sequence number to generate different ids for the unit tests. In the former, it's the 531st unit test, the latter, the first.
Feb 26 2016
next sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Saturday, 27 February 2016 at 03:51:00 UTC, Walter Bright 
wrote:
 On 2/26/2016 4:45 AM, Atila Neves wrote:
 '_D3bar16__unittestL2_531FZv'
 '_D3bar14__unittestL2_1FZv'
It uses a sequence number to generate different ids for the unit tests. In the former, it's the 531st unit test, the latter, the first.
In both cases though, there's only one unit test. The only difference is how the files were compiled. What's the number after "bar" then? It's 16 in one case and 14 in the other. Atila
Feb 27 2016
next sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Saturday, 27 February 2016 at 09:11:02 UTC, Atila Neves wrote:
 On Saturday, 27 February 2016 at 03:51:00 UTC, Walter Bright 
 wrote:
 On 2/26/2016 4:45 AM, Atila Neves wrote:
 '_D3bar16__unittestL2_531FZv'
 '_D3bar14__unittestL2_1FZv'
It uses a sequence number to generate different ids for the unit tests. In the former, it's the 531st unit test, the latter, the first.
In both cases though, there's only one unit test. The only difference is how the files were compiled.
531 is the number across all modules, because they weren't separately compiled.
 What's the number after "bar" then? It's 16 in one case and 14 
 in the other.
That's the length of the next token: __unittestL2_531 is 16 characters long, while __unittestL2_1 is 14 characters long. The FZv characters at the end are usually some kind of type mangling, I believe.
Feb 27 2016
parent reply Atila Neves <atila.neves gmail.com> writes:
On Saturday, 27 February 2016 at 10:18:53 UTC, Joakim wrote:
 On Saturday, 27 February 2016 at 09:11:02 UTC, Atila Neves 
 wrote:
 On Saturday, 27 February 2016 at 03:51:00 UTC, Walter Bright 
 wrote:
 On 2/26/2016 4:45 AM, Atila Neves wrote:
 '_D3bar16__unittestL2_531FZv'
 '_D3bar14__unittestL2_1FZv'
It uses a sequence number to generate different ids for the unit tests. In the former, it's the 531st unit test, the latter, the first.
In both cases though, there's only one unit test. The only difference is how the files were compiled.
531 is the number across all modules, because they weren't separately compiled.
Which "all modules"? Phobos? druntime? The only `import` in that code is `import bar;`. Atila
Feb 27 2016
parent Joakim <dlang joakim.fea.st> writes:
On Saturday, 27 February 2016 at 10:56:08 UTC, Atila Neves wrote:
 On Saturday, 27 February 2016 at 10:18:53 UTC, Joakim wrote:
 On Saturday, 27 February 2016 at 09:11:02 UTC, Atila Neves 
 wrote:
 On Saturday, 27 February 2016 at 03:51:00 UTC, Walter Bright 
 wrote:
 On 2/26/2016 4:45 AM, Atila Neves wrote:
 '_D3bar16__unittestL2_531FZv'
 '_D3bar14__unittestL2_1FZv'
It uses a sequence number to generate different ids for the unit tests. In the former, it's the 531st unit test, the latter, the first.
In both cases though, there's only one unit test. The only difference is how the files were compiled.
531 is the number across all modules, because they weren't separately compiled.
Which "all modules"? Phobos? druntime? The only `import` in that code is `import bar;`.
Oh, I didn't read your previous posts carefully, I thought the 531 was from compiling everything together, but it looks it's from separate compilation. I guess it's generating a different number from somewhere else. What I know is that when I tried to separately compile druntime and phobos and use getUnitTests in the test runner, dmd would get confused because it would increment that number across all modules in the test runner (also the ordering of imported modules would matter), but the separately compiled modules unittests' would have much smaller numbers in their mangling. I'm guessing you're hitting some other version of the same problem with your example, though the underlying reason you're getting 531 might be different.
Feb 27 2016
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/27/2016 1:11 AM, Atila Neves wrote:
 On Saturday, 27 February 2016 at 03:51:00 UTC, Walter Bright wrote:
 On 2/26/2016 4:45 AM, Atila Neves wrote:
 '_D3bar16__unittestL2_531FZv'
 '_D3bar14__unittestL2_1FZv'
It uses a sequence number to generate different ids for the unit tests. In the former, it's the 531st unit test, the latter, the first.
In both cases though, there's only one unit test. The only difference is how the files were compiled.
It's a global number in the compiler.
 What's the number after "bar" then? It's 16 in one case and 14 in the other.
"31" adds 2 characters to the identifier
Feb 27 2016
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2016-02-27 04:51, Walter Bright wrote:

 It uses a sequence number to generate different ids for the unit tests.
 In the former, it's the 531st unit test, the latter, the first.
It sounds like the compiler is seeing different unit tests during the different compilation. But why is that the case. -- /Jacob Carlborg
Feb 28 2016
parent reply Chris Wright <dhasenan gmail.com> writes:
On Sun, 28 Feb 2016 20:23:59 +0100, Jacob Carlborg wrote:

 On 2016-02-27 04:51, Walter Bright wrote:
 
 It uses a sequence number to generate different ids for the unit tests.
 In the former, it's the 531st unit test, the latter, the first.
It sounds like the compiler is seeing different unit tests during the different compilation. But why is that the case.
Or like there's a global counter that is incremented for each unittest. A different order of traversal of unittests would result in a different mangled name for the same unittest in different runs.
Feb 28 2016
parent reply Jacob Carlborg <doob me.com> writes:
On 2016-02-28 22:03, Chris Wright wrote:

 Or like there's a global counter that is incremented for each unittest. A
 different order of traversal of unittests would result in a different
 mangled name for the same unittest in different runs.
The question is why it would be different. Here's a guess: * The compiler will perform semantic analysis of the code in the current module before processing any imports * It will also process all files on the command line before processing any imports * The compiler will process object.d before processing any other imports That would mean in the case when compiling foo.d during separate compilation it would process object.d before processing bar.d. That would mean the compiler is processing a lot unittest blocks through object.d and its imports before processing the unittest block in bar.d. -- /Jacob Carlborg
Feb 28 2016
parent reply Chris Wright <dhasenan gmail.com> writes:
On Mon, 29 Feb 2016 08:22:45 +0100, Jacob Carlborg wrote:

 On 2016-02-28 22:03, Chris Wright wrote:
 
 Or like there's a global counter that is incremented for each unittest.
 A different order of traversal of unittests would result in a different
 mangled name for the same unittest in different runs.
The question is why it would be different. Here's a guess: * The compiler will perform semantic analysis of the code in the current module before processing any imports
Simpler than that. Module foo.a imports std.stdio; module foo.b imports foo.a and std.path. When compiling separately, you're not necessarily changing the ordering property of the modules you run semantic on, but you are changing the set of modules.
Feb 29 2016
parent Jacob Carlborg <doob me.com> writes:
On 2016-02-29 17:27, Chris Wright wrote:

 Simpler than that. Module foo.a imports std.stdio; module foo.b imports
 foo.a and std.path. When compiling separately, you're not necessarily
 changing the ordering property of the modules you run semantic on, but
 you are changing the set of modules.
I'm looking at Atila's original example. There's only one import, "bar". -- /Jacob Carlborg
Feb 29 2016
prev sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Friday, 26 February 2016 at 12:45:35 UTC, Atila Neves wrote:
 foo.d:
 -----
 void main() {
     import bar;
     foreach(ut; __traits(getUnitTests, bar)) ut();
 }
 -----

 bar.d:
 -----
 unittest { assert(1 == 2); }
 -----


 # compile all at once
 dmd -unittest foo.d bar.d # fine


 # compile separately
 dmd -c -unittest foo.d
 dmd -c -unittest bar.d
 dmd foo.o bar.o
 foo.o:foo.d:function _Dmain: error: undefined reference to 
 '_D3bar16__unittestL2_531FZv'
 collect2: error: ld returned 1 exit status
 --- errorlevel 1


 objdump shows me that bar.o has a '_D3bar14__unittestL2_1FZv' 
 symbol, which is nearly, but not quite, what foo.o is trying to 
 call.

 Huh?
As Walter notes, it's because the mangling changes based on whether you separately compile the unit tests. It's why you needed to generate the main D source file for unit-threaded too, becuase __traits(getUnitTests) gets confused by this same issue if the modules containing tests are separately compiled: https://github.com/atilaneves/unit-threaded/blob/master/source/unit_threaded/runtime.d#L175
Feb 26 2016
parent reply Atila Neves <atila.neves gmail.com> writes:
On Saturday, 27 February 2016 at 06:34:19 UTC, Joakim wrote:
 On Friday, 26 February 2016 at 12:45:35 UTC, Atila Neves wrote:
 foo.d:
 -----
 void main() {
     import bar;
     foreach(ut; __traits(getUnitTests, bar)) ut();
 }
 -----

 bar.d:
 -----
 unittest { assert(1 == 2); }
 -----


 # compile all at once
 dmd -unittest foo.d bar.d # fine


 # compile separately
 dmd -c -unittest foo.d
 dmd -c -unittest bar.d
 dmd foo.o bar.o
 foo.o:foo.d:function _Dmain: error: undefined reference to 
 '_D3bar16__unittestL2_531FZv'
 collect2: error: ld returned 1 exit status
 --- errorlevel 1


 objdump shows me that bar.o has a '_D3bar14__unittestL2_1FZv' 
 symbol, which is nearly, but not quite, what foo.o is trying 
 to call.

 Huh?
As Walter notes, it's because the mangling changes based on whether you separately compile the unit tests. It's why you needed to generate the main D source file for unit-threaded too, becuase __traits(getUnitTests) gets confused by this same issue if the modules containing tests are separately compiled: https://github.com/atilaneves/unit-threaded/blob/master/source/unit_threaded/runtime.d#L175
Well, guess how I found out about this issue? ;) It turns out all my unit test files were always in the same package due to laziness, and as soon as I tried compiling separately, things broke. Sigh. I've had similar problems in the past with template mixins. It seems D's compile-time features don't mix with any kind of separate compilation, which is a shame. Atila
Feb 27 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/27/2016 1:12 AM, Atila Neves wrote:
 I've had similar problems in the past with template mixins. It seems D's
 compile-time features don't mix with any kind of separate compilation, which is
 a shame.
Any ideas on how unit tests should be named?
Feb 27 2016
parent reply Joakim <dlang joakim.fea.st> writes:
On Saturday, 27 February 2016 at 11:27:39 UTC, Walter Bright 
wrote:
 On 2/27/2016 1:12 AM, Atila Neves wrote:
 I've had similar problems in the past with template mixins. It 
 seems D's
 compile-time features don't mix with any kind of separate 
 compilation, which is
 a shame.
Any ideas on how unit tests should be named?
Why has the additional count been added? You're already using the line number to differentiate unit test blocks. For unit test blocks that are all on one line? ;)
Feb 27 2016
parent reply Atila Neves <atila.neves gmail.com> writes:
On Saturday, 27 February 2016 at 11:31:53 UTC, Joakim wrote:
 On Saturday, 27 February 2016 at 11:27:39 UTC, Walter Bright 
 wrote:
 On 2/27/2016 1:12 AM, Atila Neves wrote:
 I've had similar problems in the past with template mixins. 
 It seems D's
 compile-time features don't mix with any kind of separate 
 compilation, which is
 a shame.
Any ideas on how unit tests should be named?
Why has the additional count been added? You're already using the line number to differentiate unit test blocks. For unit test blocks that are all on one line? ;)
I guess that makes sense. And it'd link! Atila
Feb 28 2016
parent John Colvin <john.loughran.colvin gmail.com> writes:
On Sunday, 28 February 2016 at 12:59:53 UTC, Atila Neves wrote:
 On Saturday, 27 February 2016 at 11:31:53 UTC, Joakim wrote:
 On Saturday, 27 February 2016 at 11:27:39 UTC, Walter Bright 
 wrote:
 On 2/27/2016 1:12 AM, Atila Neves wrote:
 I've had similar problems in the past with template mixins. 
 It seems D's
 compile-time features don't mix with any kind of separate 
 compilation, which is
 a shame.
Any ideas on how unit tests should be named?
Why has the additional count been added? You're already using the line number to differentiate unit test blocks. For unit test blocks that are all on one line? ;)
I guess that makes sense. And it'd link! Atila
You could always add an additional number to uniquely identify them if there are multiple unittests on one line. It would seem weird to have a special case in the grammar for unittests.
Feb 28 2016