www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DM linker vs GCC linker?

reply Andrea Fontana <advmail katamail.com> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

I'm working on a complex project for my company.
It's a text-parser that handles complex/ambiguos word-based grammars. It
reads a set of xml files that define rules, build some structures
(trees, dictionaries, etc..) and then parses your phrases.  It works
fine.

I compile sources in this (simple) way:
dmd -c *.d   (three source files)

If i link objects with dmd execution of unit tests takes:

Parsing: 722 ms, 419 =CE=BCs, and 3 hnsecs

real	5m10.199s
user	5m9.771s
sys	0m0.236s


using gcc as linker:

Parsing: 425 ms, 677 =CE=BCs, and 8 hnsecs

real	0m13.919s
user	0m13.845s
sys	0m0.088s


unittest outputs are identical and execution time of test phrases has
similar speed but dmd version takes a lot of time parsing xml and
building trees and other structures.

I can't post company code, but i wonder if this is a known issue... If
not i'll try to write some code to reproduce this problem...

Configuration:
Ubuntu 11.10 64bit
DMD64 D Compiler v2.056
gcc version 4.6.1 (Ubuntu/Linaro 4.6.1-9ubuntu3)=20
Jan 18 2012
parent reply Trass3r <un known.com> writes:
 Configuration:
 Ubuntu 11.10 64bit
 DMD64 D Compiler v2.056
 gcc version 4.6.1 (Ubuntu/Linaro 4.6.1-9ubuntu3)

? dmd also uses ld to link.
Jan 18 2012
next sibling parent reply Andrea Fontana <advmail katamail.com> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

I can't understand why it's 22 times slower...

Il giorno mer, 18/01/2012 alle 14.54 +0100, Trass3r ha scritto:

 Configuration:
 Ubuntu 11.10 64bit
 DMD64 D Compiler v2.056
 gcc version 4.6.1 (Ubuntu/Linaro 4.6.1-9ubuntu3)

? dmd also uses ld to link.

Jan 18 2012
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-01-18 15:37, Andrea Fontana wrote:
 Ok, here a test:

 test.d :

 import std.xml;
 import std.stdio;
 import std.conv;

 int main(string[] args)
 {
 if (args.length != 2)
 {
 writeln("Usage: ", args[0], " file.xml");
 return 0;
 }

 string content = to!string(std.file.read(args[1]));

 try{
 check(content);
 } catch (CheckException ex) { writeln("Exception: ", ex); }
 return 0;
 }


 dmd -c test.d && gcc test.o -lphobos2 -lrt -lpthread -o testgcc
 vs
 dmd -c test.d && dmd test.o -oftestd

 try this code using a long xml file (tested here with a 2.0 mb list of
 italian cities) ...

Have you tried compiling and linking in one step with dmd? Just like this: dmd test.d BTW, you can pass "-v2 to dmd to see which flags it uses when invoking gcc. -- /Jacob Carlborg
Jan 18 2012
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-01-18 16:00, Andrea Fontana wrote:
 This param:
 --Xlinker --export-dynamic

 slows down the entire app (check() std.xml function) by 29x on test.

 Is it useful to add it on default config? Probably if you use external
 dl, you should add it...

 andrea ububox <mailto:andrea ububox>:~/src/xml-test$ dmd -c test.d &&
 gcc test.o -o testgcc2 -m64 -Xlinker --export-dynamic -Xlinker
 --no-warn-search-mismatch -lphobos2 -lpthread -lm -lrt
 andrea ububox <mailto:andrea ububox>:~/src/xml-test$ time ./testgcc2
 cities.xml

 real 3m23.426s
 user 3m22.901s
 sys 0m0.276s

 andrea ububox:~/src/xml-test$ dmd -c test.d && gcc test.o -o testgcc2
 -m64 -Xlinker --no-warn-search-mismatch -lphobos2 -lpthread -lm -lrt
 andrea ububox <mailto:andrea ububox>:~/src/xml-test$ time ./testgcc2
 cities.xml

 real 0m7.433s
 user 0m7.316s
 sys 0m0.112s

 Il giorno mer, 18/01/2012 alle 15.43 +0100, Trass3r ha scritto:
  dmd -c test.d&&  gcc test.o -lphobos2 -lrt -lpthread -o testgcc
  vs
  dmd -c test.d&&  dmd test.o -oftestd

Well dmd test.d calls gcc test.o -o test -m64 -Xlinker -L/dmd/linux/lib64 -Xlinker -L/dmd/linux/lib32 -Xlinker --no-warn-search-mismatch -Xlinker --export-dynamic -lphobos2 -lpthread -lm -lrt


How about DMD passing --export-dynamic to GCC if -shared is passed to DMD? -- /Jacob Carlborg
Jan 18 2012
prev sibling next sibling parent Andrea Fontana <advmail katamail.com> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable


Ok, here a test:

test.d :

import std.xml;
import std.stdio;
import std.conv;

int main(string[] args)
{
        if (args.length !=3D 2)
        {
                writeln("Usage: ", args[0], " file.xml");
                return 0;
        }

        string content =3D to!string(std.file.read(args[1]));
       =20
        try{
                check(content);
        } catch (CheckException ex) { writeln("Exception: ", ex); }
        return 0;
}


dmd -c test.d && gcc test.o -lphobos2 -lrt -lpthread -o testgcc
vs
dmd -c test.d && dmd test.o -oftestd

try this code using a long xml file (tested here with a 2.0 mb list of
italian cities) ...

Il giorno mer, 18/01/2012 alle 15.27 +0100, Andrea Fontana ha scritto:

 I can't understand why it's 22 times slower...
=20
 Il giorno mer, 18/01/2012 alle 14.54 +0100, Trass3r ha scritto:=20
=20
 Configuration:
 Ubuntu 11.10 64bit
 DMD64 D Compiler v2.056
 gcc version 4.6.1 (Ubuntu/Linaro 4.6.1-9ubuntu3)

? dmd also uses ld to link.

=20

Jan 18 2012
prev sibling next sibling parent Trass3r <un known.com> writes:
 dmd -c test.d && gcc test.o -lphobos2 -lrt -lpthread -o testgcc
 vs
 dmd -c test.d && dmd test.o -oftestd

Well dmd test.d calls gcc test.o -o test -m64 -Xlinker -L/dmd/linux/lib64 -Xlinker -L/dmd/linux/lib32 -Xlinker --no-warn-search-mismatch -Xlinker --export-dynamic -lphobos2 -lpthread -lm -lrt
Jan 18 2012
prev sibling next sibling parent Andrea Fontana <advmail katamail.com> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

This param:
--Xlinker --export-dynamic =20

slows down the entire app (check() std.xml function) by 29x on test.=20

Is it useful to add it on default config? Probably if you use external
dl, you should add it...

andrea ububox:~/src/xml-test$ dmd -c test.d && gcc test.o -o testgcc2
-m64 -Xlinker --export-dynamic -Xlinker --no-warn-search-mismatch
-lphobos2 -lpthread -lm -lrt
andrea ububox:~/src/xml-test$ time ./testgcc2 cities.xml=20

real	3m23.426s
user	3m22.901s
sys	0m0.276s

andrea ububox:~/src/xml-test$ dmd -c test.d && gcc test.o -o testgcc2
-m64 -Xlinker --no-warn-search-mismatch -lphobos2 -lpthread -lm -lrt
andrea ububox:~/src/xml-test$ time ./testgcc2 cities.xml=20

real	0m7.433s
user	0m7.316s
sys	0m0.112s

Il giorno mer, 18/01/2012 alle 15.43 +0100, Trass3r ha scritto:

 dmd -c test.d && gcc test.o -lphobos2 -lrt -lpthread -o testgcc
 vs
 dmd -c test.d && dmd test.o -oftestd

Well dmd test.d calls gcc test.o -o test -m64 -Xlinker -L/dmd/linux/lib64 -Xlinker =20 -L/dmd/linux/lib32 -Xlinker --no-warn-search-mismatch -Xlinker =20 --export-dynamic -lphobos2 -lpthread -lm -lrt

Jan 18 2012
prev sibling next sibling parent "Martin Nowak" <dawg dawgfoto.de> writes:
On Wed, 18 Jan 2012 16:00:05 +0100, Andrea Fontana <advmail katamail.com>  
wrote:

 This param:
 --Xlinker --export-dynamic

 slows down the entire app (check() std.xml function) by 29x on test.

 Is it useful to add it on default config? Probably if you use external
 dl, you should add it...

 andrea ububox:~/src/xml-test$ dmd -c test.d && gcc test.o -o testgcc2
 -m64 -Xlinker --export-dynamic -Xlinker --no-warn-search-mismatch
 -lphobos2 -lpthread -lm -lrt
 andrea ububox:~/src/xml-test$ time ./testgcc2 cities.xml

 real	3m23.426s
 user	3m22.901s
 sys	0m0.276s

 andrea ububox:~/src/xml-test$ dmd -c test.d && gcc test.o -o testgcc2
 -m64 -Xlinker --no-warn-search-mismatch -lphobos2 -lpthread -lm -lrt
 andrea ububox:~/src/xml-test$ time ./testgcc2 cities.xml

 real	0m7.433s
 user	0m7.316s
 sys	0m0.112s

 Il giorno mer, 18/01/2012 alle 15.43 +0100, Trass3r ha scritto:

 dmd -c test.d && gcc test.o -lphobos2 -lrt -lpthread -o testgcc
 vs
 dmd -c test.d && dmd test.o -oftestd

Well dmd test.d calls gcc test.o -o test -m64 -Xlinker -L/dmd/linux/lib64 -Xlinker -L/dmd/linux/lib32 -Xlinker --no-warn-search-mismatch -Xlinker --export-dynamic -lphobos2 -lpthread -lm -lrt


Jan 18 2012
prev sibling next sibling parent Andrea Fontana <advmail katamail.com> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Not so much. Maybe inside check() of std.xml that check for xml
correctness and throw exception. It's the function that slow down the
entire app.

Il giorno mer, 18/01/2012 alle 16.26 +0100, Martin Nowak ha scritto:

 On Wed, 18 Jan 2012 16:00:05 +0100, Andrea Fontana <advmail katamail.com>=

 wrote:
=20
 This param:
 --Xlinker --export-dynamic

 slows down the entire app (check() std.xml function) by 29x on test.

 Is it useful to add it on default config? Probably if you use external
 dl, you should add it...

 andrea ububox:~/src/xml-test$ dmd -c test.d && gcc test.o -o testgcc2
 -m64 -Xlinker --export-dynamic -Xlinker --no-warn-search-mismatch
 -lphobos2 -lpthread -lm -lrt
 andrea ububox:~/src/xml-test$ time ./testgcc2 cities.xml

 real	3m23.426s
 user	3m22.901s
 sys	0m0.276s

 andrea ububox:~/src/xml-test$ dmd -c test.d && gcc test.o -o testgcc2
 -m64 -Xlinker --no-warn-search-mismatch -lphobos2 -lpthread -lm -lrt
 andrea ububox:~/src/xml-test$ time ./testgcc2 cities.xml

 real	0m7.433s
 user	0m7.316s
 sys	0m0.112s

 Il giorno mer, 18/01/2012 alle 15.43 +0100, Trass3r ha scritto:

 dmd -c test.d && gcc test.o -lphobos2 -lrt -lpthread -o testgcc
 vs
 dmd -c test.d && dmd test.o -oftestd

Well dmd test.d calls gcc test.o -o test -m64 -Xlinker -L/dmd/linux/lib64 -Xlinker -L/dmd/linux/lib32 -Xlinker --no-warn-search-mismatch -Xlinker --export-dynamic -lphobos2 -lpthread -lm -lrt



Jan 18 2012
prev sibling next sibling parent Andrea Fontana <advmail katamail.com> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

I'm agree but it seems it's used to provide stack traces too (I just
read it on an old post). So maybe if -shared or if we're in debug
mode...

Il giorno mer, 18/01/2012 alle 16.35 +0100, Jacob Carlborg ha scritto:

=20
 How about DMD passing --export-dynamic to GCC if -shared is passed to DMD=

=20

Jan 18 2012
prev sibling next sibling parent "Martin Nowak" <dawg dawgfoto.de> writes:
 try this code using a long xml file (tested here with a 2.0 mb list of
 italian cities) ...

Jan 18 2012
prev sibling next sibling parent Andrea Fontana <advmail katamail.com> writes:
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

I took a random medium-sized xml :)

By the way for this test I downloaded it:
http://www.ollie10.it/file.axd?file=3D2010%2f4%
2fDatabase_Comuni_Italiani_2010.zip (folder /xml/Cities.xml)

Il giorno mer, 18/01/2012 alle 18.28 +0100, Martin Nowak ha scritto:

 try this code using a long xml file (tested here with a 2.0 mb list of
 italian cities) ...


Jan 19 2012
prev sibling parent "Marco Leise" <Marco.Leise gmx.de> writes:
Am 18.01.2012, 16:35 Uhr, schrieb Jacob Carlborg <doob me.com>:

 How about DMD passing --export-dynamic to GCC if -shared is passed to  
 DMD?

--export-dynamic is required for the symbol names in exception back traces and comes from dmd.conf. That said, you can remove it there and enable it only for debug builds inside a Makefile to still get meaningful messages for failed asserts (back traces with files/line numbers).
Jan 19 2012