www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - GDC fails to link with GSL and fortran code

reply "Andrew Brown" <aabrown24 hotmail.com> writes:
Hi,

I'm trying to compile code which calls C and fortan routines from 
D on the linux cluster at work. I've managed to get it to work 
with all 3 compilers on my laptop, but LDC and GDC fail on the 
cluster (though DMD works perfectly). I'm using the precompiled 
compiler binaries on these systems, the cluster doesn't have the 
prerequistites for building them myself and I don't have admin 
rights.

For GDC the commands I run are:

gcc -c C_code.c Fortran_code.f
gdc D_code.d C_code.o Fortran_code.f -lblas -lgsl -lgslcblas -lm 
-lgfortran -o out

The error messages are:

/software/lib64/libgsl.so: undefined reference to 
`memcpy GLIBC_2.14'
/software/lib64/libgfortran.so.3: undefined reference to 
`clock_gettime GLIBC_2.17'
/software/lib64/libgfortran.so.3: undefined reference to 
`secure_getenv GLIBC_2.17'
collect2: error: ld returned 1 exit status

I can remove the gsl messages by statically linking to libgsl.a, 
but this doesn't solve the gfortran issues.

If anyone knows a way round these issues, I'd be very grateful. 
I'd also eventually like to find a way to easily share linux 
biniaries with people, so they can use this code without these 
kinds of headaches. If anyone has any advice for making this 
portable, that would also help me out a lot.

Thanks very much

Andrew
Mar 16 2015
next sibling parent ketmar <ketmar ketmar.no-ip.org> writes:
On Mon, 16 Mar 2015 16:44:45 +0000, Andrew Brown wrote:

can you please show output of `dmd -v` and `gdc -v`, so we can see the=20
actual commands they both using for linking?
=
Mar 16 2015
prev sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Mon, 16 Mar 2015 16:44:45 +0000
schrieb "Andrew Brown" <aabrown24 hotmail.com>:

 Hi,
 
 I'm trying to compile code which calls C and fortan routines from 
 D on the linux cluster at work. I've managed to get it to work 
 with all 3 compilers on my laptop, but LDC and GDC fail on the 
 cluster (though DMD works perfectly). I'm using the precompiled 
 compiler binaries on these systems, the cluster doesn't have the 
 prerequistites for building them myself and I don't have admin 
 rights.
 
 For GDC the commands I run are:
 
 gcc -c C_code.c Fortran_code.f
 gdc D_code.d C_code.o Fortran_code.f -lblas -lgsl -lgslcblas -lm 
 -lgfortran -o out
 
You could try to do the linking with the local compiler: gdc D_code.d gcc D_code.o C_code.o Fortran_code.o -lgphobos2 -lpthread -lblas -lgsl -lgslcblas -lm -L path/to/x86_64-gdcproject-linux-gnu/lib/
 The error messages are:
 
 /software/lib64/libgsl.so: undefined reference to 
 `memcpy GLIBC_2.14'
 /software/lib64/libgfortran.so.3: undefined reference to 
 `clock_gettime GLIBC_2.17'
 /software/lib64/libgfortran.so.3: undefined reference to 
 `secure_getenv GLIBC_2.17'
 collect2: error: ld returned 1 exit status
 
Seems like the binary GDC toolchain somehow picks up a wrong libc. The toolchains are built with GLIBC 2.14. But IIRC we don't ship the libc in the binary packages (for native compilers) and it should pick up the local libc. Please run gdc with the '-v' and '-Wl,--verbose' options and post a link to the full output.
 I can remove the gsl messages by statically linking to libgsl.a, 
 but this doesn't solve the gfortran issues.
 
 If anyone knows a way round these issues, I'd be very grateful. 
 I'd also eventually like to find a way to easily share linux 
 biniaries with people, so they can use this code without these 
 kinds of headaches. If anyone has any advice for making this 
 portable, that would also help me out a lot.
 
Usually the best option is to compile on old linux systems. Binaries often run on newer systems but not on older ones. You could setup debian wheezy or an older version in a VM or using docker. Or you use docker.io ;-) I personally think the docker approach is kind of overkill but avoiding compatibility issues is one of docker's main selling points.
Mar 16 2015
parent reply "Andrew Brown" <aabrown24 hotmail.com> writes:
Thank you very much for your replies, I now have 2 solutions to 
my problem! Both compiling on a virtual machine running debian 
wheezy, and using gcc to do the linking produced executables that 
would run on the cluster.

Compiling with the verbose flags for linker and compiler produced 
the following output:

failed gdc attempt: http://dpaste.com/0Z5V4PV

successful dmd attempt: http://dpaste.com/0S5WKJ5

successful use of gcc to link: http://dpaste.com/0YYR39V

It seems a bit of a mess, with various libraries in various 
places. I'll see if I can get to the bottom of it, I think it'll 
be a learning experience.

Thanks again for the swift and useful help and guidance.

Andrew

On Monday, 16 March 2015 at 19:22:18 UTC, Johannes Pfau wrote:
 Am Mon, 16 Mar 2015 16:44:45 +0000
 schrieb "Andrew Brown" <aabrown24 hotmail.com>:

 Hi,
 
 I'm trying to compile code which calls C and fortan routines 
 from D on the linux cluster at work. I've managed to get it to 
 work with all 3 compilers on my laptop, but LDC and GDC fail 
 on the cluster (though DMD works perfectly). I'm using the 
 precompiled compiler binaries on these systems, the cluster 
 doesn't have the prerequistites for building them myself and I 
 don't have admin rights.
 
 For GDC the commands I run are:
 
 gcc -c C_code.c Fortran_code.f
 gdc D_code.d C_code.o Fortran_code.f -lblas -lgsl -lgslcblas 
 -lm -lgfortran -o out
 
You could try to do the linking with the local compiler: gdc D_code.d gcc D_code.o C_code.o Fortran_code.o -lgphobos2 -lpthread -lblas -lgsl -lgslcblas -lm -L path/to/x86_64-gdcproject-linux-gnu/lib/
 The error messages are:
 
 /software/lib64/libgsl.so: undefined reference to 
 `memcpy GLIBC_2.14'
 /software/lib64/libgfortran.so.3: undefined reference to 
 `clock_gettime GLIBC_2.17'
 /software/lib64/libgfortran.so.3: undefined reference to 
 `secure_getenv GLIBC_2.17'
 collect2: error: ld returned 1 exit status
 
Seems like the binary GDC toolchain somehow picks up a wrong libc. The toolchains are built with GLIBC 2.14. But IIRC we don't ship the libc in the binary packages (for native compilers) and it should pick up the local libc. Please run gdc with the '-v' and '-Wl,--verbose' options and post a link to the full output.
 I can remove the gsl messages by statically linking to 
 libgsl.a, but this doesn't solve the gfortran issues.
 
 If anyone knows a way round these issues, I'd be very 
 grateful. I'd also eventually like to find a way to easily 
 share linux biniaries with people, so they can use this code 
 without these kinds of headaches. If anyone has any advice for 
 making this portable, that would also help me out a lot.
 
Usually the best option is to compile on old linux systems. Binaries often run on newer systems but not on older ones. You could setup debian wheezy or an older version in a VM or using docker. Or you use docker.io ;-) I personally think the docker approach is kind of overkill but avoiding compatibility issues is one of docker's main selling points.
Mar 17 2015
parent reply Johannes Pfau <nospam example.com> writes:
Am Tue, 17 Mar 2015 12:13:44 +0000
schrieb "Andrew Brown" <aabrown24 hotmail.com>:

 Thank you very much for your replies, I now have 2 solutions to 
 my problem! Both compiling on a virtual machine running debian 
 wheezy, and using gcc to do the linking produced executables that 
 would run on the cluster.
 
 Compiling with the verbose flags for linker and compiler produced 
 the following output:
 
 failed gdc attempt: http://dpaste.com/0Z5V4PV
 
 successful dmd attempt: http://dpaste.com/0S5WKJ5
 
 successful use of gcc to link: http://dpaste.com/0YYR39V
 
 It seems a bit of a mess, with various libraries in various 
 places. I'll see if I can get to the bottom of it, I think it'll 
 be a learning experience.
 
 Thanks again for the swift and useful help and guidance.
 
 Andrew
GCC's verbose output can indeed be quite confusing but if you know what to look for it's possible to find some useful information :-) In your case the linker messages hinted at a problem with libc. And as there were only a few errors it's likely a version compatibility problem. If you search for libc.so in these logs you'll find this: Failed GDC: attempt to open /usr/lib/../lib64/libc.so succeeded opened script file /usr/lib/../lib64/libc.so opened script file /usr/lib/../lib64/libc.so attempt to open /lib64/libc.so.6 succeeded /lib64/libc.so.6 GCC: attempt to open /software/lib/gcc/x86_64-redhat-linux/4.9.1/../../../../lib64/libc.so succeeded opened script file /software/lib/gcc/x86_64-redhat-linux/4.9.1/../../../../lib64/libc.so opened script file /software/lib/gcc/x86_64-redhat-linux/4.9.1/../../../../lib64/libc.so attempt to open /software/lib64/libc.so.6 succeeded /software/lib64/libc.so.6 The binary gdc searches for libaries in the 'usual' places, including /usr/lib64. Your gcc doesn't search in /usr/lib64 but in /software. You seem to have an incompatible libc in /usr/lib64 which gets picked up by gdc. This is one reason why binary compiler releases are difficult to maintain and we usually recommend to compile gdc from source. DMD avoids this mess by simply calling the local gcc instead of ld to link. GCC unfortunately doesn't support this and forces us to always call the linker directly.
Mar 17 2015
parent "Andrew Brown" <aabrown24 hotmail.com> writes:
I'm getting some idea how difficult compilers could be to 
maintain across  distros, but I'm probably still a long way from 
knowing the true horror.

Thank you for taking the time to help me out, and then when my 
immediate problem was solved, more time to help me learn 
something.

I see there's a lot of discussion on these forums about the state 
of the documentation. Between googling historical questions here, 
and people's willingness to help on this forum, I've always found 
my answers. I think that's quite amazing.

Andrew

On Tuesday, 17 March 2015 at 21:00:46 UTC, Johannes Pfau wrote:
 Am Tue, 17 Mar 2015 12:13:44 +0000
 schrieb "Andrew Brown" <aabrown24 hotmail.com>:

 Thank you very much for your replies, I now have 2 solutions 
 to my problem! Both compiling on a virtual machine running 
 debian wheezy, and using gcc to do the linking produced 
 executables that would run on the cluster.
 
 Compiling with the verbose flags for linker and compiler 
 produced the following output:
 
 failed gdc attempt: http://dpaste.com/0Z5V4PV
 
 successful dmd attempt: http://dpaste.com/0S5WKJ5
 
 successful use of gcc to link: http://dpaste.com/0YYR39V
 
 It seems a bit of a mess, with various libraries in various 
 places. I'll see if I can get to the bottom of it, I think 
 it'll be a learning experience.
 
 Thanks again for the swift and useful help and guidance.
 
 Andrew
GCC's verbose output can indeed be quite confusing but if you know what to look for it's possible to find some useful information :-) In your case the linker messages hinted at a problem with libc. And as there were only a few errors it's likely a version compatibility problem. If you search for libc.so in these logs you'll find this: Failed GDC: attempt to open /usr/lib/../lib64/libc.so succeeded opened script file /usr/lib/../lib64/libc.so opened script file /usr/lib/../lib64/libc.so attempt to open /lib64/libc.so.6 succeeded /lib64/libc.so.6 GCC: attempt to open /software/lib/gcc/x86_64-redhat-linux/4.9.1/../../../../lib64/libc.so succeeded opened script file /software/lib/gcc/x86_64-redhat-linux/4.9.1/../../../../lib64/libc.so opened script file /software/lib/gcc/x86_64-redhat-linux/4.9.1/../../../../lib64/libc.so attempt to open /software/lib64/libc.so.6 succeeded /software/lib64/libc.so.6 The binary gdc searches for libaries in the 'usual' places, including /usr/lib64. Your gcc doesn't search in /usr/lib64 but in /software. You seem to have an incompatible libc in /usr/lib64 which gets picked up by gdc. This is one reason why binary compiler releases are difficult to maintain and we usually recommend to compile gdc from source. DMD avoids this mess by simply calling the local gcc instead of ld to link. GCC unfortunately doesn't support this and forces us to always call the linker directly.
Mar 17 2015