www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - ImportC linking issue

reply confuzzled <con fuzzled.com> writes:
Wondering if someone can help me with this. Mr. Adam D. Ruppe got 
me 90% there, but I'm a little lost after reaching the 99% mark. 
I want to use XLSX I/O to manipulate excel spreadsheets.

I've cloned, built, and installed the library. And with the help 
of Adam, I am now able to import it and link to D program (as 
long as I'm not using anything from the lib). Wanted to continue 
the discussion with Adam, but I realize the announce forum isn't 
the place to ask for help.

Anyway, the library has two dependencies:

- [expat](http://www.libexpat.org/) (only for libxlsxio_read)
- [minizip](http://www.winimage.com/zLibDll/minizip.html) or 
libzip (libxlsxio_read and libxlsxio_write)

Since I'm on a Mac, I installed them using homebrew. In a demo.d 
file I have the following:

```d
import xlsxio_read;
import std.stdio;

pragma(lib, "libxlsxio_read.a");
pragma(lib, "libexpat.a");
pragma(lib, "libminizip.a");

string filename = "logistic_regression.xlsx";

void main (string[] args)
{
     if (args.length > 1)
         filename = args[1];

     xlsxioreader xlsxioread;

     writef("XLSX I/O library version %s\n", 
xlsxioread_get_version_string());
}
```

And in xlsxio_read.c (probably should call it something else), I 
simply import the library header:
```c
#include <xlsxio_read.h>
```

When I try to compile it, I get a slew of errors about undefined 
symbols. It wasn't finding libexpat and libminizip so I copied 
those to my work directory and tried again. With that, most of 
the errors disappeared. The remaining ones are:

```
dmd demo.d xlsxio_read.c
ld: warning: ignoring file libminizip.a, building for 
macOS-x86_64 but attempting to link with file built for 
macOS-x86_64
Undefined symbols for architecture x86_64:
   "_unzClose", referenced from:
       _xlsxioread_close in libxlsxio_read.a(xlsxio_read.c.o)
   "_unzCloseCurrentFile", referenced from:
       _expat_process_zip_file in libxlsxio_read.a(xlsxio_read.c.o)
       _xlsxioread_sheetlist_close in 
libxlsxio_read.a(xlsxio_read.c.o)
       _xlsxioread_sheet_close in libxlsxio_read.a(xlsxio_read.c.o)
   "_unzGetCurrentFileInfo", referenced from:
       _iterate_files_by_contenttype_expat_callback_element_start 
in libxlsxio_read.a(xlsxio_read.c.o)
   "_unzGetGlobalInfo", referenced from:
       _iterate_files_by_contenttype_expat_callback_element_start 
in libxlsxio_read.a(xlsxio_read.c.o)
   "_unzGoToFirstFile", referenced from:
       _iterate_files_by_contenttype_expat_callback_element_start 
in libxlsxio_read.a(xlsxio_read.c.o)
   "_unzGoToNextFile", referenced from:
       _iterate_files_by_contenttype_expat_callback_element_start 
in libxlsxio_read.a(xlsxio_read.c.o)
   "_unzLocateFile", referenced from:
       _XML_Char_openzip in libxlsxio_read.a(xlsxio_read.c.o)
   "_unzOpen", referenced from:
       _xlsxioread_open in libxlsxio_read.a(xlsxio_read.c.o)
   "_unzOpen2", referenced from:
       _xlsxioread_open_filehandle in 
libxlsxio_read.a(xlsxio_read.c.o)
       _xlsxioread_open_memory in libxlsxio_read.a(xlsxio_read.c.o)
   "_unzOpenCurrentFile", referenced from:
       _XML_Char_openzip in libxlsxio_read.a(xlsxio_read.c.o)
   "_unzReadCurrentFile", referenced from:
       _expat_process_zip_file in libxlsxio_read.a(xlsxio_read.c.o)
       _expat_process_zip_file_resume in 
libxlsxio_read.a(xlsxio_read.c.o)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to 
see invocation)
Error: linker exited with status 1
```
 From the first line of the error message, I gather I'm using the 
wrong minizip but I'm not sure how or why. What does **"building 
for macOS-x86_64 but attempting to link with file built for 
macOS-x86_64"** even mean?

Thanks,
confuzzled!!!
Nov 10 2022
parent reply confuzzled <con fuzzled.com> writes:
On Friday, 11 November 2022 at 03:15:17 UTC, confuzzled wrote:

 When I try to compile it, I get a slew of errors about 
 undefined symbols. It wasn't finding libexpat and libminizip so 
 I copied those to my work directory and tried again. With that, 
 most of the errors disappeared. The remaining ones are:


It seems that every time I resolve one of these undefined symbols 
issues, the compiler finds more. So I keep copying lib files from 
locations that are a path, to my working directory and linking 
them to my script. Is that the norm? Do I need to configure DMD 
somehow to recognize C libraries that are already in the path?


 Thanks,
 confuzzled!!!
Nov 11 2022
parent reply Mike Parker <aldacron gmail.com> writes:
On Saturday, 12 November 2022 at 02:45:52 UTC, confuzzled wrote:


 It seems that every time I resolve one of these undefined 
 symbols issues, the compiler finds more. So I keep copying lib 
 files from locations that are a path, to my working directory 
 and linking them to my script. Is that the norm? Do I need to 
 configure DMD somehow to recognize C libraries that are already 
 in the path?


The linker doesn't care if the libraries are C or D, and the 
compiler is only involved in that you can pass flags to the 
linker via the compiler command line.

These two things need to be true:

* any link-time dependencies (object files, static libraries, 
shared libraries (link libraries on Windows)) need to be passed 
to the linker
* the linker needs to know where to find libraries not on the 
default search path

The only thing the compiler passes along automatically are the 
object files generated from the source and the standard library 
it's building a binary. Anything else (including separately 
compiled object files), you have to pass along explicitly.

If you aren't explicitly passing any libraries along, then the 
linker won't know anything about them. The compiler doesn't know 
about them either, so can't pass them along for you.

If you are passing them along but they aren't on the default lib 
path, then the linker won't be able to find them.

Based on your description, it sounds like the last case is true 
for you. If so, if your libraries are in a common location, you 
can pass that path to the linker through dmd via 
`-L<linker-specific-flag>`. E.g., on Linux, `-L-L/path/to/libs`. 
On Windows, it depends on which linker you're using. For the 
Microsoft linker, it's `-L/LIBPATH path\\to\\libs`.

You can also just pass the full path to each library.
Nov 12 2022
parent reply confuzzled <con fuzzled.com> writes:
On Saturday, 12 November 2022 at 08:43:13 UTC, Mike Parker wrote:

 On Saturday, 12 November 2022 at 02:45:52 UTC, confuzzled wrote:

 The linker doesn't care if the libraries are C or D, and the 
 compiler is only involved in that you can pass flags to the 
 linker via the compiler command line.


Mike, first of all, thanks for the in depth response. That all 
makes sense. The issue I'm having is this: having made sure the 
two dependencies are available and building the 
libxlsxio_reader.a from the source without errors, why would I 
need to hunt down all the dependencies from that library to 
include them in my program?

I figured that importing the header and passing libxlsxio_read.a 
on the command line would be enough? Why would I have to search 
for libcrypto, libminizip, libexpat, and more that I haven't even 
figured out what library they are? I thought those dependencies 
would already be linked into libxlsxio_read.a which is a 
statically linked library.

I guess my noob is showing a lot. Again, thanks for your 
assistance.

confuzzled!!!
Nov 12 2022
next sibling parent reply Mike Parker <aldacron gmail.com> writes:
On Saturday, 12 November 2022 at 10:02:12 UTC, confuzzled wrote:

 On Saturday, 12 November 2022 at 08:43:13 UTC, Mike Parker 
 wrote:

 On Saturday, 12 November 2022 at 02:45:52 UTC, confuzzled 
 wrote:

 The linker doesn't care if the libraries are C or D, and the 
 compiler is only involved in that you can pass flags to the 
 linker via the compiler command line.


 Mike, first of all, thanks for the in depth response. That all 
 makes sense. The issue I'm having is this: having made sure the 
 two dependencies are available and building the 
 libxlsxio_reader.a from the source without errors, why would I 
 need to hunt down all the dependencies from that library to 
 include them in my program?

 I figured that importing the header and passing 
 libxlsxio_read.a on the command line would be enough? Why would 
 I have to search for libcrypto, libminizip, libexpat, and more 
 that I haven't even figured out what library they are? I 
 thought those dependencies would already be linked into 
 libxlsxio_read.a which is a statically linked library.


Static library dependencies are resolved at link time. Anything 
they need to link with, your binary must link with. It's shared 
libraries that have their static dependencies all baked in.
Nov 12 2022
parent reply confuzzled <con fuzzled.com> writes:
On Saturday, 12 November 2022 at 11:44:06 UTC, Mike Parker wrote:

 On Saturday, 12 November 2022 at 10:02:12 UTC, confuzzled wrote:

 On Saturday, 12 November 2022 at 08:43:13 UTC, Mike Parker 
 wrote:

 On Saturday, 12 November 2022 at 02:45:52 UTC, confuzzled 
 wrote:

 The linker doesn't care if the libraries are C or D, and the 
 compiler is only involved in that you can pass flags to the 
 linker via the compiler command line.


 Mike, first of all, thanks for the in depth response. That all 
 makes sense. The issue I'm having is this: having made sure 
 the two dependencies are available and building the 
 libxlsxio_reader.a from the source without errors, why would I 
 need to hunt down all the dependencies from that library to 
 include them in my program?

 I figured that importing the header and passing 
 libxlsxio_read.a on the command line would be enough? Why 
 would I have to search for libcrypto, libminizip, libexpat, 
 and more that I haven't even figured out what library they 
 are? I thought those dependencies would already be linked into 
 libxlsxio_read.a which is a statically linked library.


 Static library dependencies are resolved at link time. Anything 
 they need to link with, your binary must link with. It's shared 
 libraries that have their static dependencies all baked in.


Right, so I figured that the dependencies for for libxlsxio_read 
would be resolved when it was being compiled/linked. Therefore, 
when I used it later, I would just need to import its and include 
it on the command line. I didn't realize I would have to hunt 
down those libraries again and link them to my program, 
especially since I don't know what most of them are. I didn't 
have to provide them to the linker when I compiled libxlsxio_read.

Anyway, thanks for the explanation. I was definitely thinking and 
going about this all wrong. Appreciate the clarification.

confuzzled!!!
Nov 12 2022
parent confuzzled <con fuzzled.com> writes:
On Saturday, 12 November 2022 at 12:48:40 UTC, confuzzled wrote:

 Right, so I figured that the dependencies for for 
 libxlsxio_read would be resolved when it was being 
 compiled/linked. Therefore, when I used it later, I would just 
 need to import its and include it on the command line. I didn't



**import its header and provide the library on the command line***
Nov 12 2022
prev sibling parent Andrey Zherikov <andrey.zherikov gmail.com> writes:
On Saturday, 12 November 2022 at 10:02:12 UTC, confuzzled wrote:

 Why would I have to search for libcrypto, libminizip, libexpat, 
 and more that I haven't even figured out what library they are? 
 I thought those dependencies would already be linked into 
 libxlsxio_read.a which is a statically linked library.


This is called transitive dependencies.

For Unix systems linking a static library is not a linking per 
se, it means putting object files (.o) into an archive (.a) using 
[ar](https://linux.die.net/man/1/ar). So "libfoo.a" is just a set 
of object files that don't know anything about their dependencies 
and if you use something from libfoo.a, you might need to add 
another library to you link line. Technically there are two 
possible cases: (1) function that you use doesn't depend on 
anything else, then you don't need to add anything to your link 
line; and (2) function uses something from,  say libbar.a - then 
you have to add libbar.a to your link line.

Dynamic libraries (.so) are treated the same way as executables: 
they are created by linker ([ld](https://linux.die.net/man/1/ld) 
for example) and so they have to have all symbols resolved.  This 
means that if libfoo.so depends on libbar.a then the latter is 
already baked in into the former so your don't need to specify it 
when you link with libfoo.so. TBH I don't remember whether you 
should add libbar.so (dynamic library) which is a dependency of 
libfoo.so to your link line.
Nov 13 2022