www.digitalmars.com         C & C++   DMDScript  

D - Why is the linker necessary?

reply "Giancarlo Bellido" <vicentico1 hotmail.com> writes:
I don't know that much about compilers, and I've been wondering why is
the linker necessary. Is it really necessary to first compile the code into
an
.obj file and then link it with the .lib files and other .obj files in the
program? I believe that the source code of a program has the necessary
information to build an executable, except for DLLs and other Libraries but
that can be implemented like Borland did in Delphi.

what about compiled modules instead of .obj files, imported with the
"import" keyword.

library function calls can be implemented like this...
extern(Windows, "dllname.dll") name(pars);

it would make the compilation process extremely easy. you could just type:

dmc main.d (-flags...) to compile a program with multiple modules and
library calls instead of
dmc module1.d module2.d module3.d main.d library1.lib library2.lib
library3.lib .. etc.

these are just some ideas.
Aug 21 2003
next sibling parent "Sean L. Palmer" <palmer.sean verizon.net> writes:
Good idea.  Internally there will be a link step, otherwise separate
compilation and incremental compilation would never work.  But we shouldn't
have to deal with it.

Along with that comes some other stuff:  A module has to be a file, the file
has to be in the right place, someplace very predictable so it can find it.
Maybe modules could "incorporate" some other module, which would then extend
it into multiple files.  This is like public imports.

This could slow compilation down somewhat.  Obviously it would be more than
made up for by not having to parse text headers.  But in order to discover
that in order to build module A, it must first build module B, but by the
time it finds out it needs B, it has already started compiling A.  It would
not want to discard the useful work it has done on A, so it chains out and
compiles be and comes back and finishes compiling A.  But that means you
could run low on memory as the compiler crawls thru potentially hundreds of
files figuring out what to do.  Once it's finished it should definitely
write out dependency files so it knows whether anything needs recompiled
without having to do the whole thing again.

Anyway I would love to be spared the details of the linking.  ;)

Sean

"Giancarlo Bellido" <vicentico1 hotmail.com> wrote in message
news:bi41d2$26ip$1 digitaldaemon.com...
 I don't know that much about compilers, and I've been wondering why is
 the linker necessary. Is it really necessary to first compile the code

 an
 .obj file and then link it with the .lib files and other .obj files in the
 program? I believe that the source code of a program has the necessary
 information to build an executable, except for DLLs and other Libraries

 that can be implemented like Borland did in Delphi.

 what about compiled modules instead of .obj files, imported with the
 "import" keyword.

 library function calls can be implemented like this...
 extern(Windows, "dllname.dll") name(pars);

 it would make the compilation process extremely easy. you could just type:

 dmc main.d (-flags...) to compile a program with multiple modules and
 library calls instead of
 dmc module1.d module2.d module3.d main.d library1.lib library2.lib
 library3.lib .. etc.

 these are just some ideas.

Aug 22 2003
prev sibling next sibling parent reply Richard Krehbiel <rich kastle.com> writes:
Giancarlo Bellido wrote:
 I don't know that much about compilers, and I've been wondering why is
 the linker necessary. Is it really necessary to first compile the code into
 an
 .obj file and then link it with the .lib files and other .obj files in the
 program? I believe that the source code of a program has the necessary
 information to build an executable, except for DLLs and other Libraries but
 that can be implemented like Borland did in Delphi.
 
 what about compiled modules instead of .obj files, imported with the
 "import" keyword.
 
 library function calls can be implemented like this...
 extern(Windows, "dllname.dll") name(pars);
 
 it would make the compilation process extremely easy. you could just type:
 
 dmc main.d (-flags...) to compile a program with multiple modules and
 library calls instead of
 dmc module1.d module2.d module3.d main.d library1.lib library2.lib
 library3.lib .. etc.
 
 these are just some ideas.

Well, once upon a time, compiling was a very lengthy procedure. It could take HOURS to compile your program AND the source for all the library functions it uses. If the library functions were precompiled, and then (say) LINKED with your program, then youd only have to compile your source, which is much faster (only 30 minutes or so). Of course, computers have gotten tens of thousands of times faster since then. It might not be so impractical to build everything from source each time. But there's still one concern, that the libraries you use might not be your own. In this case, the owner of the library might not want you to be able to see the source, but they're happy to sell you a pre-compiled library. Still... there's dynamic linking these days, too. Maybe all the pre-compiled libraries would be .DLL (or .so) libraries, and everything else would need to be compiled from source. Each and every time you change one line in one file of 70, all 70 get recompiled. Or all 170. Or all 700. Nah. I think we should keep the linker, for a while anyway.
Aug 22 2003
parent "Samuel Barber" <opendtv yahoo.com> writes:
"Richard Krehbiel" <rich kastle.com> wrote in message
news:bi4u6j$gld$1 digitaldaemon.com...
 Well, once upon a time, compiling was a very lengthy procedure.  It
 could take HOURS to compile your program AND the source for all the
 library functions it uses.  If the library functions were precompiled,
 and then (say) LINKED with your program, then youd only have to compile
 your source, which is much faster (only 30 minutes or so).

 Of course, computers have gotten tens of thousands of times faster since
 then.  It might not be so impractical to build everything from source
 each time.

 But there's still one concern, that the libraries you use might not be
 your own.  In this case, the owner of the library might not want you to
 be able to see the source, but they're happy to sell you a pre-compiled
 library.

A more basic point is that object modules can be created by different compilers/assemblers. It's no help to see the source if it isn't buildable with your (single) tool. Sam
Aug 25 2003
prev sibling next sibling parent Bill Cox <bill viasic.com> writes:
Giancarlo Bellido wrote:
 I don't know that much about compilers, and I've been wondering why is
 the linker necessary. Is it really necessary

I like to say (just because it's fun) that "The C linker is the Root of All Evil." Being compatible with it leads to hacked C++ template support. Most C++ programmers know about name mangling. Optimization across files still generally sucks (there are some fairly new compilers that globally optimize). The reason for the linker is mainly historical. Back in high school, I had a job programming a PDP-11/45. It had a whopping 128K of core memory (actual magnetic cores used for each bit). We shared it between 8 users at a time. My max memory allocation was 16K bytes. The compilers had to run off disk, in little pieces. We had "4 pass compilers", which were slower than "3 pass compilers." That meant that the source code had to be read 3 times instead of 4, which was the bottleneck in compilation speed. After creating all the obect files, the linker ran. That was always fun. We had these washing machine sized hard disks, and the linker sensitized an oscillation mode. Whenever we linked, one of these huge things would rock back and forth making a terrible niose, and then start walking around the room! I guess I haven't been fond of linkers ever since. Bill
Aug 22 2003
prev sibling next sibling parent "Mike Wynn" <mike.wynn l8night.co.uk> writes:
"Giancarlo Bellido" <vicentico1 hotmail.com> wrote in message
news:bi41d2$26ip$1 digitaldaemon.com...
 I don't know that much about compilers, and I've been wondering why is
 the linker necessary. Is it really necessary to first compile the code

 an
 .obj file and then link it with the .lib files and other .obj files in the
 program? I believe that the source code of a program has the necessary
 information to build an executable, except for DLLs and other Libraries

 that can be implemented like Borland did in Delphi.

if you have a big project incremental compilation saves a lot of time when making changes i.e. a make file that compiles d->obj then make libs then link the libs + program also there was a limit on command line length so you can't pass the whole project as one command. (Some OS's limit command line length anyway)
 what about compiled modules instead of .obj files, imported with the
 "import" keyword.

 library function calls can be implemented like this...
 extern(Windows, "dllname.dll") name(pars);

(explicit loading rather than implicit though)
 it would make the compilation process extremely easy. you could just type:

 dmc main.d (-flags...) to compile a program with multiple modules and
 library calls instead of
 dmc module1.d module2.d module3.d main.d library1.lib library2.lib
 library3.lib .. etc.

 these are just some ideas.

I guess ppl are all thinking a long similar lines .... I would also like to be able to do `dmd -build -Imylib -I/shared/otherlib -o myapp.exe main.d` and have the d compiler find and compile all files into myapp.exe I believe that the D compiler should use obj files in the same way C compiler use precompiled headers, so there should be a tempary dir (defaults to c:\dmd\inter\<mangled-path> or similar) which contains the precompiled Objs the D compiler just has to check that <path>file.d is older than c:\dmd\inter\<mangled-path>file.obj so work out if it need to read in the d file or the obj there is an issue with templates the easiest solution is that the obj file gets marked in someway to say its incomplete and its original source file will be read in every time. the required options are build me an exe, build a shared object (.so/.dll), build me a lib or for backward compatibility `-c` => build me an object file. this removes the requirement to have make files that contain the dependancy info, or tools that scan the D files looking for imports (he says wound up again 'cos adding a new member function to a class requires all the objects to be rebuild that use it or the prog crashes ... (obviously). and tracking dependacies is a pain). it would also seem in these days of IDE's that the D compiler should export an interface to allow it to be intergrated into an IDE again why write a tool to workout what the D compiler imports, when the D compiler is doing all that work anyway. dmd should just be a front end that loads dcompiler.dll (.so) and passes it the start file etc. the required interface (C or COM) should allow the following send a file to D to build in to an exe, .dll/.so /lib or obj the D compiler then calls back to request source or object (flags to say either or source only) and callbacks to store intermedite and final data. I would invisage that searching and intermediate storage are all processed in the frontend, so the D compiler does not have to bother with where things came from just that they are the right thing. this could be taken a step further (as D is intended to be useable as a backend lang) to perform jit like (well just ahead of time) compilation. and if an added callback (DGetProcAddress or sililar was added then a script could be run and link to the exe that started the dcompiler up). my vision .... extern (C) bool DGetDCompiler( out IDCompiler dc ); // in dcompiler.dll interface IDCompiler : IUnknown { enum OutputFormat { RAW, EXE, SHARED, LIB } bool beginCompilation( wchar[] filename, int tag, OutputFormat format, IDCompilerHelper helper ); } interface IDCompilerHelper : IUnknown { enum FileType { ANY = 0xF, SOURCE=1, OBJECT=2, LIB=3 } bool getFileItem( wchar[] name, int tag, FileType type, IDCompiler from, out IDInputStream src ); bool setFileItem( wchar[] name, int tag, FileType type, IDCompiler from, IDOutputStream src ); } // tag values >0 are helper side identifiers, values <0 are compiler generated tags for items // this allows you to send beginCompilation( "internal", 1, .... ) // then when item with tag 1 is requested you return generated D rather than a file. interface IDStream : IUnknown { void close(); bit valid(); } interface IDInputStream : IDStream { int read( byte[] buf, int max_len ); // returns the number of bytes read <0 == error } interface IDOutputStream : IDStream { int write( byte[] buf, int len ); // returns the number of bytes written <0 == error can write less than len }
Aug 22 2003
prev sibling parent "Carlos Santander B." <carlos8294 msn.com> writes:
Like Walter has said, he's trying to reuse the few resources he has at hand.
That's why he uses the linker.
However, I also think there should be a way for not having those really long
lines when linking. Maybe, in some way, the compiler could look for all the
needed .obj or .lib and pass directly to the linker. I mean, I have 2
versions of the same program: a console mode one and a graphics mode, but
both use the same core. Why should I be compiling each time that core? I
know I can pass just the .obj, but I'd rather just do
dmd -c core.d
dmd textmode.d
dmd guimode.d
And both compilations would automagically link with core.obj too. (I don't
use digc much because I don't know what the effect of that .res file is. I
only use it for compiling dig programs.)
And that idea of having some sort of a central repository could be really
useful too.

覧覧覧覧覧覧覧覧覧覧覧覧
Carlos Santander


---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.512 / Virus Database: 309 - Release Date: 2003-08-19
Aug 22 2003