www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - What can be done to reduce executable size?

reply Bane <branimir.milosavljevic gmail.com> writes:
Short term and long term suggestions ? Anything we can do ? I heard it 
is some problem with linking dead code?



import std.stdio;
int main(){
   writefln("Hello Bloat!");
   return 0;
}

dmd -release -O hello.d

On Windows:
v1.071 = 339 Kb
v2.056 = 1017 Kb

It looks very ugly and might distract some people.
Dec 10 2011
next sibling parent reply Mirko Pilger <mirko.pilger gmail.com> writes:
 On Windows:
 v1.071 = 339 Kb
 v2.056 = 1017 Kb
v2.057b= 840 kb (upx --best = 151 kb)
Dec 10 2011
parent reply Bane <branimir.milosavljevic gmail.com> writes:
Mirko Pilger Wrote:

 On Windows:
 v1.071 = 339 Kb
 v2.056 = 1017 Kb
v2.057b= 840 kb (upx --best = 151 kb)
That is improvement. 2.07 is not released yet ? And I don't think UPX is solution. It makes things look even worse, like too much makeup on ugly chick.
Dec 10 2011
parent reply David Nadlinger <see klickverbot.at> writes:
On 12/10/11 4:55 PM, Bane wrote:
 That is improvement. 2.07 is not released yet ?
Yeah, 2.057 is still in beta (expect a release during the next few days, though). It has some Phobos/druntime changes geared specifically towards reducing executable size. David
Dec 10 2011
parent reply Bane <branimir.milosavljevic gmail.com> writes:
Is it possible to move  phobos or runtime to shared lib ? It would reduces code
significantly.
Dec 10 2011
next sibling parent reply Bane <branimir.milosavljevic gmail.com> writes:
I am dealing with scenario of large numbers of programs written in D placed on
same host/1 installer, when it all sums up size does matters.

 Is it possible to move  phobos or runtime to shared lib ? It would reduces
code significantly.
Dec 10 2011
parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
Try using the unilink linker:

ftp://ftp.styx.cabel.net/pub/UniLink/

Get ulnb0329.zip

You have to configure ulink.cfg to this:
-zsnn.lib
-LC:\dmd\windows\lib
-LC:\dm\lib
-Go

-zkernel32;advapi32;user32;wsock32;shell32;snn.lib
-LC:\dmd2\windows\lib
-Go

Then linking is just: ulink <file1>.obj <file2>.obj <lib>.obj etc..
Dec 10 2011
parent Bane <branimir.milosavljevic gmail.com> writes:
Andrej Mitrovic Wrote:

 Try using the unilink linker:
 
 ftp://ftp.styx.cabel.net/pub/UniLink/
 
 Get ulnb0329.zip
 
 You have to configure ulink.cfg to this:
 -zsnn.lib
 -LC:\dmd\windows\lib
 -LC:\dm\lib
 -Go
 
 -zkernel32;advapi32;user32;wsock32;shell32;snn.lib
 -LC:\dmd2\windows\lib
 -Go
 
 Then linking is just: ulink <file1>.obj <file2>.obj <lib>.obj etc..
It is nice tool. Too bad it is still beta & proprietary, or am I mistaken ?
Dec 10 2011
prev sibling parent Trass3r <un known.com> writes:
Am 10.12.2011, 17:17 Uhr, schrieb Bane <branimir.milosavljevic gmail.com>:

 Is it possible to move  phobos or runtime to shared lib ? It would  
 reduces code significantly.
Theoretically but there are some hurdles (esp. regarding the runtime/gc). Some people have already tried this (on Linux of course), I'm not sure what the current status is.
Dec 10 2011
prev sibling next sibling parent reply Trass3r <un known.com> writes:
 Short term and long term suggestions ? Anything we can do ? I heard it  
 is some problem with linking dead code?
Well most space is covered by the runtime, TypeInfo and stuff like struct .init data. Use http://thecybershadow.net/d/mapview to get a graphical view of your app. E.g. on Linux it sometimes shows Afterpadding way too large.
Dec 10 2011
next sibling parent Bane <branimir.milosavljevic gmail.com> writes:
Trass3r Wrote:

 Short term and long term suggestions ? Anything we can do ? I heard it  
 is some problem with linking dead code?
Well most space is covered by the runtime, TypeInfo and stuff like struct .init data. Use http://thecybershadow.net/d/mapview to get a graphical view of your app. E.g. on Linux it sometimes shows Afterpadding way too large.
Yes, I use it, great piece of work.
Dec 10 2011
prev sibling parent Trass3r <un known.com> writes:
 Use http://thecybershadow.net/d/mapview to get a graphical view of your  
 app.
 E.g. on Linux it sometimes shows Afterpadding way too large.
I meant there still are some quirks though like Afterpadding shown way too large cause of ld map file parsing errors.
Dec 10 2011
prev sibling next sibling parent reply Mehrdad <wfunction hotmail.com> writes:
On 12/10/2011 7:39 AM, Bane wrote:
 Short term and long term suggestions ? Anything we can do ? I heard it 
 is some problem with linking dead code?



 import std.stdio;
 int main(){
   writefln("Hello Bloat!");
   return 0;
 }

 dmd -release -O hello.d

 On Windows:
 v1.071 = 339 Kb
 v2.056 = 1017 Kb

 It looks very ugly and might distract some people.
If Walter simply releases a few small parts of SNN.lib, it's quite possible to use your own C runtime instead. And that will lower the executable size down to ~10 KB.
Dec 10 2011
parent reply Trass3r <un known.com> writes:
 If Walter simply releases a few small parts of SNN.lib, it's quite  
 possible to use your own C runtime instead. And that will lower the  
 executable size down to ~10 KB.
Impossible. You forget the D runtime, type info and all the crap from phobos that gets pulled in. http://thecybershadow.net/d/mapview/view.php?id=4ee3af86c32f2
Dec 10 2011
parent reply Mehrdad <wfunction hotmail.com> writes:
On 12/10/2011 11:18 AM, Trass3r wrote:
 If Walter simply releases a few small parts of SNN.lib, it's quite 
 possible to use your own C runtime instead. And that will lower the 
 executable size down to ~10 KB.
Impossible. You forget the D runtime, type info and all the crap from phobos that gets pulled in. http://thecybershadow.net/d/mapview/view.php?id=4ee3af86c32f2
Sorry, I meant EXCLUDING the 'hello world' part. i.e. a program that (literally) does nothing should not require anything except argument-parsing, which can be done through msvcrt.dll's __wgetmainargs(). So really, it should hardly need any code at all.
Dec 10 2011
parent reply Trass3r <un known.com> writes:
 Sorry, I meant EXCLUDING the 'hello world' part.

 i.e. a program that (literally) does nothing should not require anything  
 except argument-parsing, which can be done through msvcrt.dll's  
 __wgetmainargs(). So really, it should hardly need any code at all.
I know it shouldn't. But currently you still have to statically link the runtime, gc, etc. even in an empty program: http://thecybershadow.net/d/mapview/view.php?id=4ee3b98636422
Dec 10 2011
parent reply Mehrdad <wfunction hotmail.com> writes:
On 12/10/2011 11:59 AM, Trass3r wrote:
 Sorry, I meant EXCLUDING the 'hello world' part.

 i.e. a program that (literally) does nothing should not require 
 anything except argument-parsing, which can be done through 
 msvcrt.dll's __wgetmainargs(). So really, it should hardly need any 
 code at all.
I know it shouldn't. But currently you still have to statically link the runtime, gc, etc. even in an empty program: http://thecybershadow.net/d/mapview/view.php?id=4ee3b98636422
Right, but my point is, I could work around (I've gotten within inches of it!) it if simply Walter released a TINY part of snn.lib -- just a handful of tiny source files regarding the TLS-related stuff, EXE segment markers, and whatnot. (I've already raised this issue before, and precisely what we would need, but it seemed to go completely ignored.)
Dec 10 2011
parent reply Trass3r <un known.com> writes:
 Right, but my point is, I could work around (I've gotten within inches  
 of it!) it if simply Walter released a TINY part of snn.lib -- just a  
 handful of tiny source files regarding the TLS-related stuff, EXE  
 segment markers, and whatnot. (I've already raised this issue before,  
 and precisely what we would need, but it seemed to go completely  
 ignored.)
Nag again and again :)
Dec 10 2011
parent "Nick Sabalausky" <a a.a> writes:
"Trass3r" <un known.com> wrote in message news:op.v6ai1yjc3ncmek enigma...
 Right, but my point is, I could work around (I've gotten within inches 
 of it!) it if simply Walter released a TINY part of snn.lib -- just a 
 handful of tiny source files regarding the TLS-related stuff, EXE 
 segment markers, and whatnot. (I've already raised this issue before, 
 and precisely what we would need, but it seemed to go completely 
 ignored.)
Nag again and again :)
That's how we got "-wi" ;)
Dec 11 2011
prev sibling next sibling parent reply Trass3r <un known.com> writes:
 import std.stdio;
 int main(){
    writefln("Hello Bloat!");
    return 0;
 }

 dmd -release -O hello.d

 On Windows:
 v1.071 = 339 Kb
 v2.056 = 1017 Kb
$ dmd -release -O test.d 867K $ strip -s test 572K I don't know where the 300KB come from. The map files are equal: dmd -release -O -map test.d dmd -release -O -L-s -map test.d This is what Hello World looks like on x64 Linux: http://thecybershadow.net/d/mapview/view.php?id=4ee3af86c32f2
Dec 10 2011
next sibling parent kenji hara <k.hara.pg gmail.com> writes:
Maybe it increases from 2.055.
Now std.uni module has big tables for dealing with Unicode code points
correctly.

And, the pair of 2.057 and new std.regex module has same issue.

Kenji Hara

2011/12/11 Trass3r <un known.com>:
 import std.stdio;
 int main(){
 =A0 writefln("Hello Bloat!");
 =A0 return 0;
 }

 dmd -release -O hello.d

 On Windows:
 v1.071 =3D 339 Kb
 v2.056 =3D 1017 Kb
$ dmd -release -O test.d 867K $ strip -s test 572K I don't know where the 300KB come from. The map files are equal: dmd -release -O -map test.d dmd -release -O -L-s -map test.d This is what Hello World looks like on x64 Linux: http://thecybershadow.net/d/mapview/view.php?id=3D4ee3af86c32f2
Dec 10 2011
prev sibling parent reply "Martin Nowak" <dawg dawgfoto.de> writes:
The symbol table.
Probably you also had debug infos for phobos?
By the way if I strip this, my executable is only 292K.

On Sat, 10 Dec 2011 20:16:54 +0100, Trass3r <un known.com> wrote:

 import std.stdio;
 int main(){
    writefln("Hello Bloat!");
    return 0;
 }

 dmd -release -O hello.d

 On Windows:
 v1.071 = 339 Kb
 v2.056 = 1017 Kb
$ dmd -release -O test.d 867K $ strip -s test 572K I don't know where the 300KB come from. The map files are equal: dmd -release -O -map test.d dmd -release -O -L-s -map test.d This is what Hello World looks like on x64 Linux: http://thecybershadow.net/d/mapview/view.php?id=4ee3af86c32f2
Dec 12 2011
parent Trass3r <un known.com> writes:
Am 12.12.2011, 20:33 Uhr, schrieb Martin Nowak <dawg dawgfoto.de>:

 The symbol table.
 Probably you also had debug infos for phobos?
Possible.
 By the way if I strip this, my executable is only 292K.
32bits?
Dec 12 2011
prev sibling next sibling parent "RivenTheMage" <riven-mage id.ru> writes:
 Short term and long term suggestions ? Anything we can do ?
I'm using UniLink.
 import std.stdio;
 int main(){
 writefln("Hello Bloat!");
 return 0;
 }
On Windows: DMD v2.056 UniLink 1.07 build 3.21 --- import std.stdio; int main() { writefln("Hello World!"); return 0; } --- dmd.exe -I"C:\DMD2\src\phobos" -c helloworld.d -of"helloworld.obj" ulink.exe -ap -zsnn.lib -L"C:\DMD2\windows\lib" -L"C:\DMC\lib" -Go "helloworld.obj", "helloworld.exe", , phobos.lib, , --- helloworld.exe = 276 Kb
Dec 10 2011
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2011-12-10 16:39, Bane wrote:
 Short term and long term suggestions ? Anything we can do ? I heard it
 is some problem with linking dead code?



 import std.stdio;
 int main(){
 writefln("Hello Bloat!");
 return 0;
 }

 dmd -release -O hello.d

 On Windows:
 v1.071 = 339 Kb
 v2.056 = 1017 Kb

 It looks very ugly and might distract some people.
As long as the runtime and standard library is statically linked the executables will be bigger than the corresponding C/C++ executable. -- /Jacob Carlborg
Dec 11 2011
parent reply Adam Ruppe <destructionator gmail.com> writes:
Jacob Carlborg Wrote:
 As long as the runtime and standard library is statically linked the 
 executables will be bigger than the corresponding C/C++ executable.
I just want to say it's very important to me that static linking still just works very easily even if we start to offer dynamic linking.
Dec 11 2011
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, December 11, 2011 17:28:58 Adam Ruppe wrote:
 Jacob Carlborg Wrote:
 As long as the runtime and standard library is statically linked the
 executables will be bigger than the corresponding C/C++ executable.
I just want to say it's very important to me that static linking still just works very easily even if we start to offer dynamic linking.
Most definitely. I consider dynamic linking to be a necessary evil which should not be used unless you have to. I _much_ prefer having my programs completely self-contained. The less that they rely on in terms of external libraries the better. Sure, there are plenty of cases where dynamic libraries are necessary (e.g. plugins), and the fact that they generally reduce disk space consumption is useful, but it's _so_ nice to not have to worry about the exact versions of everything else installed on the system. - Jonathan M Davis
Dec 11 2011
next sibling parent "news.digitalmars.com" <Oleg.K a.a> writes:
 Most definitely. I consider dynamic linking to be a necessary evil which 
 should
 not be used unless you have to. I _much_ prefer having my programs 
 completely
 self-contained. The less that they rely on in terms of external libraries 
 the
 better. Sure, there are plenty of cases where dynamic libraries are 
 necessary
 (e.g. plugins), and the fact that they generally reduce disk space 
 consumption
 is useful, but it's _so_ nice to not have to worry about the exact 
 versions of
 everything else installed on the system.

 - Jonathan M Davis
Nobody cares about disk space this time. Imagine x00 applications run in memory with exact copy of libc. Oleg.
Dec 11 2011
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2011-12-11 23:55, Jonathan M Davis wrote:
 On Sunday, December 11, 2011 17:28:58 Adam Ruppe wrote:
 Jacob Carlborg Wrote:
 As long as the runtime and standard library is statically linked the
 executables will be bigger than the corresponding C/C++ executable.
I just want to say it's very important to me that static linking still just works very easily even if we start to offer dynamic linking.
Most definitely. I consider dynamic linking to be a necessary evil which should not be used unless you have to. I _much_ prefer having my programs completely self-contained. The less that they rely on in terms of external libraries the better. Sure, there are plenty of cases where dynamic libraries are necessary (e.g. plugins), and the fact that they generally reduce disk space consumption is useful, but it's _so_ nice to not have to worry about the exact versions of everything else installed on the system. - Jonathan M Davis
It is very nice to not have to think about external dependencies when installing a tool or library, but as you say for plugins it is important. I would hope that it is possible to have the application completely statically linked but at the same time provide plugins for the application. -- /Jacob Carlborg
Dec 12 2011
parent Andrea Fontana <advmail katamail.com> writes:
You can try with upx :)

Il giorno lun, 12/12/2011 alle 14.42 +0100, Jacob Carlborg ha scritto:

 On 2011-12-11 23:55, Jonathan M Davis wrote:
 On Sunday, December 11, 2011 17:28:58 Adam Ruppe wrote:
 Jacob Carlborg Wrote:
 As long as the runtime and standard library is statically linked the
 executables will be bigger than the corresponding C/C++ executable.
I just want to say it's very important to me that static linking still just works very easily even if we start to offer dynamic linking=
.
 Most definitely. I consider dynamic linking to be a necessary evil whic=
h should
 not be used unless you have to. I _much_ prefer having my programs comp=
letely
 self-contained. The less that they rely on in terms of external librari=
es the
 better. Sure, there are plenty of cases where dynamic libraries are nec=
essary
 (e.g. plugins), and the fact that they generally reduce disk space cons=
umption
 is useful, but it's _so_ nice to not have to worry about the exact vers=
ions of
 everything else installed on the system.

 - Jonathan M Davis
=20 It is very nice to not have to think about external dependencies when=20 installing a tool or library, but as you say for plugins it is=20 important. I would hope that it is possible to have the application=20 completely statically linked but at the same time provide plugins for=20 the application. =20
Dec 12 2011
prev sibling next sibling parent reply Martin Krejcirik <mk-junk i-line.cz> writes:
On 10.12.2011 16:39, Bane wrote:
 Short term and long term suggestions ? Anything we can do ? I heard it
 is some problem with linking dead code?
I don't know about D2, but for D1 it helps to recompile Phobos without -lib (use lib.exe, see win32.mak). Hello.d - 84k I'm attaching my makefiles, the smaller one belongs to phobos/internal/gc Martin
Dec 13 2011
parent reply bearophile <bearophileHUGS lycos.com> writes:
Martin Krejcirik:

 I don't know about D2, but for D1 it helps to recompile Phobos without
 -lib (use lib.exe, see win32.mak).
What are the effects/disadvantages of doing this? Bye, bearophile
Dec 14 2011
parent Jacob Carlborg <doob me.com> writes:
On 2011-12-14 12:47, bearophile wrote:
 Martin Krejcirik:

 I don't know about D2, but for D1 it helps to recompile Phobos without
 -lib (use lib.exe, see win32.mak).
What are the effects/disadvantages of doing this? Bye, bearophile
You need to explicitly invoke the tool that creates libraries on the given system (ar on Posix, lib on Windows) making it platform dependent. -- /Jacob Carlborg
Dec 14 2011
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/10/11 9:39 AM, Bane wrote:
 Short term and long term suggestions ? Anything we can do ? I heard it
 is some problem with linking dead code?



 import std.stdio;
 int main(){
 writefln("Hello Bloat!");
 return 0;
 }

 dmd -release -O hello.d

 On Windows:
 v1.071 = 339 Kb
 v2.056 = 1017 Kb

 It looks very ugly and might distract some people.
In fact there was a low-hanging fruit, and I'm sure there are some more. This diff reduces the size of hello, world (compiled with -O -release -inline and after strip) from 700KB to 220 KB: https://github.com/D-Programming-Language/phobos/commit/b7f42ec925fb1d64564d48ea419e201bfc65ed53 Right now an executable starts at around 218KB, which includes druntime (gc, type info, the works). Importing std.stdio and using writeln() only adds a couple of KBs. Andrei
Dec 15 2011
next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, December 15, 2011 21:40:57 Andrei Alexandrescu wrote:
 On 12/10/11 9:39 AM, Bane wrote:
 Short term and long term suggestions ? Anything we can do ? I heard it
 is some problem with linking dead code?
 
 
 
 import std.stdio;
 int main(){
 writefln("Hello Bloat!");
 return 0;
 }
 
 dmd -release -O hello.d
 
 On Windows:
 v1.071 = 339 Kb
 v2.056 = 1017 Kb
 
 It looks very ugly and might distract some people.
In fact there was a low-hanging fruit, and I'm sure there are some more. This diff reduces the size of hello, world (compiled with -O -release -inline and after strip) from 700KB to 220 KB: https://github.com/D-Programming-Language/phobos/commit/b7f42ec925fb1d64564d 48ea419e201bfc65ed53 Right now an executable starts at around 218KB, which includes druntime (gc, type info, the works). Importing std.stdio and using writeln() only adds a couple of KBs.
Simply making it so that std.file is only imported in std.stdio with version(unittest) cut off _that_ much? - Jonathan M Davis
Dec 15 2011
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/16/11 1:12 AM, Jonathan M Davis wrote:
 Simply making it so that std.file is only imported in std.stdio with
 version(unittest) cut off _that_ much?
Yah, but the matter is more complex. The issue is that std.file pulls std.datetime, which (a) has static this() code, and (b) pulls core.time, which in turn has static this() code. The issue with that is as follows. Any file that transitively imports a module with constructors will have its own module info generated. When that happens, all vtables in that module will be instantiated, so all methods will be linked in. That in turn causes all functions they call to also be linked in. That's why many programs using std are large. We can attack this in two ways: 1. Revise and reduce all static this() uses in phobos and druntime; 2. Improve the compiler to do minimal linking when static this() does come about. Andrei
Dec 16 2011
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, December 16, 2011 02:38:09 Andrei Alexandrescu wrote:
 On 12/16/11 1:12 AM, Jonathan M Davis wrote:
 Simply making it so that std.file is only imported in std.stdio with
 version(unittest) cut off _that_ much?
Yah, but the matter is more complex. The issue is that std.file pulls std.datetime, which (a) has static this() code, and (b) pulls core.time, which in turn has static this() code. The issue with that is as follows. Any file that transitively imports a module with constructors will have its own module info generated. When that happens, all vtables in that module will be instantiated, so all methods will be linked in. That in turn causes all functions they call to also be linked in. That's why many programs using std are large. We can attack this in two ways: 1. Revise and reduce all static this() uses in phobos and druntime; 2. Improve the compiler to do minimal linking when static this() does come about.
Well, both std.datetime and core.time need static this() and can't not have it. There may be other places in Phobos where module and class constructors can be avoided or removed, but aside from unit tests, when they're used, they're generally required. If some _can_ be removed though, that would be great, since their presence also risks circular dependencies, which is a far worse issue than the executable's size IMHO. But we can't get rid of them all. Any work that can be done in the compiler to reduce the executable's size due to static this would be great though. - Jonathan M Davis
Dec 16 2011
next sibling parent reply "Adam D. Ruppe" <destructionator gmail.com> writes:
On Friday, 16 December 2011 at 09:50:30 UTC, Jonathan M Davis 
wrote:
 Well, both std.datetime and core.time need static this() and 
 can't not have it.
Why are they necessary? It looks like it sets the time zone... wouldn't it work to put that into DateTime's regular constructor?
Dec 16 2011
parent reply "Adam D. Ruppe" <destructionator gmail.com> writes:
What I have in mind is if the timezone was something along
the lines of a singleton property, so it still works
the same way, except it is lazy loaded on first use.

(if this is indeed the right static constructor!)
Dec 16 2011
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, December 16, 2011 16:16:53 Adam D. Ruppe wrote:
 What I have in mind is if the timezone was something along
 the lines of a singleton property, so it still works
 the same way, except it is lazy loaded on first use.
 
 (if this is indeed the right static constructor!)
That would break purity, so no that doesn't work. The singletons are pure. - Jonathan M Davis
Dec 16 2011
parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Friday, 16 December 2011 at 16:35:27 UTC, Jonathan M Davis 
wrote:
 That would break purity, so no that doesn't work. The 
 singletons are pure.
I'm tempted to say just cast it away, since you aren't actually breaking purity in any meaningful way; the return value is always the same and it should have no other side effects (except on the internal variable). Lying around pure was a bit of a pain... but this seems to have done the trick: alias pure string function () hax; private string impureConstructor() { static string cache; if(cache is null) cache = "lol pure defeated"; return cache; } private pure hax getPureConstructor() { return cast(hax) &impureConstructor; } public system property pure string test() { return getPureConstructor()(); } // test now works
Dec 16 2011
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/16/11 3:49 AM, Jonathan M Davis wrote:
 On Friday, December 16, 2011 02:38:09 Andrei Alexandrescu wrote:
 On 12/16/11 1:12 AM, Jonathan M Davis wrote:
 Simply making it so that std.file is only imported in std.stdio with
 version(unittest) cut off _that_ much?
Yah, but the matter is more complex. The issue is that std.file pulls std.datetime, which (a) has static this() code, and (b) pulls core.time, which in turn has static this() code. The issue with that is as follows. Any file that transitively imports a module with constructors will have its own module info generated. When that happens, all vtables in that module will be instantiated, so all methods will be linked in. That in turn causes all functions they call to also be linked in. That's why many programs using std are large. We can attack this in two ways: 1. Revise and reduce all static this() uses in phobos and druntime; 2. Improve the compiler to do minimal linking when static this() does come about.
Well, both std.datetime and core.time need static this() and can't not have it.
I am pretty sure they don't need static this(). Only last night I removed static this() from core.time. Andrei
Dec 16 2011
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, December 16, 2011 11:45:42 Andrei Alexandrescu wrote:
 On 12/16/11 3:49 AM, Jonathan M Davis wrote:
 On Friday, December 16, 2011 02:38:09 Andrei Alexandrescu wrote:
 On 12/16/11 1:12 AM, Jonathan M Davis wrote:
 Simply making it so that std.file is only imported in std.stdio with
 version(unittest) cut off _that_ much?
Yah, but the matter is more complex. The issue is that std.file pulls std.datetime, which (a) has static this() code, and (b) pulls core.time, which in turn has static this() code. The issue with that is as follows. Any file that transitively imports a module with constructors will have its own module info generated. When that happens, all vtables in that module will be instantiated, so all methods will be linked in. That in turn causes all functions they call to also be linked in. That's why many programs using std are large. We can attack this in two ways: 1. Revise and reduce all static this() uses in phobos and druntime; 2. Improve the compiler to do minimal linking when static this() does come about.
Well, both std.datetime and core.time need static this() and can't not have it.
I am pretty sure they don't need static this(). Only last night I removed static this() from core.time.
I don't know how you could do that in core.time, since ticksPerSec and appOrigin are immutable and have to be set at runtime. How on earth can you do that without a static constructor? std.datetime has the same problem with the added fun of having to avoid breaking purity, because the functions for getting the singletons are pure. - Jonathan M Davis
Dec 16 2011
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/16/11 12:40 PM, Jonathan M Davis wrote:
 On Friday, December 16, 2011 11:45:42 Andrei Alexandrescu wrote:
 I am pretty sure they don't need static this(). Only last night I
 removed static this() from core.time.
I don't know how you could do that in core.time, since ticksPerSec and appOrigin are immutable and have to be set at runtime. How on earth can you do that without a static constructor? std.datetime has the same problem with the added fun of having to avoid breaking purity, because the functions for getting the singletons are pure.
This goes back to the issue of lazy initialization. Today you need a cast to do that. Here's my code: static trusted property long ticksPerSec() pure nothrow { return (cast(immutable(long) function() pure nothrow) &ticksPerSecImpl)(); } static property immutable(long) ticksPerSecImpl() nothrow { static long result; if (result) { return result; } ... initialization ... return result; } The presence of the cast is unsightly but the code does something unusual (modifies what looks from the outside like a constant) so it is justifiable, particularly since we're talking about the language's core library. Andrei
Dec 16 2011
next sibling parent reply Sean Kelly <sean invisibleduck.org> writes:
On Dec 16, 2011, at 11:04 AM, Andrei Alexandrescu wrote:

 On 12/16/11 12:40 PM, Jonathan M Davis wrote:
 On Friday, December 16, 2011 11:45:42 Andrei Alexandrescu wrote:
 I am pretty sure they don't need static this(). Only last night I
 removed static this() from core.time.
=20 I don't know how you could do that in core.time, since ticksPerSec =
and
 appOrigin are immutable and have to be set at runtime. How on earth =
can you do
 that without a static constructor?
=20
 std.datetime has the same problem with the added fun of having to =
avoid
 breaking purity, because the functions for getting the singletons are =
pure.
=20
 This goes back to the issue of lazy initialization. Today you need a =
cast to do that. Here's my code:
=20
    static  trusted  property long ticksPerSec() pure nothrow
    {
        return (cast(immutable(long) function() pure nothrow) =
&ticksPerSecImpl)();
    }
This is fine, but the whole point of static ctors in D is to eliminate = all the stupid workarounds required to use statics in C++. I'd much = rather we find a way to make the use of static ctors more efficient than = give up on the feature.=
Dec 16 2011
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/16/11 1:07 PM, Sean Kelly wrote:
 This is fine, but the whole point of static ctors in D is to
 eliminate all the stupid workarounds required to use statics in C++.
 I'd much rather we find a way to make the use of static ctors more
 efficient than give up on the feature.
I agree, but then I think we have a design that's already there. This discusses working some kinks out of the implementation. Also, the context of the runtime/standard library is an appropriate place to take less usual measures for the benefit of many. Andrei
Dec 16 2011
parent reply Sean Kelly <sean invisibleduck.org> writes:
On Dec 16, 2011, at 12:26 PM, Andrei Alexandrescu wrote:

 On 12/16/11 1:07 PM, Sean Kelly wrote:
 This is fine, but the whole point of static ctors in D is to
 eliminate all the stupid workarounds required to use statics in C++.
 I'd much rather we find a way to make the use of static ctors more
 efficient than give up on the feature.
=20 I agree, but then I think we have a design that's already there. This =
discusses working some kinks out of the implementation. Also, the = context of the runtime/standard library is an appropriate place to take = less usual measures for the benefit of many. But at the same time, the standard library should be an example of how = to do things "the right way." By preferring the C++ approach over = static ctors in the standard library, we're suggesting that static ctors = are not the right approach for the discriminating programmer. I do = agree that the design is already there, but perhaps the implementation = needs refinement?=
Dec 16 2011
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/16/11 4:21 PM, Sean Kelly wrote:
 On Dec 16, 2011, at 12:26 PM, Andrei Alexandrescu wrote:

 On 12/16/11 1:07 PM, Sean Kelly wrote:
 This is fine, but the whole point of static ctors in D is to
 eliminate all the stupid workarounds required to use statics in
 C++. I'd much rather we find a way to make the use of static
 ctors more efficient than give up on the feature.
I agree, but then I think we have a design that's already there. This discusses working some kinks out of the implementation. Also, the context of the runtime/standard library is an appropriate place to take less usual measures for the benefit of many.
But at the same time, the standard library should be an example of how to do things "the right way."
More often, APIs and examples given in the docs are examples of how to do things the right way; the standard library's implementation has a bit of a different charter than most application code, and this is triply true for systems languages. This is emphatically true for e.g. C, C++, and Perl. I also remember I was surprised when I peeked inside a functional language's library implementation. ("That's not how they teach them to write sort!")
 By preferring the C++ approach
 over static ctors in the standard library, we're suggesting that
 static ctors are not the right approach for the discriminating
 programmer.
And they may as well not be, subject to whatever unique constraints to overcome.
 I do agree that the design is already there, but perhaps
 the implementation needs refinement?
That's a given! Andrei
Dec 16 2011
parent Sean Kelly <sean invisibleduck.org> writes:
On Dec 16, 2011, at 2:54 PM, Andrei Alexandrescu wrote:

 On 12/16/11 4:21 PM, Sean Kelly wrote:
 On Dec 16, 2011, at 12:26 PM, Andrei Alexandrescu wrote:
=20
 On 12/16/11 1:07 PM, Sean Kelly wrote:
 This is fine, but the whole point of static ctors in D is to
 eliminate all the stupid workarounds required to use statics in
 C++. I'd much rather we find a way to make the use of static
 ctors more efficient than give up on the feature.
=20 I agree, but then I think we have a design that's already there. This discusses working some kinks out of the implementation. Also, the context of the runtime/standard library is an appropriate place to take less usual measures for the benefit of many.
=20 But at the same time, the standard library should be an example of how to do things "the right way."
=20 More often, APIs and examples given in the docs are examples of how to =
do things the right way; the standard library's implementation has a bit = of a different charter than most application code, and this is triply = true for systems languages. This is emphatically true for e.g. C, C++, = and Perl. I also remember I was surprised when I peeked inside a = functional language's library implementation. ("That's not how they = teach them to write sort!") Perhaps it's just that I come from a systems programming background and = have books like "Large Scale C++ Software Design" sitting on the shelf = next to me. I think the important distinction to be made is between = sample code and real world code. Or perhaps between code where = performance is and is not an issue. You've historically derided the = quicksort example for functional programs as useless because, while it's = a very clean example of the algorithm, it's ridiculously inefficient. = So anyone who really cares about the efficiency of their code is going = to end up writing stuff that looks nothing like what you'd find in a = textbook. In short, they're going to write code that looks like = standard library code to whatever extent the skill of their programmers = can achieve. I really don't want the line between whether or not to use = really useful language features like static ctors to be whether I'm = writing sample code or professional code. That said, I will grant that library code in general can't make any = assumptions about how the code will be used, so this is the one case = where premature optimization really is prudent. Even performance-minded = application code typically can't make the same claim because there is = generally some idea of how that code will be run, and thus tuning can be = done based on profiler data. So I suppose I'll somewhat concede your = point.=
Dec 16 2011
prev sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, December 16, 2011 11:07:14 Sean Kelly wrote:
 On Dec 16, 2011, at 11:04 AM, Andrei Alexandrescu wrote:
 On 12/16/11 12:40 PM, Jonathan M Davis wrote:
 On Friday, December 16, 2011 11:45:42 Andrei Alexandrescu wrote:
 I am pretty sure they don't need static this(). Only last night I
 removed static this() from core.time.
I don't know how you could do that in core.time, since ticksPerSec and appOrigin are immutable and have to be set at runtime. How on earth can you do that without a static constructor? std.datetime has the same problem with the added fun of having to avoid breaking purity, because the functions for getting the singletons are pure.>
This goes back to the issue of lazy initialization. Today you need a cast
to do that. Here's my code:
 static  trusted  property long ticksPerSec() pure nothrow
 {
 
 return (cast(immutable(long) function() pure nothrow)
 &ticksPerSecImpl)();> 
 }
This is fine, but the whole point of static ctors in D is to eliminate all the stupid workarounds required to use statics in C++. I'd much rather we find a way to make the use of static ctors more efficient than give up on the feature.
Agreed. - Jonathan M Davis
Dec 16 2011
prev sibling next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Andrei Alexandrescu:

 Right now an executable starts at around 218KB, which includes druntime 
 (gc, type info, the works). Importing std.stdio and using writeln() only 
 adds a couple of KBs.
Now using ulink the hello world exe becomes 129_564 bytes. Bye, bearophile
Dec 16 2011
parent reply Trass3r <un known.com> writes:
Am 16.12.2011, 10:15 Uhr, schrieb bearophile <bearophileHUGS lycos.com>:

 Andrei Alexandrescu:

 Right now an executable starts at around 218KB, which includes druntime
 (gc, type info, the works). Importing std.stdio and using writeln() only
 adds a couple of KBs.
Now using ulink the hello world exe becomes 129_564 bytes.
What is its secret?
Dec 16 2011
next sibling parent Trass3r <un known.com> writes:
Am 16.12.2011, 14:52 Uhr, schrieb Trass3r <un known.com>:

 Am 16.12.2011, 10:15 Uhr, schrieb bearophile <bearophileHUGS lycos.com>:

 Andrei Alexandrescu:

 Right now an executable starts at around 218KB, which includes druntime
 (gc, type info, the works). Importing std.stdio and using writeln()  
 only
 adds a couple of KBs.
Now using ulink the hello world exe becomes 129_564 bytes.
What is its secret?
Didn't it also compress the exe?
Dec 16 2011
prev sibling parent bearophile <bearophileHUGS lycos.com> writes:
Trass3r:

 Now using ulink the hello world exe becomes 129_564 bytes.
What is its secret?
Linkers use grey magic, as you know. And it doesn't use compression. Bye, bearophile
Dec 16 2011
prev sibling next sibling parent Trass3r <un known.com> writes:
Am 16.12.2011, 04:40 Uhr, schrieb Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org>:
 https://github.com/D-Programming-Language/phobos/commit/b7f42ec925fb1d64564d48ea419e201bfc65ed53
Yeah one could also use the new (function-)local imports. However, this also shows another problem common to C and D: You don't get any warnings if an import is unused. A tool that detects removable import declarations would be awesome. I wish dmd was designed modularly and as a library like Clang...
Dec 16 2011
prev sibling parent Sean Kelly <sean invisibleduck.org> writes:
On Dec 15, 2011, at 7:40 PM, Andrei Alexandrescu wrote:

 On 12/10/11 9:39 AM, Bane wrote:
 Short term and long term suggestions ? Anything we can do ? I heard =
it
 is some problem with linking dead code?
=20
=20
=20
 import std.stdio;
 int main(){
 writefln("Hello Bloat!");
 return 0;
 }
=20
 dmd -release -O hello.d
=20
 On Windows:
 v1.071 =3D 339 Kb
 v2.056 =3D 1017 Kb
=20
 It looks very ugly and might distract some people.
=20 In fact there was a low-hanging fruit, and I'm sure there are some =
more. This diff reduces the size of hello, world (compiled with -O = -release -inline and after strip) from 700KB to 220 KB:
=20
 =
https://github.com/D-Programming-Language/phobos/commit/b7f42ec925fb1d6456= 4d48ea419e201bfc65ed53
=20
 Right now an executable starts at around 218KB, which includes =
druntime (gc, type info, the works). Importing std.stdio and using = writeln() only adds a couple of KBs. So importing std.file adds 800K to a executable?=
Dec 16 2011