www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - hello world executable size

reply "Joakim" <dlang joakim.fea.st> writes:
I was curious if binary sizes had decreased because of the 
changes Ilya had been making to try and scope imports better and 
make them more selective:

https://github.com/D-Programming-Language/phobos/pulls?utf8=%E2%9C%93&q=is%3Apr+author%3A9il+clean

Hello world (void main(){ import std.stdio; writefln("hello 
roboto"); }) size went from 464 KB to 548 KB when going from 
2.066.1 to 2.067.1 on linux/x86, an increase of 18% (dmd -O 
-release main.d).  I used nm to try and find some of the symbols 
using the most space (command taken from SO):

nm -B -r --size-sort --print-size -t d main

I noticed that the symbol taking up the third-most space was 
_d_arraysetlengthT, which wasn't in the older executable 
generated by 2.066.1.  Disassembling the newer executable 
(objdump -rD main), it appears that it's called from exactly one 
function, std.uni.GcPolicy.realloc, which is in turn only called 
from one templated struct's member function, 
std.uni.CowArray.length.  That instantiated function isn't called 
from anywhere else in the binary.

The templated struct std.uni.CowArray is only instantiated by the 
templated struct std.uni.InversionList in the source, but I'm not 
sure why neither is instantiated in the older executable and a 
diff of the two versions of std.stdio doesn't produce anything 
that stands out.  None of this appears to be used when the binary 
is run, as having gdb break on _d_arraysetlengthT does nothing.

But std.uni isn't actually imported directly by std.stdio, where 
does it come from?  Nearest I can tell from adding the -v flag to 
dmd, std.stdio has a couple scoped, selective imports to some 
functions from std.utf.  std.utf has exactly _one_ scoped, 
selective import of std.string.format in its UTFException class, 
and std.string has several selective imports from std.uni, 
including one at module scope.

I tried commenting out that single selective import of 
std.string.format in std.utf and the same binary compiled and ran 
fine without any imports of std.string or std.uni, plus it was 
now 36 KB smaller. :)

I realize executable size may not be a priority, but this 
exploration shows how easy it is to get a bunch of template 
garbage pulled in to executables (I know this is not news for 
some).  Perhaps the binary would have been twice as big if not 
for Ilya's work!  Maybe this isn't considered something that 
should be fixed at the compiler level, but rather by properly 
working with the linker to remove these, as David did with 
--gc-sections for ldc.  Either way, some kind of dashboard that 
charts binary sizes for dmd PRs can't come soon enough, so we can 
keep better tabs on this.
Jun 25 2015
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Thursday, 25 June 2015 at 11:05:00 UTC, Joakim wrote:
 I was curious if binary sizes had decreased because of the 
 changes Ilya had been making to try and scope imports better 
 and make them more selective:
http://digger.k3.1azy.net/trend/
 I used nm to try and find some of the symbols using the most 
 space (command taken from SO):
http://thecybershadow.net/d/mapview/
Jun 25 2015
next sibling parent reply "Joakim" <dlang joakim.fea.st> writes:
On Thursday, 25 June 2015 at 11:07:11 UTC, Vladimir Panteleev 
wrote:
 On Thursday, 25 June 2015 at 11:05:00 UTC, Joakim wrote:
 I was curious if binary sizes had decreased because of the 
 changes Ilya had been making to try and scope imports better 
 and make them more selective:
http://digger.k3.1azy.net/trend/
Took 90 MiB of JSON to see it, but finally got it, funny how executable size swings wildly up to five times larger over the years. :) Anyway, I saw that viewer when you announced it before: any plans to add it to the github PR checks, along with your recent check for documentation info?
 I used nm to try and find some of the symbols using the most 
 space (command taken from SO):
http://thecybershadow.net/d/mapview/
Does it show sizes somewhere? Here's the dependency list for my binary: http://thecybershadow.net/d/mapview/data/558be3dfde121.html It also shows that the relevant setter calling GcPolicy.realloc isn't used anywhere: http://thecybershadow.net/d/mapview/data/558be3dfde121.html#_D3std3uni32__T8CowArrayTS3std3uni8GcPolicyZ8CowArray6lengthMFNaNbNdNekZv
Jun 25 2015
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Thursday, 25 June 2015 at 11:59:24 UTC, Joakim wrote:
 On Thursday, 25 June 2015 at 11:07:11 UTC, Vladimir Panteleev 
 wrote:
 On Thursday, 25 June 2015 at 11:05:00 UTC, Joakim wrote:
 I was curious if binary sizes had decreased because of the 
 changes Ilya had been making to try and scope imports better 
 and make them more selective:
http://digger.k3.1azy.net/trend/
Took 90 MiB of JSON to see it, but finally got it, funny how executable size swings wildly up to five times larger over the years. :) Anyway, I saw that viewer when you announced it before: any plans to add it to the github PR checks, along with your recent check for documentation info?
Would you believe me if I said that this obvious (in retrospect) idea hasn't crossed my mind yet?
 Does it show sizes somewhere?  Here's the dependency list for 
 my binary:

 http://thecybershadow.net/d/mapview/data/558be3dfde121.html
Try the treemap form (above the dependency explorer form).
Jun 25 2015
parent reply "Joakim" <dlang joakim.fea.st> writes:
On Thursday, 25 June 2015 at 12:04:26 UTC, Vladimir Panteleev 
wrote:
 On Thursday, 25 June 2015 at 11:59:24 UTC, Joakim wrote:
 Took 90 MiB of JSON to see it, but finally got it, funny how 
 executable size swings wildly up to five times larger over the 
 years. :) Anyway, I saw that viewer when you announced it 
 before: any plans to add it to the github PR checks, along 
 with your recent check for documentation info?
Would you believe me if I said that this obvious (in retrospect) idea hasn't crossed my mind yet?
I figured that's where you were going when you announced it. :)
 Does it show sizes somewhere?  Here's the dependency list for 
 my binary:

 http://thecybershadow.net/d/mapview/data/558be3dfde121.html
Try the treemap form (above the dependency explorer form).
I didn't know how to generate the .map file, I now see it's mentioned on the wiki: http://wiki.dlang.org/Development_tools#File_size_profiling Looks nice: http://thecybershadow.net/d/mapview/view.php?id=558bef76234eb Surprising that core.* and gc.* are almost as large as std.*, but I guess "hello world" isn't going to exercise that much of phobos. :)
Jun 25 2015
parent reply "Joakim" <dlang joakim.fea.st> writes:
On Thursday, 25 June 2015 at 12:15:26 UTC, Joakim wrote:
 On Thursday, 25 June 2015 at 12:04:26 UTC, Vladimir Panteleev 
 wrote:
 On Thursday, 25 June 2015 at 11:59:24 UTC, Joakim wrote:
 Took 90 MiB of JSON to see it, but finally got it, funny how 
 executable size swings wildly up to five times larger over 
 the years. :) Anyway, I saw that viewer when you announced it 
 before: any plans to add it to the github PR checks, along 
 with your recent check for documentation info?
Would you believe me if I said that this obvious (in retrospect) idea hasn't crossed my mind yet?
I figured that's where you were going when you announced it. :)
Another check that would be more worthwhile but harder to measure would be speed of compilation of druntime/phobos, especially since speed of compilation is considered a key selling point of D. Harder to measure because it depends on what else is going on on that machine, but with some care and enough samples, you could get something representative.
Jun 28 2015
parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Sunday, 28 June 2015 at 09:27:56 UTC, Joakim wrote:
 Another check that would be more worthwhile but harder to 
 measure would be speed of compilation of druntime/phobos, 
 especially since speed of compilation is considered a key 
 selling point of D.  Harder to measure because it depends on 
 what else is going on on that machine, but with some care and 
 enough samples, you could get something representative.
Compilation/linking time are measured for the sample programs.
Jun 28 2015
prev sibling parent reply "rsw0x" <anonymous anonymous.com> writes:
On Thursday, 25 June 2015 at 11:07:11 UTC, Vladimir Panteleev 
wrote:
 On Thursday, 25 June 2015 at 11:05:00 UTC, Joakim wrote:
 I was curious if binary sizes had decreased because of the 
 changes Ilya had been making to try and scope imports better 
 and make them more selective:
http://digger.k3.1azy.net/trend/
looks like this commit more than doubled the size of hello world https://github.com/D-Programming-Language/phobos/pull/3443
Jun 28 2015
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Sunday, 28 June 2015 at 09:46:45 UTC, rsw0x wrote:
 On Thursday, 25 June 2015 at 11:07:11 UTC, Vladimir Panteleev 
 wrote:
 On Thursday, 25 June 2015 at 11:05:00 UTC, Joakim wrote:
 I was curious if binary sizes had decreased because of the 
 changes Ilya had been making to try and scope imports better 
 and make them more selective:
http://digger.k3.1azy.net/trend/
looks like this commit more than doubled the size of hello world https://github.com/D-Programming-Language/phobos/pull/3443
Woah. Why would removing an import increase the filesize?
Jun 28 2015
parent reply "Joakim" <dlang joakim.fea.st> writes:
On Sunday, 28 June 2015 at 09:55:53 UTC, Vladimir Panteleev wrote:
 On Sunday, 28 June 2015 at 09:46:45 UTC, rsw0x wrote:
 looks like this commit more than doubled the size of hello 
 world

 https://github.com/D-Programming-Language/phobos/pull/3443
Woah. Why would removing an import increase the filesize?
I didn't get that either, maybe he meant the PR that yours fixed is the one that doubled it? On Sunday, 28 June 2015 at 09:58:35 UTC, Vladimir Panteleev wrote:
 On Sunday, 28 June 2015 at 09:27:56 UTC, Joakim wrote:
 Another check that would be more worthwhile but harder to 
 measure would be speed of compilation of druntime/phobos, 
 especially since speed of compilation is considered a key 
 selling point of D.  Harder to measure because it depends on 
 what else is going on on that machine, but with some care and 
 enough samples, you could get something representative.
Compilation/linking time are measured for the sample programs.
Yeah, I saw that, but I was talking about adding a github check for D PRs and how they affect compilation speed, especially for dmd PRs. Druntime/Phobos and eventually ddmd may not be the best way to check it, but it's the closest lamppost. ;) Smaller binary size is nice to have, but not that important, especially since we've been neglecting it for some time now. Compilation speed is something we're always trumpeting, we better track it.
Jun 28 2015
next sibling parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Sunday, 28 June 2015 at 10:06:20 UTC, Joakim wrote:
 On Sunday, 28 June 2015 at 09:55:53 UTC, Vladimir Panteleev 
 wrote:
 On Sunday, 28 June 2015 at 09:46:45 UTC, rsw0x wrote:
 looks like this commit more than doubled the size of hello 
 world

 https://github.com/D-Programming-Language/phobos/pull/3443
Woah. Why would removing an import increase the filesize?
I didn't get that either, maybe he meant the PR that yours fixed is the one that doubled it?
No, he's right. Removing the import doubled the filesize of a helloworld binary.
 On Sunday, 28 June 2015 at 09:58:35 UTC, Vladimir Panteleev 
 wrote:
 On Sunday, 28 June 2015 at 09:27:56 UTC, Joakim wrote:
 Another check that would be more worthwhile but harder to 
 measure would be speed of compilation of druntime/phobos, 
 especially since speed of compilation is considered a key 
 selling point of D.  Harder to measure because it depends on 
 what else is going on on that machine, but with some care and 
 enough samples, you could get something representative.
Compilation/linking time are measured for the sample programs.
Yeah, I saw that, but I was talking about adding a github check for D PRs and how they affect compilation speed, especially for dmd PRs. Druntime/Phobos and eventually ddmd may not be the best way to check it, but it's the closest lamppost. ;) Smaller binary size is nice to have, but not that important, especially since we've been neglecting it for some time now. Compilation speed is something we're always trumpeting, we better track it.
It's not really possible to meaningfully track such an inaccurate statistic on a per-commit basis. See it yourself - select one of the time tests in AWSY and zoom in. It works in aggregate - when zoomed out, you see the medians and can get the general big picture. But when comparing any two commits directly, there is just too much error.
Jun 28 2015
parent reply "Joakim" <dlang joakim.fea.st> writes:
On Sunday, 28 June 2015 at 10:11:08 UTC, Vladimir Panteleev wrote:
 No, he's right. Removing the import doubled the filesize of a 
 helloworld binary.
Ah, I didn't want to download the full 90 MBs graph data again to see it. Yes, I see it now.
 On Sunday, 28 June 2015 at 09:58:35 UTC, Vladimir Panteleev 
 wrote:
It's not really possible to meaningfully track such an inaccurate statistic on a per-commit basis. See it yourself - select one of the time tests in AWSY and zoom in. It works in aggregate - when zoomed out, you see the medians and can get the general big picture. But when comparing any two commits directly, there is just too much error.
Seems pretty stable to me, almost as much as file size even, which is surprising. I did note that you'd have to be careful to measure it on a relatively unloaded machine and average multiple runs, but I don't see why it couldn't be done. There is some variability on some of those, but as long as you didn't overreact on small changes and maybe compared one PR's results to averaged past data, ie over multiple PRs, as the baseline, it should work.
Jun 28 2015
parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Sunday, 28 June 2015 at 10:37:15 UTC, Joakim wrote:
 On Sunday, 28 June 2015 at 10:11:08 UTC, Vladimir Panteleev 
 wrote:
 No, he's right. Removing the import doubled the filesize of a 
 helloworld binary.
Ah, I didn't want to download the full 90 MBs graph data again to see it. Yes, I see it now.
It's only about 5 MB compressed. Some browsers show the decompressed size.
Jun 28 2015
prev sibling parent "rsw0x" <anonymous anonymous.com> writes:
On Sunday, 28 June 2015 at 10:06:20 UTC, Joakim wrote:
 On Sunday, 28 June 2015 at 09:55:53 UTC, Vladimir Panteleev 
 wrote:
 On Sunday, 28 June 2015 at 09:46:45 UTC, rsw0x wrote:
 looks like this commit more than doubled the size of hello 
 world

 https://github.com/D-Programming-Language/phobos/pull/3443
Woah. Why would removing an import increase the filesize?
I didn't get that either, maybe he meant the PR that yours fixed is the one that doubled it?
it's the PR that's linked when I zoomed in on executable size in 'Hello World'. It's not visible at first(I guess because the PR is so new?,) you have to zoom in once or twice. I guess a picture is worth a thousand words. http://i.imgur.com/p0r5tFH.png
Jun 28 2015