www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Remember when make -f posix.mak just worked for dmd from zip?

reply Adam D. Ruppe <destructionator gmail.com> writes:
I do. It was a long, long time ago, but it was a glorious age 
when you could hack on dmd with ease. Just download the zip, cd 
dmd/src/dmd and make. Seconds later, you'd have a new dmd binary.

But then the dark times came.

dmd/globals.d(362): Error: file "VERSION" cannot be found or not 
in a path specified with -J

The compiler barked.

make: *** No rule to make target 
`../generated/linux/release/64/SYSCONFDIR.imp', needed by 
`../generated/linux/release/64/dmd'.  Stop.

The make program howled.

Fine, I'll play your game, the hacker asserted and did create the 
files but the shell hissed

/bin/sh: ../config.sh: No such file or directory
make: *** No rule to make target 
`../res/default_ddoc_theme.ddoc', needed by 
`../generated/linux/release/64/dmd'.  Stop.



It might be boost licensed now, but it has moved further away 
from the most basic principle of free software: that the source 
actually works.
Mar 12 2018
next sibling parent reply Seb <seb wilzba.ch> writes:
On Monday, 12 March 2018 at 12:17:55 UTC, Adam D. Ruppe wrote:
 I do. It was a long, long time ago, but it was a glorious age 
 when you could hack on dmd with ease. Just download the zip, cd 
 dmd/src/dmd and make. Seconds later, you'd have a new dmd 
 binary.

 [...]
Where did you grab the source ball? The zip/tar.gz source balls from GitHub are tested and run on all CIs from scratch.
Mar 12 2018
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 12 March 2018 at 12:24:54 UTC, Seb wrote:
 Where did you grab the source ball?
an extremely unlikely place to download the dlang compiler. https://dlang.org/download.html
Mar 12 2018
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 12 March 2018 at 12:27:42 UTC, Adam D. Ruppe wrote:
 https://dlang.org/download.html
specifically I grab the linux tarballs http://downloads.dlang.org/releases/2.x/2.079.0/dmd.2.079.0.linux.tar.xz and just cd dmd2/src/dmd and make -f posix.mak this always worked perfectly a year or two ago, then it broke when they added the new ddoc theme and now the version is missing too.
Mar 12 2018
parent reply Seb <seb wilzba.ch> writes:
On Monday, 12 March 2018 at 12:30:48 UTC, Adam D. Ruppe wrote:
 On Monday, 12 March 2018 at 12:27:42 UTC, Adam D. Ruppe wrote:
 https://dlang.org/download.html
specifically I grab the linux tarballs http://downloads.dlang.org/releases/2.x/2.079.0/dmd.2.079.0.linux.tar.xz and just cd dmd2/src/dmd and make -f posix.mak this always worked perfectly a year or two ago, then it broke when they added the new ddoc theme and now the version is missing too.
Out of interest: Why would you compile the compiler from the release sources? After all, it's the binary release bundle and not the development build. Also there are tools like digger that automate everything for you... I understand that sometimes it can be nice to do that, but imho you are making a big deal out of this, while it's not really one in practice as you can simply fetch the source from GitHub. Also there is no CI for the release builds and there's only one person who can build them (Martin) as it requires virtual machines for all supported OSes. Martin does an amazing job, but there's only so much one person can do...
Mar 12 2018
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 12 March 2018 at 13:14:08 UTC, Seb wrote:
 Out of interest: Why would you compile the compiler from the 
 release sources?
Why are there releases at all? Because the release is (in theory) tested and working and installed and I want to do a minor tweak to it, not go all-in to the strange land of in-development dmd which may be broken in unknown ways (I've had *terrible* experiences with it before, though I hear you guys have improved it lately).
 After all, it's the binary release bundle and not the 
 development build.
It used to work to just `cd dmd2/src/dmd && make -f posix.mak`. Most the files are still there, implying it is supposed to still work, but it fails because of trivial things like VERSION being in the wrong directory (the file IS actually there, just not where -J looks for it), and the ddoc file not being included (it is apparently in a directory not bundled). This is arguably a regression. And it leaves me with little confidence that you actually fixed the historically-much-MUCH-more-difficult git build when the zip build that historically have just worked is now broken....
Mar 12 2018
parent Seb <seb wilzba.ch> writes:
On Monday, 12 March 2018 at 13:54:08 UTC, Adam D. Ruppe wrote:
 It used to work to just `cd dmd2/src/dmd && make -f posix.mak`. 
 Most the files are still there, implying it is supposed to 
 still work, but it fails because of trivial things like VERSION 
 being in the wrong directory (the file IS actually there, just 
 not where -J looks for it), and the ddoc file not being 
 included (it is apparently in a directory not bundled).


 This is arguably a regression
Yes, it's because VERSION is now auto-generated.
 And it leaves me with little confidence that you actually fixed 
 the historically-much-MUCH-more-difficult git build
Well, as you noticed the layout in release archives is a bit different to the original git source.
  when the zip build that historically have just worked is now 
 broken....
I'm super-confident in this. We have too many CIs that all `git clone` the dmd repo on _every_ PR. ``` git clone https://github.com/dlang/dmd cd dmd make -f posix.mak ``` FWIW at the moment we have six different CIs for dmd which are all passing nicely on master: https://circleci.com/gh/dlang/dmd/tree/master https://semaphoreci.com/dlang/dmd-2 https://ci.appveyor.com/project/greenify/dmd http://ci.dlang.io http://dtest.dlang.io https://auto-tester.puremagic.com/platform-history.ghtml?projectid=1&os=Linux_64_64
Mar 12 2018
prev sibling parent ketmar <ketmar ketmar.no-ip.org> writes:
Seb wrote:

 Out of interest: Why would you compile the compiler from the release 
 sources? After all, it's the binary release bundle and not the 
 development build. Also there are tools like digger that automate 
 everything for you...

 I understand that sometimes it can be nice to do that, but imho you are 
 making a big deal out of this, while it's not really one in practice as 
 you can simply fetch the source from GitHub.
 Also there is no CI for the release builds and there's only one person 
 who can build them (Martin) as it requires virtual machines for all 
 supported OSes. Martin does an amazing job, but there's only so much one 
 person can do...
every sentence in this answer is perfect. and the whole thing perfectly illustrates the reasons for my hardfork.
Mar 12 2018
prev sibling parent reply Daniel Kozak <kozzi11 gmail.com> writes:
obviosly it is unlikely place to download the dlang compiler sources these
days :D

On Mon, Mar 12, 2018 at 1:27 PM, Adam D. Ruppe via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Monday, 12 March 2018 at 12:24:54 UTC, Seb wrote:

 Where did you grab the source ball?
an extremely unlikely place to download the dlang compiler. https://dlang.org/download.html
Mar 12 2018
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 12 March 2018 at 12:50:14 UTC, Daniel Kozak wrote:
 obviosly it is unlikely place to download the dlang compiler 
 sources these days :D
See, I just want to do a small tweak on the installation I have working. Since druntime, phobos, and dmd are so ridiculously tightly coupled, you need to have matching releases of all three (and apparently now of the compiler used to build the compiler... if i try to just build dmd from git i get `/home/me/d/dmd2/linux/bin32/../../src/druntime/import/core/exception.d(686): _store is thread local`) so you can't just change one little piece and keep the rest intact anymore. But in the old days, when you downloaded the zip, the included source just worked. You could go in and run make in-place and it'd successfully build the replacement piece you can use with the rest of the distribution. I'd just like that to work again. And it shouldn't be hard - i think the src directory of the zip is just missing a few basic files.
Mar 12 2018
next sibling parent Daniel Kozak <kozzi11 gmail.com> writes:
Yes, I understand that need and I hope it will be fixed with next (minor)
release

On Mon, Mar 12, 2018 at 1:55 PM, Adam D. Ruppe via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Monday, 12 March 2018 at 12:50:14 UTC, Daniel Kozak wrote:

 obviosly it is unlikely place to download the dlang compiler sources
 these days :D
See, I just want to do a small tweak on the installation I have working. Since druntime, phobos, and dmd are so ridiculously tightly coupled, you need to have matching releases of all three (and apparently now of the compiler used to build the compiler... if i try to just build dmd from git i get `/home/me/d/dmd2/linux/bin32/../../src/druntime/import/core/exception.d(686): _store is thread local`) so you can't just change one little piece and keep the rest intact anymore. But in the old days, when you downloaded the zip, the included source just worked. You could go in and run make in-place and it'd successfully build the replacement piece you can use with the rest of the distribution. I'd just like that to work again. And it shouldn't be hard - i think the src directory of the zip is just missing a few basic files.
Mar 12 2018
prev sibling next sibling parent reply Seb <seb wilzba.ch> writes:
On Monday, 12 March 2018 at 12:55:41 UTC, Adam D. Ruppe wrote:
 On Monday, 12 March 2018 at 12:50:14 UTC, Daniel Kozak wrote:
 obviosly it is unlikely place to download the dlang compiler 
 sources these days :D
See, I just want to do a small tweak on the installation I have working. Since druntime, phobos, and dmd are so ridiculously tightly coupled, you need to have matching releases of all three (and apparently now of the compiler used to build the compiler... if i try to just build dmd from git i get `/home/me/d/dmd2/linux/bin32/../../src/druntime/import/core/exception.d(686): _store is thread local`) so you can't just change one little piece and keep the rest intact anymore.
That's not an error, but just a warning and been there for quite a while as everyone just ignores it. Not sure how this is related to not being able to build from git. BTW since a few months you can directly build everything by just running `make -f posix.mak` in the Phobos repository.
Mar 12 2018
next sibling parent Seb <seb wilzba.ch> writes:
On Monday, 12 March 2018 at 13:21:10 UTC, Seb wrote:
 On Monday, 12 March 2018 at 12:55:41 UTC, Adam D. Ruppe wrote:
 On Monday, 12 March 2018 at 12:50:14 UTC, Daniel Kozak wrote:
 [...]
See, I just want to do a small tweak on the installation I have working. Since druntime, phobos, and dmd are so ridiculously tightly coupled, you need to have matching releases of all three (and apparently now of the compiler used to build the compiler... if i try to just build dmd from git i get `/home/me/d/dmd2/linux/bin32/../../src/druntime/import/core/exception.d(686): _store is thread local`) so you can't just change one little piece and keep the rest intact anymore.
That's not an error, but just a warning and been there for quite a while as everyone just ignores it.
No more -> https://github.com/dlang/druntime/pull/2138
Mar 12 2018
prev sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 12 March 2018 at 13:21:10 UTC, Seb wrote:
 BTW since a few months you can directly build everything by 
 just running `make -f posix.mak` in the Phobos repository.
see i'm trying to build dmd not phobos so you can understand i wouldn't try to make phobos It actually concerns me that the phobos repo knows about the dmd repo. I really think dmd/druntime/phobos are so tightly coupled it doesn't even make sense to have them as separate repos, since you can't really do anything in one without the other two being in the same state. But, I do seem to have a working dmd out of it soooo maybe I'll poke at the bug after all. But my free morning hour is now elapsed so I should prolly get to work on the day job too.
Mar 12 2018
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Mar 12, 2018 at 02:15:23PM +0000, Adam D. Ruppe via Digitalmars-d wrote:
[...]
 It actually concerns me that the phobos repo knows about the dmd repo.
 I really think dmd/druntime/phobos are so tightly coupled it doesn't
 even make sense to have them as separate repos, since you can't really
 do anything in one without the other two being in the same state.
[...] This has been a long-standing problem. It has been this way for historical reasons, and so far nobody has been able to come up with a sane way of fixing the problem. There have been diverse proposals in the past but nobody has been able to push it through. Personally, I don't even dare to look into it, because it opens up a gigantic can of worms about repo integration, build system preferences, gmake vs. vanilla make, and lots of politics about not fixing what's not broken, etc.. Not to mention coordinating such a change with existing devs who will inevitably be unhappy that the current way they work will no longer work, and all the CIs and other automatic builds that have come to depend on the way things are currently done. I don't have the kind of patience (nor time!) it takes to push through with that kind of change. T -- The trouble with TCP jokes is that it's like hearing the same joke over and over.
Mar 12 2018
prev sibling parent ketmar <ketmar ketmar.no-ip.org> writes:
Adam D. Ruppe wrote:

 compiler used to build the compiler... if i try to just build dmd from 
 git i get 
 `/home/me/d/dmd2/linux/bin32/../../src/druntime/import/core/exception.d(686): 
 _store is thread local`)
it is just a -vtls message, btw, absolutely harmless. i don't know why that reporting is turned on.
Mar 12 2018
prev sibling next sibling parent Daniel Kozak <kozzi11 gmail.com> writes:
wierd, still working for me, and it seems working for archlinux too:
https://git.archlinux.org/svntogit/community.git/plain/trunk/PKGBUILD?h=packages/dmd

On Mon, Mar 12, 2018 at 1:17 PM, Adam D. Ruppe via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 I do. It was a long, long time ago, but it was a glorious age when you
 could hack on dmd with ease. Just download the zip, cd dmd/src/dmd and
 make. Seconds later, you'd have a new dmd binary.

 But then the dark times came.

 dmd/globals.d(362): Error: file "VERSION" cannot be found or not in a path
 specified with -J

 The compiler barked.

 make: *** No rule to make target
`../generated/linux/release/64/SYSCONFDIR.imp',
 needed by `../generated/linux/release/64/dmd'.  Stop.

 The make program howled.

 Fine, I'll play your game, the hacker asserted and did create the files
 but the shell hissed

 /bin/sh: ../config.sh: No such file or directory
 make: *** No rule to make target `../res/default_ddoc_theme.ddoc', needed
 by `../generated/linux/release/64/dmd'.  Stop.



 It might be boost licensed now, but it has moved further away from the
 most basic principle of free software: that the source actually works.
Mar 12 2018
prev sibling parent Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Monday, 12 March 2018 at 12:17:55 UTC, Adam D. Ruppe wrote:
 I do. It was a long, long time ago, but it was a glorious age 
 when you could hack on dmd with ease. Just download the zip, cd 
 dmd/src/dmd and make. Seconds later, you'd have a new dmd 
 binary.

 But then the dark times came.
All of these are bugs in the distribution / packaging / build process, and should be fixed. Shipping unbuildable source code loses half the point of shipping source code at all.
Mar 12 2018