www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Building the docs with own changes

reply "John Colvin" <john.loughran.colvin gmail.com> writes:
Is there any non-insane way of changing some documentation in a 
module and checking that any links/xrefs you may have added 
actually work?

The dlang.org makefile appears to happily erase uncommitted 
changes in ../druntime and ../phobos. I'm really struggling for 
words to describe how infuriating that is. I ran make and it 
deleted my work in a separate folder. Not OK.
Jun 08 2015
next sibling parent reply "anonymous" <anonymous example.com> writes:
On Monday, 8 June 2015 at 11:27:58 UTC, John Colvin wrote:
 Is there any non-insane way of changing some documentation in a 
 module and checking that any links/xrefs you may have added 
 actually work?

 The dlang.org makefile appears to happily erase uncommitted 
 changes in ../druntime and ../phobos. I'm really struggling for 
 words to describe how infuriating that is. I ran make and it 
 deleted my work in a separate folder. Not OK.
This shouldn't happen. Are you sure you worked in the "phobos" directory and not in some versioned one like "phobos-2.067.1"? That would be a directory created by dlang.org. Slightly different topic, related to changing documentation without losing your sanity: When building dlang.org, changes in druntime/phobos are not properly picked up. You can try deleting a strategic file like dlang.org/web/phobos-prerelease/index.html which should trigger a rebuild. See posix.mak for what files are dependencies to the different targets. Or just do a fresh build every time: `make -f posix.mak clean; make -f posix.mak phobos-prerelease`. I have an open pull request that's supposed to fix this: https://github.com/D-Programming-Language/dlang.org/pull/993
Jun 08 2015
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/8/15 6:33 AM, anonymous wrote:
 Slightly different topic, related to changing documentation without
 losing your sanity: When building dlang.org, changes in druntime/phobos
 are not properly picked up.

 You can try deleting a strategic file like
 dlang.org/web/phobos-prerelease/index.html which should trigger a
 rebuild. See posix.mak for what files are dependencies to the different
 targets.

 Or just do a fresh build every time: `make -f posix.mak clean; make -f
 posix.mak phobos-prerelease`.

 I have an open pull request that's supposed to fix this:
 https://github.com/D-Programming-Language/dlang.org/pull/993
Yah, that's annoying. Thanks for contributing a fix. For now my typical approach is: rm -f web/phobos-prerelease/index.html && make -j6 LATEST=2.067.1 phobos-prerelease Andrei
Jun 08 2015
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/8/15 7:27 AM, John Colvin wrote:
 Is there any non-insane way of changing some documentation in a module
 and checking that any links/xrefs you may have added actually work?

 The dlang.org makefile appears to happily erase uncommitted changes in
 ../druntime and ../phobos. I'm really struggling for words to describe
 how infuriating that is. I ran make and it deleted my work in a separate
 folder. Not OK.
If you just want to check that a file's content is going to show up, I use: dmd -D -o- -c std/somemod.d The output is not styled properly. But the output should have working links. FWIW, I avoid building phobos docs. It's too painful. If I change dlang.org docs, I will build those, but not phobos. -Steve
Jun 08 2015
next sibling parent reply "Meta" <jared771 gmail.com> writes:
On Monday, 8 June 2015 at 13:34:14 UTC, Steven Schveighoffer 
wrote:
 If you just want to check that a file's content is going to 
 show up, I use:

 dmd -D -o- -c std/somemod.d

 The output is not styled properly. But the output should have 
 working links.

 FWIW, I avoid building phobos docs. It's too painful. If I 
 change dlang.org docs, I will build those, but not phobos.

 -Steve
What's painful about building phobos docs? It only takes one command.
Jun 08 2015
next sibling parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/8/15 9:36 AM, Meta wrote:
 On Monday, 8 June 2015 at 13:34:14 UTC, Steven Schveighoffer wrote:
 If you just want to check that a file's content is going to show up, I
 use:

 dmd -D -o- -c std/somemod.d

 The output is not styled properly. But the output should have working
 links.

 FWIW, I avoid building phobos docs. It's too painful. If I change
 dlang.org docs, I will build those, but not phobos.
What's painful about building phobos docs? It only takes one command.
Yes, after you've set up an appropriate directory structure. And that's not easy. I've done it before, a long time ago, so maybe it's gotten better. Clearly, John can tell you more ;) -Steve
Jun 08 2015
prev sibling parent reply "Atila Neves" <atila.neves gmail.com> writes:
On Monday, 8 June 2015 at 13:36:08 UTC, Meta wrote:
 On Monday, 8 June 2015 at 13:34:14 UTC, Steven Schveighoffer 
 wrote:
 If you just want to check that a file's content is going to 
 show up, I use:

 dmd -D -o- -c std/somemod.d

 The output is not styled properly. But the output should have 
 working links.

 FWIW, I avoid building phobos docs. It's too painful. If I 
 change dlang.org docs, I will build those, but not phobos.

 -Steve
What's painful about building phobos docs? It only takes one command.
What's _not_ painful? Atila
Jun 08 2015
next sibling parent "Meta" <jared771 gmail.com> writes:
On Monday, 8 June 2015 at 15:07:11 UTC, Atila Neves wrote:
 What's _not_ painful?

 Atila
It's painful to get set up, but once you do it's smooth sailing. It took one small edit to the makefile to allow me to build the website and the docs on Windows.
Jun 08 2015
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/8/15 8:07 AM, Atila Neves wrote:
 On Monday, 8 June 2015 at 13:36:08 UTC, Meta wrote:
 On Monday, 8 June 2015 at 13:34:14 UTC, Steven Schveighoffer wrote:
 If you just want to check that a file's content is going to show up,
 I use:

 dmd -D -o- -c std/somemod.d

 The output is not styled properly. But the output should have working
 links.

 FWIW, I avoid building phobos docs. It's too painful. If I change
 dlang.org docs, I will build those, but not phobos.

 -Steve
What's painful about building phobos docs? It only takes one command.
What's _not_ painful?
I build the docs probably a few dozen times every day. I just press up arrow to fetch the previous command, and enter. Takes no perceptible time. What's the problem? -- Andrei
Jun 08 2015
parent reply "Joseph Rushton Wakeling" <joseph.wakeling webdrake.net> writes:
On Monday, 8 June 2015 at 16:27:49 UTC, Andrei Alexandrescu wrote:
 I build the docs probably a few dozen times every day. I just 
 press up arrow to fetch the previous command, and enter. Takes 
 no perceptible time. What's the problem? -- Andrei
Well, just for one, I tried building phobos docs a couple of days ago while disconnected from the internet; the whole thing failed because the clones of stable release repos couldn't be done. No doubt I'm missing a trick, but it didn't seem very obvious, on browsing the makefile, what would get me the simple case of building the development phobos docs using my development dmd, without needing to worry about any of the other options out there.
Jun 08 2015
next sibling parent reply "Andrei Alexandrescu" <SeeWebsiteForEmail erdani.org> writes:
On Monday, 8 June 2015 at 18:06:47 UTC, Joseph Rushton Wakeling 
wrote:
 On Monday, 8 June 2015 at 16:27:49 UTC, Andrei Alexandrescu 
 wrote:
 I build the docs probably a few dozen times every day. I just 
 press up arrow to fetch the previous command, and enter. Takes 
 no perceptible time. What's the problem? -- Andrei
Well, just for one, I tried building phobos docs a couple of days ago while disconnected from the internet; the whole thing failed because the clones of stable release repos couldn't be done. No doubt I'm missing a trick, but it didn't seem very obvious, on browsing the makefile, what would get me the simple case of building the development phobos docs using my development dmd, without needing to worry about any of the other options out there.
Yah, you'd need to pass LATEST=2.067.1 in the command line. Then the build process won't need to connect to the internet to fetch the latest version. Could you please make a pull request with the explanations you wish you had? Andrei
Jun 08 2015
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Monday, 8 June 2015 at 19:51:28 UTC, Andrei Alexandrescu wrote:
 Yah, you'd need to pass LATEST=2.067.1 in the command line. 
 Then the build process won't need to connect to the internet to 
 fetch the latest version.
"LATEST" detection must go. It makes bisecting impossible. In general, if we want reproducible builds (and this is required for reliable bisection of regressions), we can't rely on non-immutable online resources. Fetching / building stable compiler versions is OK, but detecting which version is latest right now is not.
Jun 08 2015
next sibling parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Monday, 8 June 2015 at 21:53:07 UTC, Vladimir Panteleev wrote:
 On Monday, 8 June 2015 at 19:51:28 UTC, Andrei Alexandrescu 
 wrote:
 Yah, you'd need to pass LATEST=2.067.1 in the command line. 
 Then the build process won't need to connect to the internet 
 to fetch the latest version.
"LATEST" detection must go. It makes bisecting impossible.
Actually, I can work around this fairly easily with Digger by calculating the correct LATEST value at the time and passing it as a make parameter :)
Jun 08 2015
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/8/15 3:05 PM, Vladimir Panteleev wrote:
 On Monday, 8 June 2015 at 21:53:07 UTC, Vladimir Panteleev wrote:
 On Monday, 8 June 2015 at 19:51:28 UTC, Andrei Alexandrescu wrote:
 Yah, you'd need to pass LATEST=2.067.1 in the command line. Then the
 build process won't need to connect to the internet to fetch the
 latest version.
"LATEST" detection must go. It makes bisecting impossible.
Actually, I can work around this fairly easily with Digger by calculating the correct LATEST value at the time and passing it as a make parameter :)
... and there's the well-engineered improvement right there. Bravo. -- Andrei
Jun 08 2015
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Monday, 8 June 2015 at 22:26:23 UTC, Andrei Alexandrescu wrote:
 ... and there's the well-engineered improvement right there. 
 Bravo. -- Andrei
The network activity for the autodetection is still a minor nuisance.
Jun 08 2015
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/8/15 3:29 PM, Vladimir Panteleev wrote:
 On Monday, 8 June 2015 at 22:26:23 UTC, Andrei Alexandrescu wrote:
 ... and there's the well-engineered improvement right there. Bravo. --
 Andrei
The network activity for the autodetection is still a minor nuisance.
It can be set in the command line, something we should clarify. Actually let me work on a PR real quick. -- Andrei
Jun 08 2015
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/8/15 3:31 PM, Andrei Alexandrescu wrote:
 On 6/8/15 3:29 PM, Vladimir Panteleev wrote:
 On Monday, 8 June 2015 at 22:26:23 UTC, Andrei Alexandrescu wrote:
 ... and there's the well-engineered improvement right there. Bravo. --
 Andrei
The network activity for the autodetection is still a minor nuisance.
It can be set in the command line, something we should clarify. Actually let me work on a PR real quick. -- Andrei
Destroy: https://github.com/D-Programming-Language/dlang.org/pull/1012 Andrei
Jun 08 2015
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/8/15 2:53 PM, Vladimir Panteleev wrote:
 On Monday, 8 June 2015 at 19:51:28 UTC, Andrei Alexandrescu wrote:
 Yah, you'd need to pass LATEST=2.067.1 in the command line. Then the
 build process won't need to connect to the internet to fetch the
 latest version.
"LATEST" detection must go. It makes bisecting impossible. In general, if we want reproducible builds (and this is required for reliable bisection of regressions), we can't rely on non-immutable online resources. Fetching / building stable compiler versions is OK, but detecting which version is latest right now is not.
Well I trust you can come up with a well-engineered improvement. -- Andrei
Jun 08 2015
parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Monday, 8 June 2015 at 22:25:41 UTC, Andrei Alexandrescu wrote:
 Well I trust you can come up with a well-engineered 
 improvement. -- Andrei
1. We can just put it in the makefile (or elsewhere in the repository). New releases need website updates anyway (front page, changelog...). Currently, the front page hard-codes the current version number - it could use LATEST instead. 2. DMD has a "VERSION" file. It could also have a "STABLE_VERSION" file (which is equal to the "VERSION" file only for release builds).
Jun 08 2015
prev sibling next sibling parent Joseph Rushton Wakeling via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 08/06/15 19:06, Joseph Rushton Wakeling via Digitalmars-d wrote:
 Well, just for one, I tried building phobos docs a couple of days ago while
 disconnected from the internet; the whole thing failed because the clones of
 stable release repos couldn't be done.

 No doubt I'm missing a trick, but it didn't seem very obvious, on browsing the
 makefile, what would get me the simple case of building the development phobos
 docs using my development dmd, without needing to worry about any of the other
 options out there.
Well, whaddayaknow, make -f posix.mak clean; make -f posix.mak phobos-prerelease just worked. I'd swear I was trying to build the phobos-prerelease target before, but maybe forgetting the 'clean' first had some kind of weird effect on how build targets interacted.
Jun 08 2015
prev sibling parent reply Joseph Rushton Wakeling via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 08/06/15 21:53, Joseph Rushton Wakeling via Digitalmars-d wrote:
 On 08/06/15 19:06, Joseph Rushton Wakeling via Digitalmars-d wrote:
 Well, just for one, I tried building phobos docs a couple of days ago while
 disconnected from the internet; the whole thing failed because the clones of
 stable release repos couldn't be done.

 No doubt I'm missing a trick, but it didn't seem very obvious, on browsing the
 makefile, what would get me the simple case of building the development phobos
 docs using my development dmd, without needing to worry about any of the other
 options out there.
Well, whaddayaknow, make -f posix.mak clean; make -f posix.mak phobos-prerelease just worked. I'd swear I was trying to build the phobos-prerelease target before, but maybe forgetting the 'clean' first had some kind of weird effect on how build targets interacted.
Ho hum, spoken too soon. I don't know what was going on, but I fear I was using some weird checkout of dlang.org at the time I wrote the above. After upgrading to latest master, I couldn't even issue 'make -f posix.mak clean' without generating a network-connectivity-related error: ----------------- $ make -f posix.mak clean fatal: unable to access 'https://github.com/D-Programming-Language/dmd/': Could not resolve host: github.com LATEST= <-- place in the command line to skip network traffic. posix.mak:69: *** Could not fetch latest version, place LATEST=2.xxx.y in the command line. Stop. ----------------- Adding LATEST=2.067.1 to the command indeed allows clean to run, but then -- joy! -- I can no longer rebuild anything, because the cached DMD 2.067 is deleted and I need internet connectivity to get it back. It's also a major "WTF??!!" that 'make clean' should need network connectivity in any circumstances. The bottom line of this for me is that I have dmd, druntime, phobos and tools repos all in the "standard" locations relative to my dlang.org repo; all of those are built (or I can build them). It's therefore really baffling to me that I should need more than that in order to be able to issue 'make phobos-prerelease' and see what my updated docs look like. Force me to add LATEST=dev or something like that to the command if necessary, but _don't_ force me to either download or already have cached a separate copy of the compiler. I cannot speak to how infuriating it was to be on a plane with hours to spare to work on phobos material, only to find myself flying blind unable to validate my documentation work, because the doc build system assumes network connectivity and has no obvious fallback or workaround. Sorry if the above sounds abrasive, but take it as a measure of how much I was enjoying working on phobos again :-)
Jun 13 2015
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/13/15 1:10 AM, Joseph Rushton Wakeling via Digitalmars-d wrote:
 Adding LATEST=2.067.1 to the command indeed allows clean to run, but
  then -- joy! -- I can no longer rebuild anything, because the cached
 DMD 2.067 is deleted and I need internet connectivity to get it
 back.
I don't think make clean deletes dmd2.067.1. It does instruct you to do so if you so want.
 It's also a major "WTF??!!" that 'make clean' should need network
 connectivity in any circumstances.
Yah, supporting 100% disconnected builds was a corner case that's not properly supported yet. It's just difficult in make to set a variable efficiently only for some targets.
 The bottom line of this for me is that I have dmd, druntime, phobos
 and tools repos all in the "standard" locations relative to my
 dlang.org repo; all of those are built (or I can build them).  It's
 therefore really baffling to me that I should need more than that in
 order to be able to issue 'make phobos-prerelease' and see what my
 updated docs look like.  Force me to add LATEST=dev or something like
 that to the command if necessary, but _don't_ force me to either
 download or already have cached a separate copy of the compiler.

 I cannot speak to how infuriating it was to be on a plane with hours
 to spare to work on phobos material, only to find myself flying blind
 unable to validate my documentation work, because the doc build
 system assumes network connectivity and has no obvious fallback or
 workaround.

 Sorry if the above sounds abrasive, but take it as a measure of how
 much I was enjoying working on phobos again :)
There's no surprise that things we didn't focus much effort don't work well. That said, I'm not sure what's happening there - things should have worked; phobos-prerelease doesn't require the legacy stuff. Should be reported as a bug. Perhaps it's dub that insists on a specific version. Could you please try "make -f posix.mak html" while disconnected and see if that goes through? Andrei
Jun 13 2015
parent Joseph Rushton Wakeling via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 13/06/15 17:30, Andrei Alexandrescu via Digitalmars-d wrote:
 On 6/13/15 1:10 AM, Joseph Rushton Wakeling via Digitalmars-d wrote:
 Adding LATEST=2.067.1 to the command indeed allows clean to run, but
  then -- joy! -- I can no longer rebuild anything, because the cached
 DMD 2.067 is deleted and I need internet connectivity to get it
 back.
I don't think make clean deletes dmd2.067.1. It does instruct you to do so if you so want.
Just for clarity, I'm talking about the auto-downloaded executable, not the git clones of the stable version to ../dmd-2.067 etc. In other words right after the make command is issued, this gets run: $ make -j2 -f posix.mak LATEST=2.067.1 phobos-prerelease mkdir -p .generated/ echo "LATEST=2.067.1" >.generated/2.067.1.ddoc mkdir -p /tmp/.stable_dmd-2.067.1 TMPFILE=$(mktemp deleteme.XXXXXXXX) && curl -fsSL http://downloads.dlang.org/releases/2.x/2.067.1/dmd.2.067.1.linux.zip > ${TMPFILE}.zip && \ unzip -qd /tmp/.stable_dmd-2.067.1 ${TMPFILE}.zip && rm ${TMPFILE}.zip Anyway, I believe you're right that this temporary download doesn't get cleaned up by "make clean" but I believe it can get lost through cleanup of the /tmp dir without any user intent. I'm just trying a general "make -f posix.mak" build (which will recreate the clones of stable dmd etc.) and will see if this allows avoiding the curl download of prebuilt dmd.
 There's no surprise that things we didn't focus much effort don't work well.
 That said, I'm not sure what's happening there - things should have worked;
 phobos-prerelease doesn't require the legacy stuff. Should be reported as a
bug.
 Perhaps it's dub that insists on a specific version. Could you please try "make
 -f posix.mak html" while disconnected and see if that goes through?
I'll give it a go. Will report back on results.
Jun 15 2015
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-06-08 15:34, Steven Schveighoffer wrote:

 If you just want to check that a file's content is going to show up, I use:

 dmd -D -o- -c std/somemod.d

 The output is not styled properly. But the output should have working
 links.
What about the custom ddoc macros? -- /Jacob Carlborg
Jun 08 2015
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/8/15 10:07 AM, Jacob Carlborg wrote:
 On 2015-06-08 15:34, Steven Schveighoffer wrote:

 If you just want to check that a file's content is going to show up, I
 use:

 dmd -D -o- -c std/somemod.d

 The output is not styled properly. But the output should have working
 links.
What about the custom ddoc macros?
Hm... I guess you could include them. I don't even know how to do that. If the docs don't turn out the way I expect, but I think it would with a full build, I usually just stick a warning on my PR that I didn't test it :P -Steve
Jun 08 2015
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/8/15 4:27 AM, John Colvin wrote:
 Is there any non-insane way of changing some documentation in a module
 and checking that any links/xrefs you may have added actually work?

 The dlang.org makefile appears to happily erase uncommitted changes in
 ../druntime and ../phobos. I'm really struggling for words to describe
 how infuriating that is. I ran make and it deleted my work in a separate
 folder. Not OK.
Bummer about that. I've never encountered this problem. What command(s) did you run? -- Andrei
Jun 08 2015
parent reply "John Colvin" <john.loughran.colvin gmail.com> writes:
On Monday, 8 June 2015 at 16:14:30 UTC, Andrei Alexandrescu wrote:
 On 6/8/15 4:27 AM, John Colvin wrote:
 Is there any non-insane way of changing some documentation in 
 a module
 and checking that any links/xrefs you may have added actually 
 work?

 The dlang.org makefile appears to happily erase uncommitted 
 changes in
 ../druntime and ../phobos. I'm really struggling for words to 
 describe
 how infuriating that is. I ran make and it deleted my work in 
 a separate
 folder. Not OK.
Bummer about that. I've never encountered this problem. What command(s) did you run? -- Andrei
with dir structure . |- dmd |- druntime |- phobos |- dlang.org clone phobos, edit clone druntime, edit clone dmd, compile compile druntime, compile phobos clone dlang.org, make -f posix.mak boom, my changes are gone
Jun 08 2015
parent "Andrei Alexandrescu" <SeeWebsiteForEmail erdani.org> writes:
On Monday, 8 June 2015 at 16:48:31 UTC, John Colvin wrote:
 with dir structure
 .
 |- dmd
 |- druntime
 |- phobos
 |- dlang.org

 clone phobos, edit
 clone druntime, edit
 clone dmd, compile
 compile druntime,
 compile phobos
 clone dlang.org, make -f posix.mak

 boom, my changes are gone
That shouldn't happen. If you get a repro, I'll look into that. -- Andrei
Jun 08 2015