www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Documentation generation

reply bearophile <bearophileHUGS lycos.com> writes:
The purpose of a static language compiler is to help the programmer spot the
bugs in the code very well and to compile it. It's not the job of a compiler to
create nice HTML/XML documentation files.

So I suggest to:
1) DMD can copy all documentation strings inside the JSON file generated with
the -X option. DMD has just to recognize the /** */ and /// inside the D code
and put only such comments inside the JSON (the JSON has to contain all the
information necessary to create the HTML pages).
2) Move the code that understands DDoc syntax and generates the HTML docs
inside an external utility written in D or Python, and keep such utility inside
the zip of the standard DMD distribution. This program can ask DMD to generate
the JSON files, can read them and generate the HTML pages. 
3) Deprecate and then remove the -D option of DMD2.

Removing that HTML ddoc parsing & HTML generation from DMD allows such code to
evolve faster and to be debugged more efficiently. Today some people are using 

[Mostly unrelated: once xfbuild is well debugged it can be added inside the zip
of the standard DMD distribution.]

Bye,
bearophile
Aug 01 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
bearophile wrote:
 Removing that HTML ddoc parsing & HTML generation from DMD allows such code
 to evolve faster and to be debugged more efficiently. Today some people are
 using

There are very good reasons why ddoc is part of dmd. 1. Being a defined part of D means it's ALWAYS there. That means there won't be D compilers without ddoc. That means that people can rely on it being there, and use it with confidence. 2. Please do not underestimate the power of something reliably being there. In my experience, it makes it FAR more likely that people will actually use it, and there's the big win. If it's a separate utility, I guarantee you that usage of ddoc will drop probably by 90%. 3. A separate ddoc utility means that, inevitably, dmd and ddoc will get out of sync with each other. People will update dmd and not ddoc, and vice versa. There is quite enough grief with people mixing one version of dmd with the wrong version of phobos and then being mystified by the results. I can't tell you how many times I've helped someone who INSISTED they were using the correct libphobos.a and there were no others on their machine, only to eventually sheepishly discover another old libphobos.a on their path that was being used. 4. Making it a separate utility means others will invent "improved" ddocs, with the resulting balkanization and non-portability of documentation comments. See C++ for the affect this has. 5. The documentation of Phobos took a HUGE leap forward with the integration of ddoc. 6. Similar advantages can be found with the integration of unittesting in D, and the integration of the preprocessor with C/C++ (I'm old enough to remember when that was a separate program). 7. Contrary to what you state, making ddoc a separate utility will make it far harder to maintain, not easier. Just outputting JSON files is not enough, as ddoc is also able to pretty-print the declarations and do color syntax highlighting of coding examples. If you look at the dmd source code changes, you'll find that ddoc tracks changes in D's grammar and syntax with rarely any changes at all in the ddoc code.
Aug 01 2010
next sibling parent bearophile <bearophileHUGS lycos.com> writes:
Walter Bright:
 There are very good reasons why ddoc is part of dmd.

Thank you for your many answers. I think you can even add them to the D2 FAQ :-) Bye, bearophile
Aug 01 2010
prev sibling next sibling parent Brad Roberts <braddr puremagic.com> writes:
On 8/1/2010 1:53 PM, bearophile wrote:
 
 I think you can...
 
 Bye,
 bearophile

I've been avoiding saying this for a while, but I suspect a lot of people have thought it at one time or another. You probably ought to find a word other than 'can' to use. Of course he's capable of... whatever. Perhaps you meant to make it a suggestion?
Aug 01 2010
prev sibling next sibling parent Lurker <lurk lurkers.org> writes:
Walter Bright Wrote:

 bearophile wrote:
 Removing that HTML ddoc parsing & HTML generation from DMD allows such code
 to evolve faster and to be debugged more efficiently. Today some people are
 using

There are very good reasons why ddoc is part of dmd. 1. Being a defined part of D means it's ALWAYS there. That means there won't be D compilers without ddoc. That means that people can rely on it being there, and use it with confidence. 2. Please do not underestimate the power of something reliably being there. In my experience, it makes it FAR more likely that people will actually use it, and there's the big win. If it's a separate utility, I guarantee you that usage of ddoc will drop probably by 90%.

That's just silly. People using 15 different good document generators is much better than one s**y one with s**y templates you need to dig from old newsgroup posts.
 
 3. A separate ddoc utility means that, inevitably, dmd and ddoc will get out
of 
 sync with each other. People will update dmd and not ddoc, and vice versa.
There 
 is quite enough grief with people mixing one version of dmd with the wrong 
 version of phobos and then being mystified by the results. I can't tell you
how 
 many times I've helped someone who INSISTED they were using the correct 
 libphobos.a and there were no others on their machine, only to eventually 
 sheepishly discover another old libphobos.a on their path that was being used.

His idea was probably to separate the tools and still provide them both as part of the standard distribution. It would be your duty to keep both up to date.
 
 4. Making it a separate utility means others will invent "improved" ddocs,
with 
 the resulting balkanization and non-portability of documentation comments. See 
 C++ for the affect this has.

D already has several document generators because ddoc is way too primitive. Luckily the main competition (Tango) died because of an organized FUD attack.
 
 5. The documentation of Phobos took a HUGE leap forward with the integration
of 
 ddoc.

The output is still a huge joke compared to tools like doxygen and javadoc. No offense. And those are rather basic document generation tools -- better ones exist.
 
 6. Similar advantages can be found with the integration of unittesting in D,
and 
 the integration of the preprocessor with C/C++ (I'm old enough to remember
when 
 that was a separate program).

The integrated unit test is a joke. Spend 30 minutes with Java 6 someday and try JUnit. Or maybe few hours. The more you use, the more you learn. If you open your eyes, you might actually see how crippled your vision of unit testing is. The integrated unittest stuff brings almost zero value when doing serious software development.
 
 7. Contrary to what you state, making ddoc a separate utility will make it far 
 harder to maintain, not easier.

One of the main reasons why 3rd party tool support sucks is that YOU have zero ability communicating technical details to the community. We can see it in documentation. Almost none exists. People make lots of questions here and instead of answering you always focus on bikeshedding.
 Just outputting JSON files is not enough, as 
 ddoc is also able to pretty-print the declarations and do color syntax 
 highlighting of coding examples. If you look at the dmd source code changes, 
 you'll find that ddoc tracks changes in D's grammar and syntax with rarely any 
 changes at all in the ddoc code.

That's not a surprise. If a docgen tool and a compiler frontend coming from a single vendor didn't share any code, I wouldn't call it software engineering. It would be a horrible toy project written by an amateur.
Aug 01 2010
prev sibling next sibling parent reply Lutger <lutger.blijdestijn gmail.com> writes:
Walter Bright wrote:

 bearophile wrote:
 Removing that HTML ddoc parsing & HTML generation from DMD allows such code
 to evolve faster and to be debugged more efficiently. Today some people are
 using

There are very good reasons why ddoc is part of dmd. 1. Being a defined part of D means it's ALWAYS there. That means there won't be D compilers without ddoc. That means that people can rely on it being there, and use it with confidence.

I agree, though the standard output is not so pleasant. When the new website design is adopted, would you please consider shipping a .ddoc plus the css needed in a similar style for user documentation? No doubt people are willing to help with this, I certainly would.
Aug 01 2010
parent Walter Bright <newshound2 digitalmars.com> writes:
Lutger wrote:
 I agree, though the standard output is not so pleasant. When the new website 
 design is adopted, would you please consider shipping a .ddoc plus the css 
 needed in a similar style for user documentation? No doubt people are willing
to 
 help with this, I certainly would.

Sure.
Aug 02 2010
prev sibling next sibling parent reply Bernard Helyer <b.helyer gmail.com> writes:
On Sun, 01 Aug 2010 13:22:53 -0700, Walter Bright wrote:
 1. Being a defined part of D means it's ALWAYS there. That means there
 won't be D compilers without ddoc.

/me waves. I'm writing a D compiler, and have zero plans to add DDoc support. I couldn't do it well enough for me to waste my time on it.
Aug 06 2010
next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Bernard Helyer:
 I'm writing a D compiler, and have zero plans to add DDoc support. I 
 couldn't do it well enough for me to waste my time on it.  

Walter has said that putting ddoc inside the compiler helps keep ddoc uniform, so people don't invent/use something different. In theory this is a nice purpose, but in practice I have seen lot of people use other documentation means for D, Doxygen. Walter seems to ignore how much often such other systems are used instead of ddoc. Bye, bearophile
Aug 07 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
bearophile wrote:
 Walter has said that putting ddoc inside the compiler helps keep ddoc
 uniform, so people don't invent/use something different. In theory this is a
 nice purpose, but in practice I have seen lot of people use other
 documentation means for D, Doxygen. Walter seems to ignore how much often
 such other systems are used instead of ddoc.

It's fine if people choose to use other documentation systems. Ddoc, though, sets a minimum standard, and it's always there, requires no additional installs, is always synced to the compiler, is always on every platform D is, etc. My experience with languages that do not have built in doc abilities is, by and large, only a tiny minority of programmers use a doc system. Having to research, download, install, read the manual, address incompatibilities, etc., takes effort and the reality is that by making such effort close to zero it greatly encourages people to use it. The same goes for unit tests. For another example, back in the 80's, it was commonplace for people to dis the C text preprocessor as "not a real macro system". Those who wanted a better one were advised to "just use m4". How many people using C or C++ today use m4 as the preprocessor? Zilch. The C preprocessor was good enough to get the job done, it was always there, always ready, and that was that. (Note that I'm not defending the preprocessor, I'm just illustrating the power of having something built in as opposed to being a separate tool.)
Aug 07 2010
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 08/07/2010 02:53 PM, Walter Bright wrote:
 bearophile wrote:
 Walter has said that putting ddoc inside the compiler helps keep ddoc
 uniform, so people don't invent/use something different. In theory
 this is a
 nice purpose, but in practice I have seen lot of people use other
 documentation means for D, Doxygen. Walter seems to ignore how much often
 such other systems are used instead of ddoc.

It's fine if people choose to use other documentation systems. Ddoc, though, sets a minimum standard, and it's always there, requires no additional installs, is always synced to the compiler, is always on every platform D is, etc. My experience with languages that do not have built in doc abilities is, by and large, only a tiny minority of programmers use a doc system. Having to research, download, install, read the manual, address incompatibilities, etc., takes effort and the reality is that by making such effort close to zero it greatly encourages people to use it. The same goes for unit tests. For another example, back in the 80's, it was commonplace for people to dis the C text preprocessor as "not a real macro system". Those who wanted a better one were advised to "just use m4". How many people using C or C++ today use m4 as the preprocessor? Zilch. The C preprocessor was good enough to get the job done, it was always there, always ready, and that was that. (Note that I'm not defending the preprocessor, I'm just illustrating the power of having something built in as opposed to being a separate tool.)

Boost's preprocessor library is a powerful exhibit in favor of your argument. I think I wouldn't be mistaken to say that everything it does can be done much easier and much better, plus with m4 you can do a lot other things of interest. But the Boost preprocessor library wins because "it's there". Andrei
Aug 07 2010
prev sibling next sibling parent div0 <div0 sourceforge.net> writes:
On 08/08/2010 01:32, Bernard Helyer wrote:
 Nope. But it doesn't matter. It doesn't affect the code built or
 generated, and that's my focus -- at least for the foreseeable future.

 I'm writing a compiler, not a half-arsed documentation generator (because
 if I wrote the DDoc stuff, that's exactly what it would be).

Sounds cool. What are you writing it in and is it going to be public? -- My enormous talent is exceeded only by my outrageous laziness. http://www.ssTk.co.uk
Aug 07 2010
prev sibling parent dsimcha <dsimcha yahoo.com> writes:
== Quote from Bernard Helyer (b.helyer gmail.com)'s article
 I'm writing a compiler, not a half-arsed documentation generator (because
 if I wrote the DDoc stuff, that's exactly what it would be).

I'll concur with Walter. I think that sometimes power users of a given tool/feature fail to realize that very often simple, good enough and works out of the box beats well-designed, feature-packed, efficient, but a PITA to set up and use, overkill for simple cases, and not available out of the box. I understand treating ddoc as a low priority feature, but **please** support it eventually, as the whole point of it is that it's just there on any implementation (nothing separate to download/configure) and just works for the simplest 80-90% of cases even if its deficiencies become obvious in the last 10-20%. In general I also think that DDoc's evolution is actually good design strategy. I wish more tools were designed by pinning down how the simplest 80-90% of cases should work first, treating that as a constraint and then figuring out how to make the most complicated 10-20% work within the constraint. Designing with "everything must be possible" as the first goal encourages overengineering such that users with simple needs end up reinventing the wheel because it's the path of least resistance. For example, I'd probably never use Ddoc if I had to use XML for it.
Aug 07 2010
prev sibling next sibling parent Jonathan M Davis <jmdavisprog gmail.com> writes:
On Friday 06 August 2010 20:06:59 Bernard Helyer wrote:
 On Sun, 01 Aug 2010 13:22:53 -0700, Walter Bright wrote:
 1. Being a defined part of D means it's ALWAYS there. That means there
 won't be D compilers without ddoc.

/me waves. I'm writing a D compiler, and have zero plans to add DDoc support. I couldn't do it well enough for me to waste my time on it.

Well, that's your choice, but then you haven't followed the D spec in that regard, so your compiler wouldn't be properly standard D. It is the sort of thing that I would expect to be somewhat lower in priority than other features though. - Jonathan M Davis
Aug 07 2010
prev sibling next sibling parent Bernard Helyer <b.helyer gmail.com> writes:
On Sat, 07 Aug 2010 01:55:35 -0700, Jonathan M Davis wrote:

 On Friday 06 August 2010 20:06:59 Bernard Helyer wrote:
 On Sun, 01 Aug 2010 13:22:53 -0700, Walter Bright wrote:
 1. Being a defined part of D means it's ALWAYS there. That means
 there won't be D compilers without ddoc.

/me waves. I'm writing a D compiler, and have zero plans to add DDoc support. I couldn't do it well enough for me to waste my time on it.

Well, that's your choice, but then you haven't followed the D spec in that regard, so your compiler wouldn't be properly standard D.

Nope. But it doesn't matter. It doesn't affect the code built or generated, and that's my focus -- at least for the foreseeable future. I'm writing a compiler, not a half-arsed documentation generator (because if I wrote the DDoc stuff, that's exactly what it would be).
Aug 07 2010
prev sibling next sibling parent Bernard Helyer <b.helyer gmail.com> writes:
On Sun, 08 Aug 2010 02:02:28 +0100, div0 wrote:

 On 08/08/2010 01:32, Bernard Helyer wrote:
 Nope. But it doesn't matter. It doesn't affect the code built or
 generated, and that's my focus -- at least for the foreseeable future.

 I'm writing a compiler, not a half-arsed documentation generator
 (because if I wrote the DDoc stuff, that's exactly what it would be).

Sounds cool. What are you writing it in and is it going to be public?

D. It's already public[1], but there's not a lot to show yet. You can compile `int main() { return 42; }`, but that's about it, so far. :P [1]:http://github.com/bhelyer/SDC
Aug 07 2010
prev sibling next sibling parent Bernard Helyer <b.helyer gmail.com> writes:
On Sun, 08 Aug 2010 02:22:30 +0000, dsimcha wrote:

 == Quote from Bernard Helyer (b.helyer gmail.com)'s article
 I'm writing a compiler, not a half-arsed documentation generator
 (because if I wrote the DDoc stuff, that's exactly what it would be).

but **please** support it eventually,

Well, sure, once basically *everything* else is done, I'll probably poke at it. But it is a very low priority.
Aug 07 2010
prev sibling parent retard <re tard.com.invalid> writes:
Sun, 08 Aug 2010 02:22:30 +0000, dsimcha wrote:

 == Quote from Bernard Helyer (b.helyer gmail.com)'s article
 I'm writing a compiler, not a half-arsed documentation generator
 (because if I wrote the DDoc stuff, that's exactly what it would be).

I'll concur with Walter. I think that sometimes power users of a given tool/feature fail to realize that very often simple, good enough and works out of the box beats well-designed, feature-packed, efficient, but a PITA to set up and use, overkill for simple cases, and not available out of the box. I understand treating ddoc as a low priority feature, but **please** support it eventually, as the whole point of it is that it's just there on any implementation (nothing separate to download/configure) and just works for the simplest 80-90% of cases even if its deficiencies become obvious in the last 10-20%. In general I also think that DDoc's evolution is actually good design strategy. I wish more tools were designed by pinning down how the simplest 80-90% of cases should work first, treating that as a constraint and then figuring out how to make the most complicated 10-20% work within the constraint. Designing with "everything must be possible" as the first goal encourages overengineering such that users with simple needs end up reinventing the wheel because it's the path of least resistance. For example, I'd probably never use Ddoc if I had to use XML for it.

The problem with ddoc is that even though 1000+ documentation formats already existed, he wanted to reinvent his own. The ddoc syntax is terrible ad-hoc crap and geared towards html (3.2/4). Has anyone seriously tried to generate any non-html documents with ddoc which look as good as better than those created with e.g. doxygen?! I've personally just filtered out the bad constructs from the sources and rendered with the buggy doxygen. Much better. I get nice pdf/tex/man/xml output.
Aug 14 2010