www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Re: Showing unittest in documentation (Was Re: std.unittests

reply foobar <foo bar.com> writes:
Steven Schveighoffer Wrote:

 On Mon, 24 Jan 2011 15:20:13 -0500, Andrei Alexandrescu  
 <SeeWebsiteForEmail erdani.org> wrote:
 
 On 1/24/11 2:15 PM, Andrei Alexandrescu wrote:
 On 1/24/11 1:50 PM, Jens Mueller wrote:
 Jonathan M Davis wrote:
 I think that it's been discussed a time or two, but nothing has been
 done about
 it. It wouldn't be entirely straightforward to do. Essentially,  
 either a
 unittest block would have to be generated from the Examples section
 in the
 documentation, or you'd have to have some way to indicate that a
 particular
 unittest block got put into the documentation as an Examples section.
 It's
 certainly true that it would be ideal to have a way to avoid the
 duplication,
 but we don't have one at the moment, and it hasn't yet been a high
 enough
 priority to sort out how to do it and implement it.

I see. I understand that it does not have high priority. Just wondered whether ... Jens

The change is much simpler than what Jonathan suggests. A change can be made such that any unittest preceded by a documentation comment is automatically considered an example. /** Example: */ unittest { writeln("This is how it works."); } Andrei

BTW I consider this a very important topic. We have _plenty_ of examples that don't work and are not mechanically verifiable. The reasons range from minor typos to language changes to implementation limitations. Generally this is what they call "documentation rot". This is terrible PR for the language. Changing ddoc to recognize documentation unittests would fix this matter once and forever. Last but not least, the "----" separators for code samples are awful because no editor recognizes them for anything - they confuse the hell out of Emacs for one thing.

This only makes sense if: 1. The unit test immediately follows the item being documented 2. The unit test *only* tests that item. The second one could be pretty annoying. Consider cases where several functions interact (I've seen this many times on Microsoft's Documentation), and it makes sense to make one example that covers all of them. Having them 'testable' means creating several identical unit tests. One way to easily fix this is to allow an additional parameter to the comment: /** Example(Foo.foo(int), Foo.bar(int)): */ unittest { auto foo = new Foo; foo.foo(5); foo.bar(6); assert(foo.toString() == "bazunga!"); } The above means, copy the example to both Foo.foo(int) and Foo.bar(int) An alternative that is more verbose, but probably more understandable: /** Example: Covers Foo.foo(int) Covers Foo.bar(int) */ Of course, a lack of target just means it applies to the item just documented. One other thing, using writefln is considered bad form in unit tests (you want *no* output if the unit test works). But many examples might want to demonstrate how e.g. an object interacts with writefln. Any suggestions? The assert line above is not very pretty for example... -Steve

Unit-tests are defined on a module level, not a function level, hence I would expect to see the unit-tests that serve as examples to appear in an examples section on the page the documents the module itself. In the online docs I would expect the function names used in the example to be links to the individual function doc and for each function have a list of links to examples it participated in. This should be automatic and the user shouldn't need to provide the "list of functions documented in this example". Just my two cents..
Jan 24 2011
next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday 24 January 2011 15:27:08 foobar wrote:
 Steven Schveighoffer Wrote:
 On Mon, 24 Jan 2011 15:20:13 -0500, Andrei Alexandrescu
 
 <SeeWebsiteForEmail erdani.org> wrote:
 On 1/24/11 2:15 PM, Andrei Alexandrescu wrote:
 On 1/24/11 1:50 PM, Jens Mueller wrote:
 Jonathan M Davis wrote:
 I think that it's been discussed a time or two, but nothing has been
 done about
 it. It wouldn't be entirely straightforward to do. Essentially,
 either a
 unittest block would have to be generated from the Examples section
 in the
 documentation, or you'd have to have some way to indicate that a
 particular
 unittest block got put into the documentation as an Examples
 section. It's
 certainly true that it would be ideal to have a way to avoid the
 duplication,
 but we don't have one at the moment, and it hasn't yet been a high
 enough
 priority to sort out how to do it and implement it.

I see. I understand that it does not have high priority. Just wondered whether ... Jens

The change is much simpler than what Jonathan suggests. A change can be made such that any unittest preceded by a documentation comment is automatically considered an example. /** Example: */ unittest { writeln("This is how it works."); } Andrei

BTW I consider this a very important topic. We have _plenty_ of examples that don't work and are not mechanically verifiable. The reasons range from minor typos to language changes to implementation limitations. Generally this is what they call "documentation rot". This is terrible PR for the language. Changing ddoc to recognize documentation unittests would fix this matter once and forever. Last but not least, the "----" separators for code samples are awful because no editor recognizes them for anything - they confuse the hell out of Emacs for one thing.

This only makes sense if: 1. The unit test immediately follows the item being documented 2. The unit test *only* tests that item. The second one could be pretty annoying. Consider cases where several functions interact (I've seen this many times on Microsoft's Documentation), and it makes sense to make one example that covers all of them. Having them 'testable' means creating several identical unit tests. One way to easily fix this is to allow an additional parameter to the comment: /** Example(Foo.foo(int), Foo.bar(int)): */ unittest { auto foo = new Foo; foo.foo(5); foo.bar(6); assert(foo.toString() == "bazunga!"); } The above means, copy the example to both Foo.foo(int) and Foo.bar(int) An alternative that is more verbose, but probably more understandable: /** Example: Covers Foo.foo(int) Covers Foo.bar(int) */ Of course, a lack of target just means it applies to the item just documented. One other thing, using writefln is considered bad form in unit tests (you want *no* output if the unit test works). But many examples might want to demonstrate how e.g. an object interacts with writefln. Any suggestions? The assert line above is not very pretty for example... -Steve

Unit-tests are defined on a module level, not a function level, hence I would expect to see the unit-tests that serve as examples to appear in an examples section on the page the documents the module itself. In the online docs I would expect the function names used in the example to be links to the individual function doc and for each function have a list of links to examples it participated in. This should be automatic and the user shouldn't need to provide the "list of functions documented in this example". Just my two cents..

Examples almost always go with functions. The whole point is to avoid having to have the function examples both in the documentation and in a unittest block. It would be a huge mess to try and put all of the examples in the module documentation. I shudder to think what that would look like for std.algorithm - or even worse (_far_ worse), std.datetime. The simple syntax of /++ Example +/ unittest { } making that unittest block go in the documentation of the preceding function should work just fine. We already have /++ Ditto +/ which puts a function in with the preceding function's documentation. So, having /++ Example +/ on top of that isn't much of a stretch. - Jonathan M Davis
Jan 24 2011
next sibling parent foobar <foo bar.com> writes:
Jonathan M Davis Wrote:

 On Monday 24 January 2011 15:27:08 foobar wrote:

 
 Unit-tests are defined on a module level, not a function level, hence I
 would expect to see the unit-tests that serve as examples to appear in an
 examples section on the page the documents the module itself. In the
 online docs I would expect the function names used in the example to be
 links to the individual function doc and for each function have a list of
 links to examples it participated in. This should be automatic and the
 user shouldn't need to provide the "list of functions documented in this
 example".
 
 Just my two cents..

Examples almost always go with functions. The whole point is to avoid having to have the function examples both in the documentation and in a unittest block. It would be a huge mess to try and put all of the examples in the module documentation. I shudder to think what that would look like for std.algorithm - or even worse (_far_ worse), std.datetime. The simple syntax of /++ Example +/ unittest { } making that unittest block go in the documentation of the preceding function should work just fine. We already have /++ Ditto +/ which puts a function in with the preceding function's documentation. So, having /++ Example +/ on top of that isn't much of a stretch. - Jonathan M Davis

Depends on what you mean by example. It seems to me that by "Example" you mean something like a line or two that just show how to call a function. I meant by "Example" a more complicated small program that exercises several functions in the module. something more like a tutorial for using the module. For my kind of example you need to provide just one or two examples for a module, it makes more sense to put it at the module level and also in a unit-test block. Your kind of example looks less useful to me and I personally wouldn't bother with unit-testing it. I guess it's just personal style and preferences.
Jan 25 2011
prev sibling parent spir <denis.spir gmail.com> writes:
On 01/25/2011 11:10 AM, foobar wrote:
 The simple syntax of
  /++ Example +/
  unittest
  {
  }


  making that unittest block go in the documentation of the preceding function
  should work just fine. We already have /++ Ditto +/ which puts a function in
with
  the preceding function's documentation. So, having /++ Example +/ on top of
that
  isn't much of a stretch.

  - Jonathan M Davis


It seems to me that by "Example" you mean something like a line or two that just show how to call a function. I meant by "Example" a more complicated small program that exercises several functions in the module. something more like a tutorial for using the module. For my kind of example you need to provide just one or two examples for a module, it makes more sense to put it at the module level and also in a unit-test block. Your kind of example looks less useful to me and I personally wouldn't bother with unit-testing it. I guess it's just personal style and preferences.

Seems doc meta-languages are usually too complicated and this complication never helps meeting practicle needs. Think at Walter' repeted argument that unittests are highly used precisely because they are that simple. The same applies to doc, imo. I would thus favor an extremely simple doc feature, along with use recommandations, and stricter guidelines for public/std packages. About the current issue of examples applying to several elements of a module, a simple solution would be to have an example section at a standard location (eg at the bottom, or just below the general module doc). Each example there would be /named/, so that function docs could point to them. A refinement would make example names link targets and pointers to them link pointers. In the same vein: named unittests would allow pointing to them, and possibly generate link targets in the html doc, under their own name. /** blah * blah blah blah * blah blah blah * * Example: * ... * See also: * example $(target MapFilterCombination) * unittest $(target testFilter) */ T[] filter (T) (T[] elements, bool function (T) predicate) { ... } Denis -- _________________ vita es estrany spir.wikidot.com
Jan 25 2011
prev sibling next sibling parent Jens Mueller <jens.k.mueller gmx.de> writes:
Jonathan M Davis wrote:
 On Monday 24 January 2011 15:27:08 foobar wrote:
 Steven Schveighoffer Wrote:
 
 This only makes sense if:
 
 1. The unit test immediately follows the item being documented
 2. The unit test *only* tests that item.
 
 The second one could be pretty annoying.  Consider cases where several
 functions interact (I've seen this many times on Microsoft's
 Documentation), and it makes sense to make one example that covers all of
 them.  Having them 'testable' means creating several identical unit
 tests.
 
 One way to easily fix this is to allow an additional parameter to the
 comment:
 
 /**
 Example(Foo.foo(int), Foo.bar(int)):
 */
 unittest
 {
 
     auto foo = new Foo;
     foo.foo(5);
     foo.bar(6);
     assert(foo.toString() == "bazunga!");
 
 }
 
 The above means, copy the example to both Foo.foo(int) and Foo.bar(int)
 
 An alternative that is more verbose, but probably more understandable:
 
 /**
 Example:
 Covers Foo.foo(int)
 Covers Foo.bar(int)
 */
 
 Of course, a lack of target just means it applies to the item just
 documented.
 
 One other thing, using writefln is considered bad form in unit tests (you
 want *no* output if the unit test works).  But many examples might want
 to demonstrate how e.g. an object interacts with writefln.  Any
 suggestions? The assert line above is not very pretty for example...
 
 -Steve

Unit-tests are defined on a module level, not a function level, hence I would expect to see the unit-tests that serve as examples to appear in an examples section on the page the documents the module itself. In the online docs I would expect the function names used in the example to be links to the individual function doc and for each function have a list of links to examples it participated in. This should be automatic and the user shouldn't need to provide the "list of functions documented in this example". Just my two cents..

Examples almost always go with functions. The whole point is to avoid having to have the function examples both in the documentation and in a unittest block. It would be a huge mess to try and put all of the examples in the module documentation. I shudder to think what that would look like for std.algorithm - or even worse (_far_ worse), std.datetime. The simple syntax of /++ Example +/ unittest { } making that unittest block go in the documentation of the preceding function should work just fine. We already have /++ Ditto +/ which puts a function in with the preceding function's documentation. So, having /++ Example +/ on top of that isn't much of a stretch.

I feel the same. When I propose how it should be and how it is right to me I feel a bit bad. Because I have not contributed anything besides talking. That's why in the end I always will follow the people's advice who actually do the work. I mean if Walter decides to leave it as it is, it's his decision until somebody provides a patch. He does the work. Who am I to tell him what he should do. Jens
Jan 25 2011
prev sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday 25 January 2011 02:10:49 foobar wrote:
 Jonathan M Davis Wrote:
 On Monday 24 January 2011 15:27:08 foobar wrote:

 Unit-tests are defined on a module level, not a function level, hence I
 would expect to see the unit-tests that serve as examples to appear in
 an examples section on the page the documents the module itself. In
 the online docs I would expect the function names used in the example
 to be links to the individual function doc and for each function have
 a list of links to examples it participated in. This should be
 automatic and the user shouldn't need to provide the "list of
 functions documented in this example".
 
 Just my two cents..

Examples almost always go with functions. The whole point is to avoid having to have the function examples both in the documentation and in a unittest block. It would be a huge mess to try and put all of the examples in the module documentation. I shudder to think what that would look like for std.algorithm - or even worse (_far_ worse), std.datetime. The simple syntax of /++ Example +/ unittest { } making that unittest block go in the documentation of the preceding function should work just fine. We already have /++ Ditto +/ which puts a function in with the preceding function's documentation. So, having /++ Example +/ on top of that isn't much of a stretch. - Jonathan M Davis

Depends on what you mean by example. It seems to me that by "Example" you mean something like a line or two that just show how to call a function. I meant by "Example" a more complicated small program that exercises several functions in the module. something more like a tutorial for using the module. For my kind of example you need to provide just one or two examples for a module, it makes more sense to put it at the module level and also in a unit-test block. Your kind of example looks less useful to me and I personally wouldn't bother with unit-testing it. I guess it's just personal style and preferences.

Look at std.algorithm if you want to see typical examples of examples (no pun intended). _Those_ are the kind of examples that we're worried about. And yes, they're generally examples per function. And if you _don't_ unit test them, then you risk them being incorrect code. By having the be in a unittest block which gets turned into an example, you know that the code is valid and that it works. And you don't have to worry about the example and the unit test being of sync. Very little in Phobos would benefit from a module-level example as you describe. Most modules hold functions which are loosely related at best (std.algorithm and std.string being prime examples). So, having a module-level example wouldn't really be worth much. - Jonathan M Davis
Jan 25 2011