www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - __traits(documentation, X)

reply Seb <seb wilzba.ch> writes:
What
----

```
/// my fancy string
enum documentedEnum = 1;
enum funcDoc = __traits(documentation, documentedFunc);
assert(funcDoc == "my fancy string")
```

See https://github.com/dlang/dmd/pull/6872 for better examples


Status
-------

The naive implementation leads to a small, but noticeable 
increase of DMD's compilation time.

Andrei's words:

 I'd say keep it on the back burner until we find a couple of 
 good ideas for using it. Discussing it in the forum may help.
So do you have a good use cases for this? If this is a useful feature, the implementation can be improved to be zero-cost for normal runs. Post here or at https://github.com/dlang/dmd/pull/6872
Jan 16
next sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
Just a fun bit of history:

A few years ago I looked into implementing this myself.
Discovering that without -D documentation is lost.

Why did I want to do this?
 From memory it was to allow me to give greater control over exportation 
of models+routes in Cmsed as part of documentation (or something along 
those lines).

I wouldn't want to do it this way now, but it does still feel like a 
hole in traits to me.

But imagine a use case of translations:

/**
  * Translation_English:
  *     Foo bar
  * Translation_X:
  *     SDFJL
  */
string welcome(Language language) {
	return getSectionFromDDOC(__traits(comments, welcome), "Translation_" ~ 
language.text);
}
Jan 16
prev sibling next sibling parent Laeeth Isharc <laeeth laeeth.com> writes:
On Wednesday, 17 January 2018 at 02:19:11 UTC, Seb wrote:
 What
 ----

 ```
 /// my fancy string
 enum documentedEnum = 1;
 enum funcDoc = __traits(documentation, documentedFunc);
 assert(funcDoc == "my fancy string")
 ```

 See https://github.com/dlang/dmd/pull/6872 for better examples


 Status
 -------

 The naive implementation leads to a small, but noticeable 
 increase of DMD's compilation time.

 Andrei's words:

 I'd say keep it on the back burner until we find a couple of 
 good ideas for using it. Discussing it in the forum may help.
So do you have a good use cases for this? If this is a useful feature, the implementation can be improved to be zero-cost for normal runs. Post here or at https://github.com/dlang/dmd/pull/6872
In my case might be useful for generating foreign doc strings for automatically generated wrappers in other languages and for D functions intended for use directly but also mounted into an interpreted DSL.. You could duplicate docstring with attributes but hardly pretty.
Jan 16
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 1/16/18 9:19 PM, Seb wrote:
 What
 ----
 
 ```
 /// my fancy string
 enum documentedEnum = 1;
 enum funcDoc = __traits(documentation, documentedFunc);
 assert(funcDoc == "my fancy string")
 ```
 
 See https://github.com/dlang/dmd/pull/6872 for better examples
 
 
 Status
 -------
 
 The naive implementation leads to a small, but noticeable increase of 
 DMD's compilation time.
 
 Andrei's words:
 
 I'd say keep it on the back burner until we find a couple of good 
 ideas for using it. Discussing it in the forum may help.
So do you have a good use cases for this? If this is a useful feature, the implementation can be improved to be zero-cost for normal runs. Post here or at https://github.com/dlang/dmd/pull/6872
I shudder at the thought of compiled code being affected by documentation. I don't like it, sorry. -Steve
Jan 16
parent reply ketmar <ketmar ketmar.no-ip.org> writes:
Steven Schveighoffer wrote:

 I shudder at the thought of compiled code being affected by documentation.

 I don't like it, sorry.
it's not really different from `version(DDoc)`, or string mixins, or UDAs. i can put documentation in UDA, and process code differently when UDA changes. yes, UDA is not a comment -- but it doesn't affect code generation directly too. there is just no way to stop people from writing abominations once we got metaprogramming. and adding artificial limitations on that just makes people more inventive, but won't stop infestation. ;-)
Jan 16
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 1/16/18 10:59 PM, ketmar wrote:
 Steven Schveighoffer wrote:
 
 I shudder at the thought of compiled code being affected by 
 documentation.

 I don't like it, sorry.
it's not really different from `version(DDoc)`, or string mixins, or UDAs. i can put documentation in UDA, and process code differently when UDA changes. yes, UDA is not a comment -- but it doesn't affect code generation directly too. there is just no way to stop people from writing abominations once we got metaprogramming. and adding artificial limitations on that just makes people more inventive, but won't stop infestation. ;-)
version(DDoc) is very different. It's a command line option passed to the build, and affecting the build with command line options is expected. This proposal has comment *contents* affecting the build. The charter of comments is to NOT affect code, ever. They can be used to affect other systems, such as the ddoc generator, ide hints, etc., but you can be sure (?) that comments won't change code. I think this is an important line not to cross. One of the features promoted in this is to have the documentation be used for command-line help. I think it would be better to allow ddoc generation be affected by code rather than the other way around in this regard. -Steve
Jan 17
next sibling parent rjframe <dlang ryanjframe.com> writes:
On Wed, 17 Jan 2018 11:52:40 -0500, Steven Schveighoffer wrote:

 version(DDoc) is very different. It's a command line option passed to
 the build, and affecting the build with command line options is
 expected.
 
 This proposal has comment *contents* affecting the build. The charter of
 comments is to NOT affect code, ever. They can be used to affect other
 systems, such as the ddoc generator, ide hints, etc., but you can be
 sure (?) that comments won't change code.
 
 I think this is an important line not to cross.
I have to agree with this. If translated documentation is the primary use case, I'd imagine we could find better ways to do it.
Jan 17
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Jan 17, 2018 at 11:52:40AM -0500, Steven Schveighoffer via
Digitalmars-d wrote:
[...]
 This proposal has comment *contents* affecting the build. The charter
 of comments is to NOT affect code, ever. They can be used to affect
 other systems, such as the ddoc generator, ide hints, etc., but you
 can be sure (?) that comments won't change code.
 
 I think this is an important line not to cross.
+1. The thought of comments affecting code scares me, and not just a little.
 One of the features promoted in this is to have the documentation be
 used for command-line help. I think it would be better to allow ddoc
 generation be affected by code rather than the other way around in
 this regard.
[...] That's actually a much better idea. Have some way of injecting, say, a string UDA into ddoc so that you can have one source that generates both documentation and command-line help. The latter is already possible, and easy to implement; now we just need the former. T -- MAS = Mana Ada Sistem?
Jan 17
prev sibling next sibling parent reply Simen =?UTF-8?B?S2rDpnLDpXM=?= <simen.kjaras gmail.com> writes:
On Wednesday, 17 January 2018 at 16:52:40 UTC, Steven 
Schveighoffer wrote:
 The charter of comments is to NOT affect code, ever. They can 
 be used to affect other systems, such as the ddoc generator, 
 ide hints, etc., but you can be sure (?) that comments won't 
 change code.

 I think this is an important line not to cross.
That's already perfectly possible today: enum a = import(__FILE__); static if (a.indexOf("//") > -1) { fireZeMissiles(); } -- Simen
Jan 17
next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Jan 17, 2018 at 06:24:50PM +0000, Simen Kjrs via Digitalmars-d wrote:
 On Wednesday, 17 January 2018 at 16:52:40 UTC, Steven Schveighoffer wrote:
 The charter of comments is to NOT affect code, ever. They can be
 used to affect other systems, such as the ddoc generator, ide hints,
 etc., but you can be sure (?) that comments won't change code.
 
 I think this is an important line not to cross.
That's already perfectly possible today: enum a = import(__FILE__); static if (a.indexOf("//") > -1) { fireZeMissiles(); }
[...] I can see it now. In the year 2030, a major exploit is found in a widely-deployed online D application, causing massive data loss and damages. The cause? A misplaced comment... :-P T -- Never trust an operating system you don't have source for! -- Martin Schulze
Jan 17
prev sibling next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Jan 17, 2018 at 06:24:50PM +0000, Simen Kjrs via Digitalmars-d wrote:
[...]
 enum a = import(__FILE__);
 static if (a.indexOf("//") > -1) {
     fireZeMissiles();
 }
[...] OTOH, this combination of string import and self-referencing __FILE__ opens up a host of curious possibilities (all of which already work today, btw): 1) Trivial self-reproducing program in D: import std.stdio; void main() { write(import(__FILE__)); } 2) Self-checking programs: import std.algorithm; static assert(import(__FILE__).canFind("\n/**"), "Dude, why did you remove the ddoc comment?!"); /** Remove this comment to get a compile error */ void main() { } 3) Underhanded / deceptive "self-modifying" code: version(none) { // This code will never get compiled, right? Right??? import std.stdio; void main() { writeln("Launch nuclear missiles"); } } else { mixin(() { import std.array : replace; return import(__FILE__).replace("none", "all"); }()); } These are just trivial examples, of course. One can easily imagine more sophisticated variations that can do all sorts of weird / dangerous things. Imagine an advanced version of (2), for example, code that will force a compilation error if it wasn't formatted correctly. Or an advanced form of (3) where the real program code is inside an encrypted comment written in a different language that gets compiled into D by a CTFE compiler. Or a CTFE implementation of AST macros that uses libdparse to parse itself, apply modifications, and mixin. :-D T -- "Maybe" is a strange word. When mom or dad says it it means "yes", but when my big brothers say it it means "no"! -- PJ jr.
Jan 17
parent Simen =?UTF-8?B?S2rDpnLDpXM=?= <simen.kjaras gmail.com> writes:
On Wednesday, 17 January 2018 at 22:15:08 UTC, H. S. Teoh wrote:
 2) Self-checking programs:

 	import std.algorithm;
 	static assert(import(__FILE__).canFind("\n/**"),
 		"Dude, why did you remove the ddoc comment?!");

 	/** Remove this comment to get a compile error */
 	void main() {
 	}
[snip]
 Imagine an advanced version of (2), for example, code that will 
 force a compilation error if it wasn't formatted correctly.
I found an interesting Rust project a few years back, called Launch-Code: https://github.com/kmcallister/launch-code The idea is that unsafe functions be cryptographically signed by some authority, and the code refuses to compile if a junior dev tries and fuck with it. On a per-function basis, so he could still mess up the code that doesn't launch missiles. We could do the same in D - parse the code with CTFE, hash the function body, then assert that the comment matches the hash. Should be fairly easy at that point to generate the signatures as well. -- Simen
Jan 18
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Jan 17, 2018 at 02:15:08PM -0800, H. S. Teoh via Digitalmars-d wrote:
[...]
 OTOH, this combination of string import and self-referencing __FILE__
 opens up a host of curious possibilities (all of which already work
 today, btw):
[...] Another interesting one: breaking through the barriers of modularization: // mod.d string getCode(string file = __FILE__)() { return import(file); } // main.d import mod; void main() { pragma(msg, getCode); } Here, module mod gains access to the source code of a file outside of itself, and it doesn't even need prior knowledge of the filenames. In this case nothing interesting happens, of course. But imagine if getCode did something else, like extract private symbol definitions, or ran a syntax checker, or AST transformations, etc.. All sorts of interesting effects ensue. Coming back on topic, one can easily imagine a CTFE D parser that basically implements __traits(documentation) as *library code*. You won't even need compiler support for that. Wow. Skeleton code: string traitsDocumentation(alias symbol, string file = __FILE__)() { enum code = import(file); auto s = searchForDocComment(symbol.stringof); return s; } T -- Let's call it an accidental feature. -- Larry Wall
Jan 17
prev sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 1/17/18 1:24 PM, Simen Kjærås wrote:
 On Wednesday, 17 January 2018 at 16:52:40 UTC, Steven Schveighoffer wrote:
 The charter of comments is to NOT affect code, ever. They can be used 
 to affect other systems, such as the ddoc generator, ide hints, etc., 
 but you can be sure (?) that comments won't change code.

 I think this is an important line not to cross.
That's already perfectly possible today: enum a = import(__FILE__); static if (a.indexOf("//") > -1) {     fireZeMissiles(); }
Ironically, this doesn't need any comments to trigger :) Furthermore, I'd ask, if it's possible today, why do we need a __traits to do it? Having a __traits option for looking at documentation sounds like a very rare need in the first place to warrant promoting such a hole in the expectations for comments. I find this to also be undesirable, and I would flag any code that used such a technique as unacceptable in code review. -Steve
Jan 18
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 18 January 2018 at 16:41:04 UTC, Steven 
Schveighoffer wrote:
 Furthermore, I'd ask, if it's possible today, why do we need a 
 __traits to do it?
It is an enormous pain to do it now.... well, sort of, actually, the way I'd do it now is do a two-step makefile, where step 1 runs a ddoc generator and step 2 import("generated.html") it. But, for stuff like little command line arg stuff, or web interface creation, it is nice to be able to print that data more conveniently. Any code that uses it in an evil way should be stupid, but writeln(__traits(documentation, foo)) is useful and not really harmless.
Jan 18
next sibling parent Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Thursday, 18 January 2018 at 16:48:45 UTC, Adam D. Ruppe wrote:
 On Thursday, 18 January 2018 at 16:41:04 UTC, Steven 
 Schveighoffer wrote:
 Furthermore, I'd ask, if it's possible today, why do we need a 
 __traits to do it?
It is an enormous pain to do it now.... well, sort of, actually, the way I'd do it now is do a two-step makefile, where step 1 runs a ddoc generator and step 2 import("generated.html") it. But, for stuff like little command line arg stuff, or web interface creation, it is nice to be able to print that data more conveniently. Any code that uses it in an evil way should be stupid, but writeln(__traits(documentation, foo)) is useful and not really harmless.
Irony?
Jan 18
prev sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 1/18/18 11:48 AM, Adam D. Ruppe wrote:
 On Thursday, 18 January 2018 at 16:41:04 UTC, Steven Schveighoffer wrote:
 Furthermore, I'd ask, if it's possible today, why do we need a 
 __traits to do it?
It is an enormous pain to do it now.... well, sort of, actually, the way I'd do it now is do a two-step makefile, where step 1 runs a ddoc generator and step 2 import("generated.html") it.
This is how I'd imagine doing something like this. I don't see it being a huge pain, just an extra build step.
 But, for stuff like little command line arg stuff, or web interface 
 creation, it is nice to be able to print that data more conveniently. 
 Any code that uses it in an evil way should be stupid, but 
 writeln(__traits(documentation, foo)) is useful and not really harmless.
 
Did you mean not really harmful? But in any case, the idea that comments affect the file you are compiling *right now*, and not some other tool-generated file makes me very nervous. Comments are supposed to not affect the code. Consider that with this feature, the documentation now becomes part of the API. -Steve
Jan 18
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 18 January 2018 at 18:31:28 UTC, Steven 
Schveighoffer wrote:
 This is how I'd imagine doing something like this. I don't see 
 it being a huge pain, just an extra build step.
Yeah, it isn't terrible, but it just seems heavy for something small like a command line app printing out its options.
 Did you mean not really harmful?
lol yeah.
 But in any case, the idea that comments affect the file you are 
 compiling *right now*, and not some other tool-generated file 
 makes me very nervous. Comments are supposed to not affect the 
 code. Consider that with this feature, the documentation now 
 becomes part of the API.
Meh, I see your point, I just don't agree it is that important. I'm iffy on a lot of the things people do with UFCS and optional parenthesis/properties and think it is ugly code*, but on the balance, I still like the feature. * did you see that SO thing a bit ago about " can if(a == 1 && a == 2 && a == 3) ever be true"? Ugly code there but impure properties are useful in other cases.
Jan 18
parent reply Timothee Cour <thelastmammoth gmail.com> writes:
 But in any case, the idea that comments affect the file you are compiling
 *right now*, and not some other tool-generated file makes me very nervous.
 Comments are supposed to not affect the code. Consider that with this
 feature, the documentation now becomes part of the API.
That's already the case today that documentation can affect the code because `import(__FILE__)` is legal. Both string import and `__traits(documentation)` are tools and can be used for solving real problems or abused. Let's not invoke (impossible) ideals at the expense of pragmatism; D is supposed to be pragmatic, not dogmatic.
Jan 18
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 1/18/18 3:21 PM, Timothee Cour wrote:
 But in any case, the idea that comments affect the file you are compiling
 *right now*, and not some other tool-generated file makes me very nervous.
 Comments are supposed to not affect the code. Consider that with this
 feature, the documentation now becomes part of the API.
That's already the case today that documentation can affect the code because `import(__FILE__)` is legal. Both string import and `__traits(documentation)` are tools and can be used for solving real problems or abused. Let's not invoke (impossible) ideals at the expense of pragmatism; D is supposed to be pragmatic, not dogmatic.
Of course, you can cobble together something that uses the docs in your build process, you don't need the compiler to help create such a monster. After all, they are just text files. The point remains -- if you normalize this feature, it will be abused far easier and more frequently than it would be if it's less trivial. In any case, I stand by my assertion -- comments should be comments, not code. A related improvement: Documented unit tests. Instead of adding a feature to D where examples in the documentation were tested directly (and I'm pretty sure this was proposed), we added the ability to document unit tests. I think this was a far saner direction. A further note: Does this need go away when we make the compiler a library? Then you can extract the docs all you want and manipulate them into whatever you want using a build tool. -Steve
Jan 18
next sibling parent Seb <seb wilzba.ch> writes:
On Thursday, 18 January 2018 at 20:35:54 UTC, Steven 
Schveighoffer wrote:
 A further note: Does this need go away when we make the 
 compiler a library? Then you can extract the docs all you want 
 and manipulate them into whatever you want using a build tool.

 -Steve
You can already do this fairly easy with libdparse. No need for a compiler with semantics here. Here's a D-Scanner test I wrote a while ago that checks whether all comments of public symbols have a Params section of all their parameters and a returns section if they don't return void: https://github.com/dlang-community/D-Scanner/blob/master/src/analysis/properly_documented_public_functions.d It's a shame it's only partially enabled on Phobos as there are many old modules which I had to blacklist initially.
Jan 18
prev sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 18 January 2018 at 20:35:54 UTC, Steven 
Schveighoffer wrote:
 In any case, I stand by my assertion -- comments should be 
 comments, not code.
Note that we aren't talking about comments. We're talking about ddoc strings. They just have very similar appearance to comments, but they are distinct entities defined by D to be part of the AST (and the compiler will parse their contents too if you ask it to!), unlike regular comments which are discarded before parsing. That's really the big difference that puts me over the top: documentation strings in D are defined to be attached to the symbol by the compiler. They're ALREADY different than plain comment-comments, just the compiler withholds this information from the code.
Jan 18
next sibling parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 1/18/18 4:01 PM, Adam D. Ruppe wrote:
 On Thursday, 18 January 2018 at 20:35:54 UTC, Steven Schveighoffer wrote:
 In any case, I stand by my assertion -- comments should be comments, 
 not code.
Note that we aren't talking about comments. We're talking about ddoc strings. They just have very similar appearance to comments, but they are distinct entities defined by D to be part of the AST (and the compiler will parse their contents too if you ask it to!), unlike regular comments which are discarded before parsing.
Hm... I don't see where Ddoc comments are defined separately from comments in the grammar. How they are stored internally is up to the compiler, and as far as I know, the D *language* makes no guarantees on what comes out: "The specification for the form of embedded documentation comments only specifies how information is to be presented to the compiler. It is implementation-defined how that information is used and the form of the final presentation." This brings up another thing: What if the compiler you are using decides to do something completely different with the documentation (or maybe even nothing)? Then __traits(documentation, X) may be completely different and cause different code to be generated. -Steve
Jan 18
prev sibling parent cosinus <dummy dummy.dummy> writes:
On Thursday, 18 January 2018 at 21:01:01 UTC, Adam D. Ruppe wrote:
 Note that we aren't talking about comments. We're talking about 
 ddoc strings. They just have very similar appearance to 
 comments, but they are distinct entities defined by D to be 
 part of the AST (and the compiler will parse their contents too 
 if you ask it to!), unlike regular comments which are discarded 
 before parsing.
I think `__traits(documentation, X)` makes sens if the documentation was generated by library (AST -> HTML) But as far as I know the documentation can't be generated by CTFE directly.
Jan 18
prev sibling parent Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Thursday, 18 January 2018 at 18:31:28 UTC, Steven 
Schveighoffer wrote:
 On 1/18/18 11:48 AM, Adam D. Ruppe wrote:
 [...]
This is how I'd imagine doing something like this. I don't see it being a huge pain, just an extra build step.
 [...]
Did you mean not really harmful? But in any case, the idea that comments affect the file you are compiling *right now*, and not some other tool-generated file makes me very nervous. Comments are supposed to not affect the code. Consider that with this feature, the documentation now becomes part of the API.
While I'm on your side of the issue, it is a fact that ddoc (or doxygen, javadoc) comments are more a language within a language (i.e. there is syntax in them) than simple comments.
Jan 18
prev sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Wednesday, January 17, 2018 11:52:40 Steven Schveighoffer via 
Digitalmars-d wrote:
 On 1/16/18 10:59 PM, ketmar wrote:
 Steven Schveighoffer wrote:
 I shudder at the thought of compiled code being affected by
 documentation.

 I don't like it, sorry.
it's not really different from `version(DDoc)`, or string mixins, or UDAs. i can put documentation in UDA, and process code differently when UDA changes. yes, UDA is not a comment -- but it doesn't affect code generation directly too. there is just no way to stop people from writing abominations once we got metaprogramming. and adding artificial limitations on that just makes people more inventive, but won't stop infestation. ;-)
version(DDoc) is very different. It's a command line option passed to the build, and affecting the build with command line options is expected. This proposal has comment *contents* affecting the build. The charter of comments is to NOT affect code, ever. They can be used to affect other systems, such as the ddoc generator, ide hints, etc., but you can be sure (?) that comments won't change code. I think this is an important line not to cross. One of the features promoted in this is to have the documentation be used for command-line help. I think it would be better to allow ddoc generation be affected by code rather than the other way around in this regard.
+1 - Jonathan M Davis
Jan 17
prev sibling parent Dukc <ajieskola gmail.com> writes:
On Wednesday, 17 January 2018 at 02:19:11 UTC, Seb wrote:
 So do you have a good use cases for this?
 If this is a useful feature, the implementation can be improved 
 to be zero-cost for normal runs.
If a GUI button simply executes a function. Tooltip for it could be generated from the function documentation. About being able to use the documentation comment in mixins, of course it would be an ill recommended thing to do. But I don't think it would become much of an issue since you can hardly do that accidently. IMO, D can already be used more dangerously than even C++ if you want to: binary imports, string mixins and static ifs can be used for far more sophiscated hacks than the C preprocessor. But it does not matter since they do not encourage or trick anybody to do the said hacks.
Jan 18