www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Where are the the comments in dmd?

reply 12345swordy <alexanderheistermann gmail.com> writes:
The lack of comments in dmd makes it extremely difficult for 
newcomers such as myself to contribute.

-Alex
May 02 2021
next sibling parent reply Basile B. <b2.temp gmx.com> writes:
On Sunday, 2 May 2021 at 18:23:16 UTC, 12345swordy wrote:

 The lack of comments in dmd makes it extremely difficult for 
 newcomers such as myself to contribute.

 -Alex


Do you have any specific questions ?
May 03 2021
parent reply 12345swordy <alexanderheistermann gmail.com> writes:
On Monday, 3 May 2021 at 07:13:30 UTC, Basile B. wrote:

 On Sunday, 2 May 2021 at 18:23:16 UTC, 12345swordy wrote:

 The lack of comments in dmd makes it extremely difficult for 
 newcomers such as myself to contribute.

 -Alex


 Do you have any specific questions ?


expresstionsem.d comes to mind.

-Alex
May 03 2021
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/3/2021 6:02 AM, 12345swordy wrote:

 expresstionsem.d comes to mind.


One thing I've been doing now that my editor supports hyperlinks is for
semantic 
implementations I add a link to the spec content for the semantics. For
assembly 
instructions I'll add links to online descriptions of the instructions.

Awkward things often have a link to the bugzilla issue that spawned it for an 
explanation.

I even rant about PRs that introduce new functions that do not include Ddoc 
descriptions of the function interface. Such PRs should not pass review.
May 03 2021
prev sibling next sibling parent reply WebFreak001 <d.forum webfreak.org> writes:
On Sunday, 2 May 2021 at 18:23:16 UTC, 12345swordy wrote:

 The lack of comments in dmd makes it extremely difficult for 
 newcomers such as myself to contribute.

 -Alex


the wiki contains some stuff to getting started:

- https://wiki.dlang.org/Starting_as_a_Contributor
- https://github.com/dlang/dmd/blob/master/CONTRIBUTING.md
- https://wiki.dlang.org/DMD_Source_Guide
- https://wiki.dlang.org/Contributing_to_Phobos
- https://wiki.dlang.org/Get_involved

The forums and chats might help you when you get stuck.
May 03 2021
next sibling parent reply 12345swordy <alexanderheistermann gmail.com> writes:
On Monday, 3 May 2021 at 08:05:42 UTC, WebFreak001 wrote:

 On Sunday, 2 May 2021 at 18:23:16 UTC, 12345swordy wrote:

 The lack of comments in dmd makes it extremely difficult for 
 newcomers such as myself to contribute.

 -Alex


 the wiki contains some stuff to getting started:

 - https://wiki.dlang.org/Starting_as_a_Contributor
 - https://github.com/dlang/dmd/blob/master/CONTRIBUTING.md
 - https://wiki.dlang.org/DMD_Source_Guide
 - https://wiki.dlang.org/Contributing_to_Phobos
 - https://wiki.dlang.org/Get_involved

 The forums and chats might help you when you get stuck.


The wiki is not a replacement for good commenting practice.

-Alex
May 03 2021
parent reply Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Monday, 3 May 2021 at 13:04:03 UTC, 12345swordy wrote:

 On Monday, 3 May 2021 at 08:05:42 UTC, WebFreak001 wrote:

 On Sunday, 2 May 2021 at 18:23:16 UTC, 12345swordy wrote:

 The lack of comments in dmd makes it extremely difficult for 
 newcomers such as myself to contribute.

 -Alex


 the wiki contains some stuff to getting started:

 - https://wiki.dlang.org/Starting_as_a_Contributor
 - https://github.com/dlang/dmd/blob/master/CONTRIBUTING.md
 - https://wiki.dlang.org/DMD_Source_Guide
 - https://wiki.dlang.org/Contributing_to_Phobos
 - https://wiki.dlang.org/Get_involved

 The forums and chats might help you when you get stuck.


 The wiki is not a replacement for good commenting practice.

 -Alex


I think we can all agree there should be more comments. For sure.

But the situation is what it is and we must help fixing it 
together.

Begin with something small that is not obvious, ask questions and 
document it :)

(I know this is not the answer one wants, but it's the truth)
May 03 2021
parent reply Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Monday, 3 May 2021 at 13:15:06 UTC, Imperatorn wrote:

 On Monday, 3 May 2021 at 13:04:03 UTC, 12345swordy wrote:

 On Monday, 3 May 2021 at 08:05:42 UTC, WebFreak001 wrote:

 On Sunday, 2 May 2021 at 18:23:16 UTC, 12345swordy wrote:

 [...]


 the wiki contains some stuff to getting started:

 - https://wiki.dlang.org/Starting_as_a_Contributor
 - https://github.com/dlang/dmd/blob/master/CONTRIBUTING.md
 - https://wiki.dlang.org/DMD_Source_Guide
 - https://wiki.dlang.org/Contributing_to_Phobos
 - https://wiki.dlang.org/Get_involved

 The forums and chats might help you when you get stuck.


 The wiki is not a replacement for good commenting practice.

 -Alex


 I think we can all agree there should be more comments. For 
 sure.

 But the situation is what it is and we must help fixing it 
 together.

 Begin with something small that is not obvious, ask questions 
 and document it :)

 (I know this is not the answer one wants, but it's the truth)


Btw, just as a comparison (not trying to make a point) I looked 
at Crystal (because it was discussed earlier) and it actually has 
even less comments, not even trying in many places.
May 03 2021
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/3/2021 6:25 AM, Imperatorn wrote:

 Btw, just as a comparison (not trying to make a point) I looked at Crystal 
 (because it was discussed earlier) and it actually has even less comments, not 
 even trying in many places.


Interestingly, when I write initial code I know what I'm doing and think it is 
obvious, and rarely bother with comments.

The comments come in the second time I work on it, as I don't recall how it 
works and have to figure it out. I'll comment the non-obvious stuff I have to 
work out again.

The third, fourth, etc., times this happens is when the comments get even
better.

It's an iterative process.

Also, quite a bit of the documentation in the code is not in the comments at 
all, it is in the coding style. For example, variable name `e` is for an 
Expression, `t` for Type, `td` for TemplateDeclaration, etc. This helps a lot.

There's a lot of compiler jargon used, too. Getting a copy of the "Dragon Book" 
will help with this.

https://www.amazon.com/Compilers-Principles-Techniques-Alfred-Aho/dp/0201100886
May 03 2021
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/3/21 4:05 AM, WebFreak001 wrote:

 On Sunday, 2 May 2021 at 18:23:16 UTC, 12345swordy wrote:

 The lack of comments in dmd makes it extremely difficult for newcomers 
 such as myself to contribute.

 -Alex


 
 the wiki contains some stuff to getting started:
 
 - https://wiki.dlang.org/Starting_as_a_Contributor
 - https://github.com/dlang/dmd/blob/master/CONTRIBUTING.md
 - https://wiki.dlang.org/DMD_Source_Guide
 - https://wiki.dlang.org/Contributing_to_Phobos
 - https://wiki.dlang.org/Get_involved
 
 The forums and chats might help you when you get stuck.


Would be really nice if dmd had some substantial ddoc-generated 
documentation.
May 06 2021
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 6 May 2021 at 13:15:18 UTC, Andrei Alexandrescu 
wrote:

 Would be really nice if dmd had some substantial ddoc-generated 
 documentation.


fun fact my website generates it:

http://dmd.dpldocs.info/dmd.html

This is *significantly* better than it was a couple years ago, 
where if there were any comments at all, it was just "compiler 
implementation for the D programming language", but it could use 
more.

Still worth recognizing the recent improvement.



My site also wastes a bunch of time generating stuff in the test 
folder, i should prolly go ahead and skip those files. but that's 
my problem
May 06 2021
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Thursday, 6 May 2021 at 13:27:43 UTC, Adam D. Ruppe wrote:

 On Thursday, 6 May 2021 at 13:15:18 UTC, Andrei Alexandrescu 
 wrote:

 Would be really nice if dmd had some substantial 
 ddoc-generated documentation.


 fun fact my website generates it:

 http://dmd.dpldocs.info/dmd.html


Nice effort, but when I look at the lexer and parser modules they 
seem to be empty... so not helpful at this stage (for me).
May 06 2021
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 6 May 2021 at 13:46:31 UTC, Ola Fosheim Grøstad 
wrote:

 Nice effort, but when I look at the lexer and parser modules 
 they seem to be empty... so not helpful at this stage (for me).


You might have a cached version since they're not empty at all 
anymore, hit refresh in your browser.

That said there are a bunch of undocumented things. I can't 
generate what isn't in the source. But there's at least something.

http://dmd.dpldocs.info/dmd.lexer.Lexer.html
http://dmd.dpldocs.info/dmd.parse.Parser.html

My search engine indexed the names too:
http://search.dpldocs.info/?q=semantic2
May 06 2021
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Thursday, 6 May 2021 at 13:54:31 UTC, Adam D. Ruppe wrote:

 You might have a cached version since they're not empty at all 
 anymore, hit refresh in your browser.


My mistake. Sorry. :-P


 That said there are a bunch of undocumented things. I can't 
 generate what isn't in the source. But there's at least 
 something.

 http://dmd.dpldocs.info/dmd.lexer.Lexer.html
 http://dmd.dpldocs.info/dmd.parse.Parser.html



But the parser is much larger than this?
I guess most of it is private. Hm.
May 06 2021
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/6/2021 6:27 AM, Adam D. Ruppe wrote:

 On Thursday, 6 May 2021 at 13:15:18 UTC, Andrei Alexandrescu wrote:

 Would be really nice if dmd had some substantial ddoc-generated documentation.


 
 fun fact my website generates it:
 
 http://dmd.dpldocs.info/dmd.html


Very nice! Thanks for doing this.
May 06 2021
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/6/2021 6:15 AM, Andrei Alexandrescu wrote:

 Would be really nice if dmd had some substantial ddoc-generated documentation.



https://dlang.org/phobos/

Click on "dmd" under "Internal API".
May 06 2021
prev sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Sunday, 2 May 2021 at 18:23:16 UTC, 12345swordy wrote:

 The lack of comments in dmd makes it extremely difficult for 
 newcomers such as myself to contribute.


I agree that the code structure is difficult to get into. The 
code base should have been broken into more files and organized 
properly in directories. The different layers should also have 
been made completely separate.

E.g. things that ought to be easy can be challenging, like adding 
an additional parser should be trivial, and it isn't too 
difficult, but you have to stick to the same tokens as there 
seems to be a dependencies in error message generation. So when 
trying to change things you get this domino-effect... Like having 
to switch tokens to 16 bits to get enough tokens for two 
parsers...

Also, just to add a new file-extension type I had to modify 
multiple source files. So might have to look in multiple 
unrelated locations in order to figure out what actually 
happens... And I've only done syntax stuff, as I haven't figured 
out the best way to rebase the code base as DMD upgrades. Anyone 
know what strategy works best for rebasing?

What changes are you planning to work on?
May 03 2021
parent reply Blatnik <blatblatnik gmail.com> writes:
On Monday, 3 May 2021 at 12:30:35 UTC, Ola Fosheim Grøstad wrote:

 The lack of comments in dmd makes it extremely difficult for 
 newcomers such as myself to contribute.


 I agree that the code structure is difficult to get into. The 
 code base should have been broken into more files and organized 
 properly in directories. The different layers should also have 
 been made completely separate.


Honestly another difficult thing to get is the file names 
themselves.

`ty.d`, `ph2.d`, `mars.d`, `melf.d`, `fp.d`, `ee.d`, `e2ir.d`, 
`eh.d`, `el.d`, I could go on and on hahaha.

I get that it's probably nice to not have to type a lot in the 
import statements when you actually know what the all the files 
are after a year of hacking, but it's really hard to guess where 
something is.

Also, a lot of comments are rather useless.

```D
/*********************************
  * Loop data structure.
  */

struct loop { ... }
```

Really comment? I never would have guessed :P

```D
struct loc_t
{
     elem *e;
     int flags; // LFxxxxx
}
```

Ahhh that's what the flags are, I totally get it now. Thanks 
comment! :P

There's basically no way to know what something is for without 
doing a grep for it across the entire codebase..
May 03 2021
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Monday, 3 May 2021 at 13:17:29 UTC, Blatnik wrote:

 Honestly another difficult thing to get is the file names 
 themselves.

 `ty.d`, `ph2.d`, `mars.d`, `melf.d`, `fp.d`, `ee.d`, `e2ir.d`, 
 `eh.d`, `el.d`, I could go on and on hahaha.

 I get that it's probably nice to not have to type a lot in the 
 import statements when you actually know what the all the files 
 are after a year of hacking, but it's really hard to guess 
 where something is.


Yes, you have to start in one end and focus on that end, then 
slowly move to other files to get the structure. Directories 
would help, better names would help. Doesn't take much time to do 
either, not sure why nobody has done it yet?



 Also, a lot of comments are rather useless.


I am not sure how much comments will help (me). I have more an 
issue with long files, and basic structure.

Grepping and inserting print-statements to see what the flow 
actually is seems to be the best way to get an intuitive 
understanding of the structure with the current sourcebase.

(I am not talking about the DMD backend... I have no intention of 
looking at that one.).



 There's basically no way to know what something is for without 
 doing a grep for it across the entire codebase..


I haven't had many problems related to "what", but I keep to run 
into the problem of "where???". :-D

But, yes, grep + print-statements... You better like puzzles.
May 03 2021
parent reply Dennis <dkorpel gmail.com> writes:
On Monday, 3 May 2021 at 13:35:20 UTC, Ola Fosheim Grøstad wrote:

 Yes, you have to start in one end and focus on that end, then 
 slowly move to other files to get the structure. Directories 
 would help, better names would help. Doesn't take much time to 
 do either, not sure why nobody has done it yet?


Walter is against it because of the command line tools he uses: 
https://github.com/dlang/dmd/pull/5730#issuecomment-223936450

He suggested to make a table of contents instead:
https://github.com/dlang/dmd/pull/9844#issuecomment-495951635

So I did:
https://github.com/dlang/dmd/pull/10904

I don't know how to improve the situation further as long as 
Walter's veto stands.
May 03 2021
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Monday, 3 May 2021 at 14:34:52 UTC, Dennis wrote:

 On Monday, 3 May 2021 at 13:35:20 UTC, Ola Fosheim Grøstad 
 wrote:

 Yes, you have to start in one end and focus on that end, then 
 slowly move to other files to get the structure. Directories 
 would help, better names would help. Doesn't take much time to 
 do either, not sure why nobody has done it yet?


 Walter is against it because of the command line tools he uses: 
 https://github.com/dlang/dmd/pull/5730#issuecomment-223936450


Ok, I don't really understand why, but even if that is a good 
reason then one could simply use long prefixes instead of 
directories:

parser_token.d
parser_lex.d
parser_parse.d

etc


 I don't know how to improve the situation further as long as 
 Walter's veto stands.


I get that.

It would be nice if everyone just agreed on a common setup and 
tools then one could build a community around that with good 
tutorials, but I guess that is not very realistic.

For instance, when reading Mac tutorials, I find it very useful 
that they all use XCode. Even if there may be better editors, it 
really helps to have the same basic setup when getting started on 
something new.
May 03 2021
prev sibling next sibling parent reply Dennis <dkorpel gmail.com> writes:
On Monday, 3 May 2021 at 13:17:29 UTC, Blatnik wrote:

 Honestly another difficult thing to get is the file names 
 themselves.

 `ty.d`, `ph2.d`, `mars.d`, `melf.d`, `fp.d`, `ee.d`, `e2ir.d`, 
 `eh.d`, `el.d`, I could go on and on hahaha.


I think this stems from Walter Bright making C/C++ compilers 
since the 80s.
In the MS DOS days the [8.3 
filename](https://en.wikipedia.org/wiki/8.3_filename) was the 
norm, so you could not name `eh.c` `exceptionhandling.c` or 
`exception.c` even if you wanted to.

You can see the meaning of each file here:
https://github.com/dlang/dmd/tree/master/src/dmd#dmd-source-code


 ```D
 struct loc_t
 {
     elem *e;
     int flags; // LFxxxxx
 }
 ```

 Ahhh that's what the flags are, I totally get it now. Thanks 
 comment! :P


Another archaism: C doesn't have proper enum types, so the 
comment says the flags are defined by the macros prefixed with 
`LF`.

Of course your point still stands that dmd is not commented well 
and not easy for new contributors get into, but a lot of the 
ugliness is there because of historic reasons.
May 03 2021
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/3/2021 7:25 AM, Dennis wrote:

 On Monday, 3 May 2021 at 13:17:29 UTC, Blatnik wrote:

 Honestly another difficult thing to get is the file names themselves.

 `ty.d`, `ph2.d`, `mars.d`, `melf.d`, `fp.d`, `ee.d`, `e2ir.d`, `eh.d`, `el.d`, 
 I could go on and on hahaha.


 
 I think this stems from Walter Bright making C/C++ compilers since the 80s.
 In the MS DOS days the [8.3 
 filename](https://en.wikipedia.org/wiki/8.3_filename) was the norm, so you
could 
 not name `eh.c` `exceptionhandling.c` or `exception.c` even if you wanted to.


You're right about 8.3. I learned programming back when it was 6.3, and some
old 
habits persist.

But also, when you're going to be typing those filenames tens of thousands of 
times, shorter is better :-)

Things always get optimized for the long term users, not newbies. For example, 
nobody says International Business Machines, they say IBM. San Francisco is 
Frisco. The United States is "the States". Oxycontin is Oxy. Elizabeth is Liz. 
The University of Washington is just "the dub" around here. It's not a problem.



 Of course your point still stands that dmd is not commented well and not easy 
 for new contributors get into, but a lot of the ugliness is there because of 
 historic reasons.


Parts of it date back to 1982, when C compilers were rather primitive.
May 03 2021
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/3/2021 6:17 AM, Blatnik wrote:

 Honestly another difficult thing to get is the file names themselves.
 
 `ty.d`


TYpes


 `ph2.d`


Precompiled Headers


 `mars.d`


Entry point. So named when D was originally the "Mars" programming language. It 
was the first source file! I tend to name the source file that contains the 
program entry point with the name of the ultimate executable.


 `melf.d`


The D translation of "melf.h", a system file


 `fp.d`


Floating Point


 `ee.d`


Execution Environment


 `e2ir.d`,


Expression To Intermediate Representation


 `eh.d`


Exception Handling


 `el.d`


ELements


 I get that it's probably nice to not have to type a lot in the import
statements 
 when you actually know what the all the files are after a year of hacking, but 
 it's really hard to guess where something is.


Now you know. No need to guess anymore!


 Also, a lot of comments are rather useless.
 
 /*********************************
   * Loop data structure.
   */
 
 struct loop { ... }



Holds the state for analyzing Loops in the optimizer



 struct loc_t
 {
      elem *e;
      int flags; // LFxxxxx
 }
 ```
 
 Ahhh that's what the flags are, I totally get it now. Thanks comment! :P


No reason to copy/paste the LF comments.


 There's basically no way to know what something is for without doing a grep
for 
 it across the entire codebase..


Indeed. Grep is your best friend, unless you have an IDE that can automatically 
take you to the definition of a type.
May 07 2021
parent reply Sputnic <sput gmail.com> writes:
On Friday, 7 May 2021 at 09:32:11 UTC, Walter Bright wrote:

 On 5/3/2021 6:17 AM, Blatnik wrote:

 Honestly another difficult thing to get is the file names 
 themselves.
 
 `ty.d`


 TYpes


 `ph2.d`


 Precompiled Headers


 `mars.d`


 Entry point. So named when D was originally the "Mars" 
 programming language. It was the first source file! I tend to 
 name the source file that contains the program entry point with 
 the name of the ultimate executable.


 `melf.d`


 The D translation of "melf.h", a system file


 `fp.d`


 Floating Point


 `ee.d`


 Execution Environment


 `e2ir.d`,


 Expression To Intermediate Representation


 `eh.d`


 Exception Handling


 `el.d`


 ELements



Couldn't these be renamed to be more readable? I don't think it 
is reasonable to expect someone to know that "ee" stands for 
execution environment. This would be a step towards reducing the 
barrier to entry for DMD.
May 07 2021
next sibling parent reply Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Friday, 7 May 2021 at 15:57:45 UTC, Sputnic wrote:

 On Friday, 7 May 2021 at 09:32:11 UTC, Walter Bright wrote:

 [...]



 Couldn't these be renamed to be more readable? I don't think it 
 is reasonable to expect someone to know that "ee" stands for 
 execution environment. This would be a step towards reducing 
 the barrier to entry for DMD.


Either that or add a comment.

Personally I don't have a problem with short names, but if I 
don't know what they mean, I have 🌞
May 07 2021
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/7/2021 9:02 AM, Imperatorn wrote:

 Either that or add a comment.


As Dennis posted:

You can see the meaning of each file here:
https://github.com/dlang/dmd/tree/master/src/dmd#dmd-source-code
May 07 2021
parent Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Friday, 7 May 2021 at 19:12:07 UTC, Walter Bright wrote:

 On 5/7/2021 9:02 AM, Imperatorn wrote:

 Either that or add a comment.


 As Dennis posted:

 You can see the meaning of each file here:
 https://github.com/dlang/dmd/tree/master/src/dmd#dmd-source-code


Nice, that's actually a quite good summary. Thx
May 07 2021
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/7/2021 8:57 AM, Sputnic wrote:

 Couldn't these be renamed to be more readable? I don't think it is reasonable
to 
 expect someone to know that "ee" stands for execution environment. This would
be 
 a step towards reducing the barrier to entry for DMD.


People hear "Central Intelligence Agency" once and ever after use CIA. The same 
goes for everything else.

DMD source code should be optimized for the people who spend thousands of hours 
on it. For new people, the link to the glossary has already been posted here.

Some years back, someone did a Pull Request to rename the "paren" token to 
"Parentheses" (s.i.c.). This is used several hundred times in the source code. 
It's more annoying than illuminating.
May 07 2021