www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - So... let's document dmd

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
https://github.com/D-Programming-Language/dlang.org/pull/1196
https://github.com/D-Programming-Language/dmd/pull/5352

Destroy!!

Andrei
Jan 14
parent reply Jack Stouffer <jack jackstouffer.com> writes:
On Friday, 15 January 2016 at 04:53:16 UTC, Andrei Alexandrescu 
wrote:
 https://github.com/D-Programming-Language/dlang.org/pull/1196
 https://github.com/D-Programming-Language/dmd/pull/5352

 Destroy!!

 Andrei
Since dmd is now in D, is there any reason to not allow people to import from it while using dmd? Wouldn't it make anyone's life who wanted to parse D code easier by just doing import ddmd.parser; (or what ever it is) and use the actual parser?
Jan 15
parent reply Joakim <dlang joakim.fea.st> writes:
On Friday, 15 January 2016 at 23:58:32 UTC, Jack Stouffer wrote:
 On Friday, 15 January 2016 at 04:53:16 UTC, Andrei Alexandrescu 
 wrote:
 https://github.com/D-Programming-Language/dlang.org/pull/1196
 https://github.com/D-Programming-Language/dmd/pull/5352

 Destroy!!

 Andrei
Since dmd is now in D, is there any reason to not allow people to import from it while using dmd? Wouldn't it make anyone's life who wanted to parse D code easier by just doing import ddmd.parser; (or what ever it is) and use the actual parser?
The ddmd lexer is already on dub, just have to add the parser next: http://code.dlang.org/packages/ddmd
Jan 15
parent reply Jack Stouffer <jack jackstouffer.com> writes:
On Saturday, 16 January 2016 at 05:11:57 UTC, Joakim wrote:
 The ddmd lexer is already on dub, just have to add the parser 
 next:

 http://code.dlang.org/packages/ddmd
You missed the main point of my question.
Jan 15
parent reply tsbockman <thomas.bockman gmail.com> writes:
On Saturday, 16 January 2016 at 05:29:03 UTC, Jack Stouffer wrote:
 On Saturday, 16 January 2016 at 05:11:57 UTC, Joakim wrote:
 The ddmd lexer is already on dub, just have to add the parser 
 next:

 http://code.dlang.org/packages/ddmd
You missed the main point of my question.
I'm confused as to your point as well. Perhaps you could rephrase?
Jan 15
next sibling parent Joakim <dlang joakim.fea.st> writes:
On Saturday, 16 January 2016 at 05:36:23 UTC, tsbockman wrote:
 On Saturday, 16 January 2016 at 05:29:03 UTC, Jack Stouffer 
 wrote:
 On Saturday, 16 January 2016 at 05:11:57 UTC, Joakim wrote:
 The ddmd lexer is already on dub, just have to add the parser 
 next:

 http://code.dlang.org/packages/ddmd
You missed the main point of my question.
I'm confused as to your point as well. Perhaps you could rephrase?
I think he's trying to emphasize that he'd like it to come with dmd. There was talk of putting the lexer/parser into phobos, but I believe dub was chosen as a staging ground for people to play with it first. So if you want to use the lexer/parser, use it from dub for now, then push for its inclusion into phobos later.
Jan 15
prev sibling parent reply Jack Stouffer <jack jackstouffer.com> writes:
On Saturday, 16 January 2016 at 05:36:23 UTC, tsbockman wrote:
 On Saturday, 16 January 2016 at 05:29:03 UTC, Jack Stouffer 
 wrote:
 On Saturday, 16 January 2016 at 05:11:57 UTC, Joakim wrote:
 The ddmd lexer is already on dub, just have to add the parser 
 next:

 http://code.dlang.org/packages/ddmd
You missed the main point of my question.
I'm confused as to your point as well. Perhaps you could rephrase?
dmd is now in D; theoretically that should allow for other projects to import from it like a normal D project. So why not make all of the ddmd modules available from any code that is complied with it, i.e. just like Phobos? The parser was just an example, there are any number of things that one could use from the code base.
Jan 15
parent reply tsbockman <thomas.bockman gmail.com> writes:
On Saturday, 16 January 2016 at 07:52:11 UTC, Jack Stouffer wrote:
 dmd is now in D; theoretically that should allow for other 
 projects to import from it like a normal D project. So why not 
 make all of the ddmd modules available from any code that is 
 complied with it, i.e. just like Phobos?

 The parser was just an example, there are any number of things 
 that one could use from the code base.
That would be cool in theory, but it would likely become a major drag on development of the compiler itself. Once you start promoting the compiler's internals as part of the standard library, you can expect sooner or later that pressure will mount to bring them up to the same high standards of API design and stability as Phobos. While this process would be beneficial to the overall quality of the DMD code being exposed, it would also turn internal refactorings into public breaking changes. I really doubt that's a good trade-off at this point in DMD's development.
Jan 16
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Sat, Jan 16, 2016 at 08:35:40AM +0000, tsbockman via Digitalmars-d wrote:
 On Saturday, 16 January 2016 at 07:52:11 UTC, Jack Stouffer wrote:
dmd is now in D; theoretically that should allow for other projects
to import from it like a normal D project. So why not make all of the
ddmd modules available from any code that is complied with it, i.e.
just like Phobos?

The parser was just an example, there are any number of things that
one could use from the code base.
That would be cool in theory, but it would likely become a major drag on development of the compiler itself. Once you start promoting the compiler's internals as part of the standard library, you can expect sooner or later that pressure will mount to bring them up to the same high standards of API design and stability as Phobos. While this process would be beneficial to the overall quality of the DMD code being exposed, it would also turn internal refactorings into public breaking changes. I really doubt that's a good trade-off at this point in DMD's development.
I disagree. I think having the dmd itself (lexer, parser, etc.) as a library (with the dmd executable merely being the default frontend) will do D a lot of good. For one thing, IDE's will no longer need to reinvent a D parser for the purposes of syntax highlighting; they can reuse the actual parser the compiler itself uses, and thus actually be correct (whereas their own parser may actually only parse a subset of D correctly). They will also be able to support instant feedback as the user types in D code (highlight syntax errors on the fly, etc.). This will also help with writing various tools that operate on D code, e.g., pretty-printers like dfmt won't need to roll its own lexer/parser anymore. Many other tools also become possible (automatic refactorings, function call tree analysis, dependency analysis, etc.). You could even use libdmd in your own D code, thus being able to have the equivalent of the `eval` available in many scripting languages (except that it will have native performance(!)). The problem of dmd refactorings becoming public breaking changes is not as big a problem if the public API is properly designed. If the compiler code itself is refactored to interface between its sub-components via cleanly-designed APIs with proper encapsulations, I don't see that it will become prohibitively difficult to minimize public API changes without hampering compiler development efforts. This is the way to go for better compiler code quality anyway, so even if libdmd never happens, the code should *still* be refactored in this way. Besides, I don't think anyone is sanely suggesting that we allow dmd source files to be free-for-all externally-importable modules. The more likely scenario is that there will be a clearly-defined external interface that is unlikely to change much, whereas the implementation underneath is still free to be changed, refactored, reorganized, rewritten, etc.. T -- BREAKFAST.COM halted...Cereal Port Not Responding. -- YHL
Jan 16
next sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Saturday, 16 January 2016 at 15:13:28 UTC, H. S. Teoh wrote:
 On Sat, Jan 16, 2016 at 08:35:40AM +0000, tsbockman via 
 Digitalmars-d wrote:
 On Saturday, 16 January 2016 at 07:52:11 UTC, Jack Stouffer 
 wrote:
dmd is now in D; theoretically that should allow for other 
projects to import from it like a normal D project. So why 
not make all of the ddmd modules available from any code that 
is complied with it, i.e. just like Phobos?

The parser was just an example, there are any number of 
things that one could use from the code base.
That would be cool in theory, but it would likely become a major drag on development of the compiler itself. Once you start promoting the compiler's internals as part of the standard library, you can expect sooner or later that pressure will mount to bring them up to the same high standards of API design and stability as Phobos. While this process would be beneficial to the overall quality of the DMD code being exposed, it would also turn internal refactorings into public breaking changes. I really doubt that's a good trade-off at this point in DMD's development.
I disagree. I think having the dmd itself (lexer, parser, etc.) as a library (with the dmd executable merely being the default frontend) will do D a lot of good. For one thing, IDE's will no longer need to reinvent a D parser for the purposes of syntax highlighting; they can reuse the actual parser the compiler itself uses, and thus actually be correct (whereas their own parser may actually only parse a subset of D correctly). They will also be able to support instant feedback as the user types in D code (highlight syntax errors on the fly, etc.). This will also help with writing various tools that operate on D code, e.g., pretty-printers like dfmt won't need to roll its own lexer/parser anymore. Many other tools also become possible (automatic refactorings, function call tree analysis, dependency analysis, etc.). You could even use libdmd in your own D code, thus being able to have the equivalent of the `eval` available in many scripting languages (except that it will have native performance(!)). The problem of dmd refactorings becoming public breaking changes is not as big a problem if the public API is properly designed. If the compiler code itself is refactored to interface between its sub-components via cleanly-designed APIs with proper encapsulations, I don't see that it will become prohibitively difficult to minimize public API changes without hampering compiler development efforts. This is the way to go for better compiler code quality anyway, so even if libdmd never happens, the code should *still* be refactored in this way. Besides, I don't think anyone is sanely suggesting that we allow dmd source files to be free-for-all externally-importable modules. The more likely scenario is that there will be a clearly-defined external interface that is unlikely to change much, whereas the implementation underneath is still free to be changed, refactored, reorganized, rewritten, etc..
While this would all be nice in principle, the reality is that dmd is mostly worked on by two people these days (https://github.com/D-Programming-Language/dmd/graphs/contributors) and they're unlikely to refactor ddmd to put forth a libdmd. It will depend on someone caring enough to take it up, as Daniel did with the port of the frontend to D. As tsbockman said, I don't think we're at the stage where anybody will put in that much effort into this.
Jan 16
parent Daniel Murphy <yebbliesnospam gmail.com> writes:
On 17/01/2016 2:29 AM, Joakim wrote:
 While this would all be nice in principle, the reality is that dmd is
 mostly worked on by two people these days
 (https://github.com/D-Programming-Language/dmd/graphs/contributors) and
 they're unlikely to refactor ddmd to put forth a libdmd.  It will depend
 on someone caring enough to take it up, as Daniel did with the port of
 the frontend to D.  As tsbockman said, I don't think we're at the stage
 where anybody will put in that much effort into this.
I've already spent a huge amount of time on refactoring the frontend, there's just so much more to do. More important than allowing new third-party uses of the frontend, is that well defined interfaces will make maintaining GDC and LDC less tricky. Having to maintain a stable API would probably hurt more than it helps at this point.
Jan 16
prev sibling parent tsbockman <thomas.bockman gmail.com> writes:
On Saturday, 16 January 2016 at 15:13:28 UTC, H. S. Teoh wrote:
 I disagree. I think having the dmd itself (lexer, parser, etc.) 
 as a
 library (with the dmd executable merely being the default 
 frontend) will
 do D a lot of good.

 For one thing, IDE's will no longer need to reinvent a D parser 
 for the purposes of syntax highlighting; they can reuse the 
 actual parser the compiler itself uses, and thus actually be 
 correct (whereas their own parser may actually only parse a 
 subset of D correctly).  They will also be able to support 
 instant feedback as the user types in D code (highlight syntax 
 errors on the fly, etc.).

 This will also help with writing various tools that operate on 
 D code, e.g., pretty-printers like dfmt won't need to roll its 
 own lexer/parser anymore. Many other tools also become possible 
 (automatic refactorings, function call tree analysis, 
 dependency analysis, etc.).

 You could even use libdmd in your own D code, thus being able 
 to have
 the equivalent of the `eval` available in many scripting 
 languages
 (except that it will have native performance(!)).

 The problem of dmd refactorings becoming public breaking 
 changes is not as big a problem if the public API is properly 
 designed. If the compiler code itself is refactored to 
 interface between its sub-components via cleanly-designed APIs 
 with proper encapsulations, I don't see that it will become 
 prohibitively difficult to minimize public API changes without 
 hampering compiler development efforts. This is the way to go 
 for better compiler code quality anyway, so even if libdmd 
 never happens, the code should *still* be refactored in this 
 way.

 Besides, I don't think anyone is sanely suggesting that we 
 allow dmd source files to be free-for-all externally-importable 
 modules. The more likely scenario is that there will be a 
 clearly-defined external interface that is unlikely to change 
 much, whereas the implementation underneath is still free to be 
 changed, refactored, reorganized, rewritten, etc..


 T
You're sort of making my point for me. I fully agree about all the benefits of making the lexer, parser, etc. available as a library. HOWEVER, as you say, this would first require a large refactoring of the code, to make it suitable for use as a library. Time spent on that refactoring is time that could have been spent improving the compiler in other ways. D is short of development resources as it is; I think fixing the many bugs in the compiler and fleshing out important complex features like C++ inter-op is of more value to the language in the long run. Also worth mentioning, is that the SDC project was actually designed from the beginning for the use you're advocating. I'm sure deadalnix would love to have more help with that...
Jan 16