www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - dmdtags 1.0.0: an accurate tag generator for D source code

reply Paul Backus <snarwin gmail.com> writes:
`dmdtags` is a tags file generator for D source code that uses 
the DMD compiler frontend for accurate parsing.

This release supports 100%-accurate parsing of arbitrary D code 
(tested on DMD and Phobos sources), as well as the most 
commonly-used command line options, `-R`, `-o`, and `-a`. The 
generated tags file has been tested for compatibility with Vim 
and is compliant with the [POSIX standard for `ctags`][posix], so 
any editor with `ctags` support should be able to use it.

[posix]: 
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/ctags.html



A _tags file_ is a lightweight plain-text index of the symbols in 
a project. Editors that support tags files, such as Vim and 
Emacs, can use this index to help with things like project 
navigation and tab completion.

A _tags file generator_ is, as you might expect, a program that 
reads source code and generates a tags file with entries for the 
symbols in that code.



[`universal-ctags`][uctags], the current most-popular and 
best-maintained tags file generator, claims support for many 
programming languages, including D. However, its D parser is not 
well-maintained, and it often excludes large numbers of symbols 
from its output due to parsing failures.

Because `dmdtags` uses the DMD frontend for parsing, its results 
will always be accurate and up-to-date. For pure D projects, it 
can be used as a replacement for `universal-ctags`. For 
mixed-language projects, it can be used together with other tag 
generators with the `--append` option.

[uctags]: https://ctags.io



* On Github: https://github.com/pbackus/dmdtags
* On Dub: https://code.dlang.org/packages/dmdtags
Aug 27
next sibling parent reply Dennis <dkorpel gmail.com> writes:
On Friday, 27 August 2021 at 21:38:58 UTC, Paul Backus wrote:
 Editors that support tags files, such as Vim and Emacs, can use 
 this index to help with things like project navigation and tab 
 completion.
Cool! Now I'll have to look if I can make this work with Visual Studio Code, since code-d which uses Dsymbol tends to be unreliable, and this looks ideal for Phobos / Druntime symbol completion.
Aug 27
parent reply WebFreak001 <d.forum webfreak.org> writes:
On Friday, 27 August 2021 at 22:01:59 UTC, Dennis wrote:
 On Friday, 27 August 2021 at 21:38:58 UTC, Paul Backus wrote:
 Editors that support tags files, such as Vim and Emacs, can 
 use this index to help with things like project navigation and 
 tab completion.
Cool! Now I'll have to look if I can make this work with Visual Studio Code, since code-d which uses Dsymbol tends to be unreliable, and this looks ideal for Phobos / Druntime symbol completion.
actually it uses D-Scanner and should be a fairly easy drop-in replacement. I'm just worried about how the memory usage will grow with this, considering dmd never frees. Maybe I should make it run as external tool instead of a library so the OS cleans up, but for that get a performance penalty especially on Windows.
Aug 27
next sibling parent Paul Backus <snarwin gmail.com> writes:
On Friday, 27 August 2021 at 22:45:15 UTC, WebFreak001 wrote:
 I'm just worried about how the memory usage will grow with 
 this, considering dmd never frees. Maybe I should make it run 
 as external tool instead of a library so the OS cleans up, but 
 for that get a performance penalty especially on Windows.
`dmdtags` does not run in the background, so any memory it uses will be freed at process exit, as soon as it has finished writing the tags file. And because it does not do any semantic analysis, only parsing, it does not use much memory while running in the first place. According to `/usr/bin/time`, on my personal machine, using `dmdtags` to generate a tags file for all of Phobos takes slightly less than 1 second and has a peak resident set size of around 100MB.
Aug 27
prev sibling parent evilrat <evilrat666 gmail.com> writes:
On Friday, 27 August 2021 at 22:45:15 UTC, WebFreak001 wrote:
 I'm just worried about how the memory usage will grow with 
 this, considering dmd never frees.
Still not sure if it *should* be used in same process, but DMD definitely has -lowmem switch for turning on GC, and for DMD as a library there is initDMD() and deinitializeDMD() pair. Haven't tested it myself wrt. to memory usage, but it is used by DMD tests. Another concern would be code analysis time, even with skipping semantic passes every run is a new invocation - DMD simply wasn't designed to be a language server where it could just iteratively update compilation database. IIRC VisualD with DMD frontend enabled has noticable long pauses on showing completion (from 100ms to few seconds on larger projects), which renders it unusable for me. Could be that same reason, no idea.
Aug 27
prev sibling next sibling parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 8/27/21 2:38 PM, Paul Backus wrote:

 `dmdtags` is a tags file generator for D source code that uses the DMD
 compiler frontend for accurate parsing.
Thanks! I had stopped using tags files with Emacs for the reasons you give. With this, I will be more efficient. :) And, I will be a happy puppy once Vladimir Panteleev finishes his D mode for Emacs, which he is writing from scratch. Ali
Aug 27
prev sibling next sibling parent =?UTF-8?B?0JLQuNGC0LDQu9C40Lkg0KTQsNC0?= =?UTF-8?B?0LXQtdCy?= writes:
On Friday, 27 August 2021 at 21:38:58 UTC, Paul Backus wrote:
 `dmdtags` is a tags file generator for D source code that uses 
 the DMD compiler frontend for accurate parsing.

 This release supports 100%-accurate parsing of arbitrary D code 
 (tested on DMD and Phobos sources), as well as the most 
 commonly-used command line options, `-R`, `-o`, and `-a`. The 
 generated tags file has been tested for compatibility with Vim 
 and is compliant with the [POSIX standard for `ctags`][posix], 
 so any editor with `ctags` support should be able to use it.

 [posix]: 
 https://pubs.opengroup.org/onlinepubs/9699919799/utilities/ctags.html



 A _tags file_ is a lightweight plain-text index of the symbols 
 in a project. Editors that support tags files, such as Vim and 
 Emacs, can use this index to help with things like project 
 navigation and tab completion.

 A _tags file generator_ is, as you might expect, a program that 
 reads source code and generates a tags file with entries for 
 the symbols in that code.



 [`universal-ctags`][uctags], the current most-popular and 
 best-maintained tags file generator, claims support for many 
 programming languages, including D. However, its D parser is 
 not well-maintained, and it often excludes large numbers of 
 symbols from its output due to parsing failures.

 Because `dmdtags` uses the DMD frontend for parsing, its 
 results will always be accurate and up-to-date. For pure D 
 projects, it can be used as a replacement for 
 `universal-ctags`. For mixed-language projects, it can be used 
 together with other tag generators with the `--append` option.

 [uctags]: https://ctags.io



 * On Github: https://github.com/pbackus/dmdtags
 * On Dub: https://code.dlang.org/packages/dmdtags
``` !_TAG_FILE_SORTED 1 /0=unsorted, 1=sorted, 2=foldcase/ App ./source/ui/app.d 13;" T App ./source/ui/app.d 15;" a App ./source/ui/app.d 19;" T App ./source/ui/app.d 21;" s __ctor ./source/ui/app.d 27;" f __ctor ./source/ui/app.d 30;" f _result ./source/ui/app.d 23;" v app ./source/ui/app.d 1;" M defaultEventLoop ./source/ui/app.d 48;" f defaultInitFunc ./source/ui/app.d 62;" T defaultInitFunc ./source/ui/app.d 62;" f window ./source/ui/app.d 26;" v ``` Perfecto! i can use it in Sublime Text editor... Need to thinks, how it will be implemented...
Aug 27
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Aug 27, 2021 at 09:38:58PM +0000, Paul Backus via
Digitalmars-d-announce wrote:
 `dmdtags` is a tags file generator for D source code that uses the DMD
 compiler frontend for accurate parsing.
 
 This release supports 100%-accurate parsing of arbitrary D code
 (tested on DMD and Phobos sources), as well as the most commonly-used
 command line options, `-R`, `-o`, and `-a`. The generated tags file
 has been tested for compatibility with Vim and is compliant with the
 [POSIX standard for `ctags`][posix], so any editor with `ctags`
 support should be able to use it.
This is AWESOME!!! Thanks a ton for this... I'll definitely be using this in the near future! [...]
 [`universal-ctags`][uctags], the current most-popular and
 best-maintained tags file generator, claims support for many
 programming languages, including D. However, its D parser is not
 well-maintained, and it often excludes large numbers of symbols from
 its output due to parsing failures.
 
 Because `dmdtags` uses the DMD frontend for parsing, its results will
 always be accurate and up-to-date. For pure D projects, it can be used
 as a replacement for `universal-ctags`. For mixed-language projects,
 it can be used together with other tag generators with the `--append`
 option.
Is there any hope of merging this back to upstream ctags? Regardless, this is awesome. T -- Long, long ago, the ancient Chinese invented a device that lets them see through walls. It was called the "window".
Aug 31
prev sibling parent Chris Piker <chris hoopjump.com> writes:
On Friday, 27 August 2021 at 21:38:58 UTC, Paul Backus wrote:
 `dmdtags` is a tags file generator for D source code that uses 
 the DMD compiler frontend for accurate parsing.
...

 * On Github: https://github.com/pbackus/dmdtags
 * On Dub: https://code.dlang.org/packages/dmdtags
Wow, what perfect timing! I just switched sublime text only 4 days ago. Time to read some docs and figure out how to set this up. Thanks a lot!
Sep 03