www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Is anyone working on a D source code formatting tool?

reply Walter Bright <newshound2 digitalmars.com> writes:
Has someone made a dfmt, like http://gofmt.com/ ?
Jan 10 2015
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-01-10 21:17, Walter Bright wrote:
 Has someone made a dfmt, like http://gofmt.com/ ?
I have thought about it a couple of times but never started. It would be really nice to have. -- /Jacob Carlborg
Jan 10 2015
parent reply "Meta" <jared771 gmail.com> writes:
On Saturday, 10 January 2015 at 20:54:56 UTC, Jacob Carlborg 
wrote:
 On 2015-01-10 21:17, Walter Bright wrote:
 Has someone made a dfmt, like http://gofmt.com/ ?
I have thought about it a couple of times but never started. It would be really nice to have.
https://github.com/Hackerpilot/dfix
Jan 10 2015
parent reply "Meta" <jared771 gmail.com> writes:
On Saturday, 10 January 2015 at 21:17:29 UTC, Meta wrote:
 On Saturday, 10 January 2015 at 20:54:56 UTC, Jacob Carlborg 
 wrote:
 On 2015-01-10 21:17, Walter Bright wrote:
 Has someone made a dfmt, like http://gofmt.com/ ?
I have thought about it a couple of times but never started. It would be really nice to have.
https://github.com/Hackerpilot/dfix
Ah, never mind me. Although the functionality between dfix and a dfmt is probably quite similar.
Jan 10 2015
parent "Brian Schott" <briancschott gmail.com> writes:
On Saturday, 10 January 2015 at 21:18:58 UTC, Meta wrote:
 https://github.com/Hackerpilot/dfix
Ah, never mind me. Although the functionality between dfix and a dfmt is probably quite similar.
dfix is more complicated than it could be because it tries to avoid changing the formatting of the file that it's operating on.
Jan 10 2015
prev sibling next sibling parent reply "weaselcat" <weaselcat gmail.com> writes:
On Saturday, 10 January 2015 at 20:18:03 UTC, Walter Bright wrote:
 Has someone made a dfmt, like http://gofmt.com/ ?
Uncrustify claims D support. http://uncrustify.sourceforge.net/
Jan 10 2015
parent Andrej Mitrovic via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 1/10/15, weaselcat via Digitalmars-d <digitalmars-d puremagic.com> wrote:
 On Saturday, 10 January 2015 at 20:18:03 UTC, Walter Bright wrote:
 Has someone made a dfmt, like http://gofmt.com/ ?
Uncrustify claims D support. http://uncrustify.sourceforge.net/
Yes, and the author is responsive whenever a new D-related bug is filed. I've used Uncrustify successfully with D for many years. It would be nice to have an implementation in D (purely to be able to more easily hack on the tool and configure it to our own liking), but as far as having a tool that works right now - uncrustify is exactly that tool.
Jan 11 2015
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/10/2015 12:17 PM, Walter Bright wrote:
 Has someone made a dfmt, like http://gofmt.com/ ?
Next question - standalone tool, or built in to dmd (like Ddoc)? BTW, I think dfmt would be a significant win for D: 1. people expect this sort of thing these days 2. it tends to end bikeshedding arguments about the right way to format things 3. it'll help standardize the format of D code in the D repositories 4. it's simply nice and convenient! 5. it's a great first step when you're faced with fixing someone else's crap code I don't think it'll be hard to do as a builtin feature of dmd. My only concern about it is if dfmt is changed, then we get faced with a blizzard of changes in the D github repositories.
Jan 10 2015
next sibling parent "Tobias Pankrath" <tobias pankrath.net> writes:
On Saturday, 10 January 2015 at 22:11:55 UTC, Walter Bright wrote:
 On 1/10/2015 12:17 PM, Walter Bright wrote:
 Has someone made a dfmt, like http://gofmt.com/ ?
Next question - standalone tool, or built in to dmd (like Ddoc)? My only concern about it is if dfmt is changed, then we get faced with a blizzard of changes in the D github repositories.
Build it into DMD with an option to change files in place or dump them to stdout. I do think though, this would require a pragma or a special comment to disable it for parts of code. There will always be the case where the standard is wrong.
Jan 10 2015
prev sibling next sibling parent reply "Joakim" <dlang joakim.fea.st> writes:
On Saturday, 10 January 2015 at 22:11:55 UTC, Walter Bright wrote:
 On 1/10/2015 12:17 PM, Walter Bright wrote:
 Has someone made a dfmt, like http://gofmt.com/ ?
Next question - standalone tool, or built in to dmd (like Ddoc)? BTW, I think dfmt would be a significant win for D: 1. people expect this sort of thing these days 2. it tends to end bikeshedding arguments about the right way to format things 3. it'll help standardize the format of D code in the D repositories 4. it's simply nice and convenient! 5. it's a great first step when you're faced with fixing someone else's crap code I don't think it'll be hard to do as a builtin feature of dmd. My only concern about it is if dfmt is changed, then we get faced with a blizzard of changes in the D github repositories.
I agree that there should be a tool like this for D, don't care if it's stand-alone or part of dmd. In my experience submitting PRs on github, 2. and 3. would definitely be alleviated by dfmt, as there's sometimes review comments about formatting and they'd go away if you just told all contributors to run dfmt first, to get the patches in a standard format. As for your concern about github formatting commits, inevitable cost of standardizing and then changing your mind.
Jan 10 2015
parent "Suliman" <evermind live.ru> writes:
Can this problem be solve on Text Editor level?

I use Sublime, and I need some tool for code formating. Maybe 
there is any ready to use addons for D?
Jan 10 2015
prev sibling next sibling parent "ponce" <contact gam3sfrommars.fr> writes:
On Saturday, 10 January 2015 at 22:11:55 UTC, Walter Bright wrote:
 On 1/10/2015 12:17 PM, Walter Bright wrote:
 Has someone made a dfmt, like http://gofmt.com/ ?
Next question - standalone tool, or built in to dmd (like Ddoc)? BTW, I think dfmt would be a significant win for D: 1. people expect this sort of thing these days 2. it tends to end bikeshedding arguments about the right way to format things 3. it'll help standardize the format of D code in the D repositories 4. it's simply nice and convenient! 5. it's a great first step when you're faced with fixing someone else's crap code I don't think it'll be hard to do as a builtin feature of dmd. My only concern about it is if dfmt is changed, then we get faced with a blizzard of changes in the D github repositories.
Since Brian Schott already wrote a dfix, it seems easier to make dfmt the same program.
Jan 11 2015
prev sibling next sibling parent reply "NVolcz" <niklas.volcz gmail.com> writes:
On Saturday, 10 January 2015 at 22:11:55 UTC, Walter Bright wrote:
 On 1/10/2015 12:17 PM, Walter Bright wrote:
 Has someone made a dfmt, like http://gofmt.com/ ?
Next question - standalone tool, or built in to dmd (like Ddoc)? BTW, I think dfmt would be a significant win for D: 1. people expect this sort of thing these days 2. it tends to end bikeshedding arguments about the right way to format things 3. it'll help standardize the format of D code in the D repositories 4. it's simply nice and convenient! 5. it's a great first step when you're faced with fixing someone else's crap code I don't think it'll be hard to do as a builtin feature of dmd. My only concern about it is if dfmt is changed, then we get faced with a blizzard of changes in the D github repositories.
I guess dfmt is to small for GSOC but could we bundle it with some other important tool to create a bigger task? Best regards, NVolcz
Jan 11 2015
parent Walter Bright <newshound2 digitalmars.com> writes:
On 1/11/2015 2:00 AM, NVolcz wrote:
 I guess dfmt is to small for GSOC
I'm not so sure. If the code has no comments, it's trivial. Handling comments, though, can be a bit of a challenge.
Jan 11 2015
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-01-10 23:11, Walter Bright wrote:

 Next question - standalone tool, or built in to dmd (like Ddoc)?
Ideally the dmd front end should be available as a library then dfmt should be a separate tool that uses the same front end library as dmd. But I know that we're not there yet. It's always nice if a tool is built as a library with a with executable on top. This allows for other tools to be built using this library. -- /Jacob Carlborg
Jan 11 2015
next sibling parent Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sun, 2015-01-11 at 11:04 +0100, Jacob Carlborg via Digitalmars-d wrote:
 On 2015-01-10 23:11, Walter Bright wrote:
 
 Next question - standalone tool, or built in to dmd (like Ddoc)?
Ideally the dmd front end should be available as a library then dfmt should be a separate tool that uses the same front end library as dmd. But I know that we're not there yet.
And written in D :-)
 It's always nice if a tool is built as a library with a with 
 executable on top. This allows for other tools to be built using 
 this library.
dmd, ldc, gdc, dfmt, ddoc, lots of programs one parsing library required. -- Russel. ============================================================================= Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.net 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Jan 11 2015
prev sibling parent reply "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Jacob Carlborg"  wrote in message news:m8thr2$pag$1 digitalmars.com...

 Ideally the dmd front end should be available as a library then dfmt 
 should be a separate tool that uses the same front end library as dmd. But 
 I know that we're not there yet.
Well, some of it is. If somebody wants to work on adding the necessary lexer/parser/etc features on the C++ side I'm happy to help get ddmd ready to be used for this.
 It's always nice if a tool is built as a library with a with executable on 
 top. This allows for other tools to be built using this library.
Agreed.
Jan 11 2015
parent reply "Stefan Koch" <uplink.coder googlemail.com> writes:
I'm  powerful writing a parser-generator, that will be able to 
transform the generated parse-tree back into source automatically.
writing a rule-based formatter should be pretty doable.
Jan 11 2015
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/11/2015 9:45 AM, Stefan Koch wrote:
 I'm  powerful writing a parser-generator, that will be able to transform the
 generated parse-tree back into source automatically.
 writing a rule-based formatter should be pretty doable.
Formatting the AST into text is straightforward, dmd already does that for .di file generation. The main problem is what to do about comments, which don't fit into the grammar. A secondary problem is what to do when the line length limit is exceeded, such as for long expressions.
Jan 11 2015
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-01-11 19:48, Walter Bright wrote:

 The main problem is what to do about comments, which don't fit into the
 grammar.

 A secondary problem is what to do when the line length limit is
 exceeded, such as for long expressions.
clang-format seems to do a pretty good job with both of these. Comments seem to be intact unless they're too long, then they're wrapped. It seems to wrap at a space or other non-identifier character. Same thing with expressions that are too long. You should download it [1] a give it a try on some C++ code. [1] http://llvm.org/releases/download.html -- /Jacob Carlborg
Jan 11 2015
parent "Ahmed Fasih" <wuzzyview gmail.com> writes:
Hi there, I'm a C++/Python refugee, new to D.

 clang-format seems to do a pretty good job with both of these. 
 Comments seem to be intact unless they're too long, then 
 they're wrapped. It seems to wrap at a space or other 
 non-identifier character. Same thing with expressions that are 
 too long.
I would love such a tool for D, especially based on the ideas of clang-format. I first heard about clang-format from a talk by Google's Chandler Carruth: the way he said it, the LLVM guys looked at what their C++ programmers wasted the most time on, and it turned out to be whitespace, surprisingly enough. So they implemented was is essentially LaTeX for source code (optimal placement of spaces and line breaks using Djikstra's algorithm, the works). And he said it changed the way he codes, and at that point I had to go get it, and I 100% agree with him: it's awesome to have. It works with Vim, Emacs, Visual Studio, Sublime, etc., because it provides a simple Python wrapper to the executable that anything can call any time to format any code. It works with C/C++, Objective-C, and Javascript; when I discovered it could do JavaScript, my use of that language rose substantially (could be coincidence :). Reading some of the discussion here about whether to integrate it into the compiler, etc., makes me realize that one of the nice things about clang-format is real-time interactivity, i.e., I can type-type-type code quickly and without bothering with whitespace, just getting the idea out of my head and into the editor, then when I reach a breathing space I can hit a keycombo, and my editor reformats the block I just typed. I find that the hits to my flow are much smaller than having to do this manually, and I think that's what Carruth meant when he said it changed his coding. Another nice thing about it is that it's fully parameterized (how many spaces, whether to indent this, what penalty to assign that, etc., example at [1]). It can search the current and parent directories for a .clang-format file with those parameters. It can also generate stock .clang-format files conforming to various coding styles, viz., LLVM, Google, Mozilla, WebKit, & Chromium. I throw this into my Git repos; happiness ensues. Hope this helps outline the paths others have taken! [1] Example Javascript .clang-format file: https://github.com/fasiha/kanjiwild/blob/gh-pages/.clang-format
Jan 16 2015
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/11/15 10:48 AM, Walter Bright wrote:
 On 1/11/2015 9:45 AM, Stefan Koch wrote:
 I'm  powerful writing a parser-generator, that will be able to
 transform the
 generated parse-tree back into source automatically.
 writing a rule-based formatter should be pretty doable.
Formatting the AST into text is straightforward, dmd already does that for .di file generation. The main problem is what to do about comments, which don't fit into the grammar.
In the first version comments might go through unchanged.
 A secondary problem is what to do when the line length limit is
 exceeded, such as for long expressions.
I think that's problem #1. Andrei
Jan 11 2015
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/11/2015 11:50 AM, Andrei Alexandrescu wrote:
 On 1/11/15 10:48 AM, Walter Bright wrote:
 The main problem is what to do about comments, which don't fit into the
 grammar.
In the first version comments might go through unchanged.
Consider: for /*comment*/ (a; b; c) Do what with that?
Jan 11 2015
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/11/15 1:15 PM, Walter Bright wrote:
 On 1/11/2015 11:50 AM, Andrei Alexandrescu wrote:
 On 1/11/15 10:48 AM, Walter Bright wrote:
 The main problem is what to do about comments, which don't fit into the
 grammar.
In the first version comments might go through unchanged.
Consider: for /*comment*/ (a; b; c) Do what with that?
I don't know. Simplest would be to punt for now - such rare embedded comments should not be blockers. -- Andrei
Jan 11 2015
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/11/2015 1:31 PM, Andrei Alexandrescu wrote:
 On 1/11/15 1:15 PM, Walter Bright wrote:
 On 1/11/2015 11:50 AM, Andrei Alexandrescu wrote:
 On 1/11/15 10:48 AM, Walter Bright wrote:
 The main problem is what to do about comments, which don't fit into the
 grammar.
In the first version comments might go through unchanged.
Consider: for /*comment*/ (a; b; c) Do what with that?
I don't know. Simplest would be to punt for now - such rare embedded comments should not be blockers. -- Andrei
Normally I would agree, but deleting peoples' comments from their source code is not a good plan. It'll make them justifiably angry.
Jan 11 2015
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/11/15 4:37 PM, Walter Bright wrote:
 On 1/11/2015 1:31 PM, Andrei Alexandrescu wrote:
 On 1/11/15 1:15 PM, Walter Bright wrote:
 On 1/11/2015 11:50 AM, Andrei Alexandrescu wrote:
 On 1/11/15 10:48 AM, Walter Bright wrote:
 The main problem is what to do about comments, which don't fit into
 the
 grammar.
In the first version comments might go through unchanged.
Consider: for /*comment*/ (a; b; c) Do what with that?
I don't know. Simplest would be to punt for now - such rare embedded comments should not be blockers. -- Andrei
Normally I would agree, but deleting peoples' comments from their source code is not a good plan. It'll make them justifiably angry.
By punt i mean leave as is. -- Andrei
Jan 11 2015
prev sibling parent reply "Zach the Mystic" <reachzach gggmail.com> writes:
On Monday, 12 January 2015 at 00:38:20 UTC, Walter Bright wrote:
 On 1/11/15 10:48 AM, Walter Bright wrote:
 The main problem is what to do about comments, which don't 
 fit into the
 grammar.
In the first version comments might go through unchanged.
Consider: for /*comment*/ (a; b; c) Do what with that?
I don't know. Simplest would be to punt for now - such rare embedded comments should not be blockers. -- Andrei
Normally I would agree, but deleting peoples' comments from their source code is not a good plan. It'll make them justifiably angry.
This conversation reminds me of something I've thought about ever since I first studied D. D takes the C preprocessor and folds it into the regular AST. But comments still seemed like the outlier. I looked through a bunch of source code and tried to figure out the most specific place anyone could possibly put a comment. The most detailed I found were something like: enum X { ONE, TWO, // We need a TWO here THREE } So I started conceiving of a language in which even the *comments* were part of the AST. For, me this would be the aesthetic ideal. It just seemed like the next step in total AST integration.
Jan 11 2015
next sibling parent reply "qznc" <qznc web.de> writes:
On Monday, 12 January 2015 at 05:43:33 UTC, Zach the Mystic wrote:
 On Monday, 12 January 2015 at 00:38:20 UTC, Walter Bright wrote:
 On 1/11/15 10:48 AM, Walter Bright wrote:
 The main problem is what to do about comments, which don't 
 fit into the
 grammar.
In the first version comments might go through unchanged.
Consider: for /*comment*/ (a; b; c) Do what with that?
I don't know. Simplest would be to punt for now - such rare embedded comments should not be blockers. -- Andrei
Normally I would agree, but deleting peoples' comments from their source code is not a good plan. It'll make them justifiably angry.
This conversation reminds me of something I've thought about ever since I first studied D. D takes the C preprocessor and folds it into the regular AST. But comments still seemed like the outlier. I looked through a bunch of source code and tried to figure out the most specific place anyone could possibly put a comment. The most detailed I found were something like: enum X { ONE, TWO, // We need a TWO here THREE } So I started conceiving of a language in which even the *comments* were part of the AST. For, me this would be the aesthetic ideal. It just seemed like the next step in total AST integration.
The clang-format approach is to make decisions based on the AST, but edit the byte array. AST nodes contain exact positions: line and column numbers. Sometimes more of them. For example, an if-statement needs to know the position of 'else' as well. Here is another example: void setX(int position /* inches */); However, I think it should really be wrapped into a struct instead to get type checker's approval. Initially, clang-format only did whitespace changes. I am not sure if they weakened this already. For example, stripping parens in expressions (((((x)))) => x) would be acceptable for me. btw +1 for https://github.com/Hackerpilot/dfmt
Jan 12 2015
parent "Brian Schott" <briancschott gmail.com> writes:
On Monday, 12 January 2015 at 15:30:34 UTC, qznc wrote:
 The clang-format approach is to make decisions based on the 
 AST, but edit the byte array.
dfix uses a similar approach. It uses the AST location information to make decisions while iterating through the token array. I think I will end up doing the same thing with dfmt.
Jan 12 2015
prev sibling parent reply "qznc" <qznc web.de> writes:
On Monday, 12 January 2015 at 05:43:33 UTC, Zach the Mystic wrote:
 On Monday, 12 January 2015 at 00:38:20 UTC, Walter Bright wrote:
 On 1/11/15 10:48 AM, Walter Bright wrote:
 The main problem is what to do about comments, which don't 
 fit into the
 grammar.
In the first version comments might go through unchanged.
Consider: for /*comment*/ (a; b; c) Do what with that?
I don't know. Simplest would be to punt for now - such rare embedded comments should not be blockers. -- Andrei
Normally I would agree, but deleting peoples' comments from their source code is not a good plan. It'll make them justifiably angry.
This conversation reminds me of something I've thought about ever since I first studied D. D takes the C preprocessor and folds it into the regular AST. But comments still seemed like the outlier. I looked through a bunch of source code and tried to figure out the most specific place anyone could possibly put a comment. The most detailed I found were something like: enum X { ONE, TWO, // We need a TWO here THREE } So I started conceiving of a language in which even the *comments* were part of the AST. For, me this would be the aesthetic ideal. It just seemed like the next step in total AST integration.
The clang-format approach is to make decisions based on the AST, but edit the byte array. AST nodes contain exact positions: line and column numbers. Sometimes more of them. For example, an if-statement needs to know the position of 'else' as well. Here is another example: void setX(int position /* inches */); However, I think it should really be wrapped into a struct instead to get type checker's approval. Initially, clang-format only did whitespace changes. I am not sure if they weakened this already. For example, stripping parens in expressions (((((x)))) => x) would be acceptable for me. btw +1 for https://github.com/Hackerpilot/dfmt
Jan 12 2015
parent "deadalnix" <deadalnix gmail.com> writes:
On Monday, 12 January 2015 at 15:30:42 UTC, qznc wrote:
 So I started conceiving of a language in which even the 
 *comments* were part of the AST. For, me this would be the 
 aesthetic ideal. It just seemed like the next step in total 
 AST integration.
The clang-format approach is to make decisions based on the AST, but edit the byte array. AST nodes contain exact positions: line and column numbers. Sometimes more of them. For example, an if-statement needs to know the position of 'else' as well. Here is another example: void setX(int position /* inches */); However, I think it should really be wrapped into a struct instead to get type checker's approval. Initially, clang-format only did whitespace changes. I am not sure if they weakened this already. For example, stripping parens in expressions (((((x)))) => x) would be acceptable for me. btw +1 for https://github.com/Hackerpilot/dfmt
Last time I checked, it work on the lexer rather than the AST.
Jan 12 2015
prev sibling next sibling parent "weaselcat" <weaselcat gmail.com> writes:
On Sunday, 11 January 2015 at 21:16:41 UTC, Walter Bright wrote:
 On 1/11/2015 11:50 AM, Andrei Alexandrescu wrote:
 On 1/11/15 10:48 AM, Walter Bright wrote:
 The main problem is what to do about comments, which don't 
 fit into the
 grammar.
In the first version comments might go through unchanged.
Consider: for /*comment*/ (a; b; c) Do what with that?
Why not just move the comment to the end of the expression?
Jan 11 2015
prev sibling parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sun, 11 Jan 2015 13:15:22 -0800
Walter Bright via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 On 1/11/2015 11:50 AM, Andrei Alexandrescu wrote:
 On 1/11/15 10:48 AM, Walter Bright wrote:
 The main problem is what to do about comments, which don't fit into the
 grammar.
In the first version comments might go through unchanged.
=20 Consider: =20 for /*comment*/ (a; b; c) =20 Do what with that? =20
it's easy: put it before `for`. /*comment*/ for (...) or just ignore it. and i must confess that i've never seen comment like this in my lifetime.
Jan 12 2015
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2015-01-12 09:11, ketmar via Digitalmars-d wrote:

 it's easy: put it before `for`.

    /*comment*/
    for (...)

 or just ignore it. and i must confess that i've never seen comment like
 this in my lifetime.
I agree. clang-format manage to keep it in the same place. -- /Jacob Carlborg
Jan 12 2015
prev sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Monday, 12 January 2015 at 08:11:27 UTC, ketmar via 
Digitalmars-d wrote:
 it's easy: put it before `for`.

   /*comment*/
   for (...)

 or just ignore it. and i must confess that i've never seen 
 comment like
 this in my lifetime.
You can't ignore. That is why building such tool in not that easy.
Jan 12 2015
parent ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Mon, 12 Jan 2015 19:17:47 +0000
deadalnix via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 On Monday, 12 January 2015 at 08:11:27 UTC, ketmar via=20
 Digitalmars-d wrote:
 it's easy: put it before `for`.

   /*comment*/
   for (...)

 or just ignore it. and i must confess that i've never seen=20
 comment like
 this in my lifetime.
=20 You can't ignore. That is why building such tool in not that easy.
by "ignore" i meant "live it where it was". if somebody is so smart that he writes such comments... oh, well, then he is smart enough to reformat the source manually. and if this is some other person's source... i'd say "don't use it, that person is sick".
Jan 12 2015
prev sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 11 January 2015 at 19:50:51 UTC, Andrei Alexandrescu 
wrote:
 A secondary problem is what to do when the line length limit is
 exceeded, such as for long expressions.
I think that's problem #1.
The usual way is to associate cost with various cesures and run something Dijkstra (or A*) like to find the best cesures.
Jan 11 2015
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2015-01-11 19:48, Walter Bright wrote:

 The main problem is what to do about comments, which don't fit into the
 grammar.
Both Clang and Eclipse JDT provide API's for accessing comments. -- /Jacob Carlborg
Jan 12 2015
prev sibling next sibling parent Bruno Medeiros <bruno.do.medeiros+dng gmail.com> writes:
On 11/01/2015 18:48, Walter Bright wrote:
 On 1/11/2015 9:45 AM, Stefan Koch wrote:
 I'm  powerful writing a parser-generator, that will be able to
 transform the
 generated parse-tree back into source automatically.
 writing a rule-based formatter should be pretty doable.
Formatting the AST into text is straightforward, dmd already does that for .di file generation. The main problem is what to do about comments, which don't fit into the grammar.
It's not that hard to do that once you have a parser that preserves the source range of all AST nodes. With that you can write a formatting algorithm than *modifies* the original source, assisted by AST info, instead of trying to recreate the whole source from the AST info alone. This way you can reformat without losing comment data, even if the AST doesn't have store comments. -- Bruno Medeiros https://twitter.com/brunodomedeiros
Jan 13 2015
prev sibling parent reply Ary Borenszweig <ary esperanto.org.ar> writes:
On 1/11/15 3:48 PM, Walter Bright wrote:
 On 1/11/2015 9:45 AM, Stefan Koch wrote:
 I'm  powerful writing a parser-generator, that will be able to
 transform the
 generated parse-tree back into source automatically.
 writing a rule-based formatter should be pretty doable.
Formatting the AST into text is straightforward, dmd already does that for .di file generation. The main problem is what to do about comments, which don't fit into the grammar. A secondary problem is what to do when the line length limit is exceeded, such as for long expressions.
The way I did it in Descent (I copied the logic from JDT) is to parse the code into an AST, and then walk the AST in sync with a lexer. So if you have this: void /* comment /* foo() {} the AST would be a FunctionDecl (whatever the name is) so you'd expect a type (consume that AST node, in sync with the lexer), then check for comments/newlines/etc., skip/print them, then consume the name, check for comments/newlines/etc. That way the AST doesn't have to know anything about comments, but comments need to be known by the lexer (via a flag, probably). Considering how flexible is JDT's formatter, I think this solution is pretty good.
Jan 16 2015
parent reply "Brian Schott" <briancschott gmail.com> writes:
On Friday, 16 January 2015 at 15:06:42 UTC, Ary Borenszweig wrote:
 The way I did it in Descent (I copied the logic from JDT) is to 
 parse the code into an AST, and then walk the AST in sync with 
 a lexer.
My dfmt tool does something similar. The parser runs over the code first and makes notes on things like the location of unary operators and which braces end function/aggregate declarations. Then the formatter iterates over the tokens (including comments) and is able to correctly print "a* b;" instead of "a * b;".
Jan 16 2015
parent "qznc" <qznc web.de> writes:
On Friday, 16 January 2015 at 15:55:53 UTC, Brian Schott wrote:
 On Friday, 16 January 2015 at 15:06:42 UTC, Ary Borenszweig 
 wrote:
 The way I did it in Descent (I copied the logic from JDT) is 
 to parse the code into an AST, and then walk the AST in sync 
 with a lexer.
My dfmt tool does something similar. The parser runs over the code first and makes notes on things like the location of unary operators and which braces end function/aggregate declarations. Then the formatter iterates over the tokens (including comments) and is able to correctly print "a* b;" instead of "a * b;".
This is a short talk about clang-format's design and implementation: https://www.youtube.com/watch?v=s7JmdCfI__c They have this concept of "unwrapped lines" into which the source code is split by a "structural parser". I am not sure, if dfmt really needs it, because we don't have cpp macros littered around in D code. While dfmt already works well for some code (e.g. the snippet on dlang.org frontpage), the biggest hurdle (imho) is an expression formatter, which for example understands operator precedences. Currently, dfmt uses a greedy line fill algorithm.
Jan 16 2015
prev sibling next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Saturday, 10 January 2015 at 22:11:55 UTC, Walter Bright wrote:
 On 1/10/2015 12:17 PM, Walter Bright wrote:
 Has someone made a dfmt, like http://gofmt.com/ ?
Next question - standalone tool, or built in to dmd (like Ddoc)?
Please don't make the compiler-executable modify source code. It makes makefile editing more "dangerous". A standalone tool is appropriate.
Jan 11 2015
prev sibling next sibling parent reply Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sat, 2015-01-10 at 14:11 -0800, Walter Bright via Digitalmars-d wrote:
 On 1/10/2015 12:17 PM, Walter Bright wrote:
 Has someone made a dfmt, like http://gofmt.com/ ?
Next question - standalone tool, or built in to dmd (like Ddoc)?
Why in the compiler, source code formatting is not a compilation issue. Of course the issue is parsing to AST, which the compiler has, but doesn't provide as a library. In Go, the parsing code is separated from the compiler so that different tools can be constructed using different combinations of the same code. Also why in DMD and not in LDC or GDC?
 BTW, I think dfmt would be a significant win for D:
 
 1. people expect this sort of thing these days
Thanks to Python PEP-8 and Go, yes. Personalized code formatting is increasingly a thing of the past, programming languages are now opinionated and fascist when it comes to formatting. Even if the rules are wrong.
 2. it tends to end bikeshedding arguments about the right way to 
 format things
Except when the tool implements the wrong style.
 3. it'll help standardize the format of D code in the D repositories 
Even if it is the wrong formatting standard? I have to admit that the Phobos format rules make the code look appallingly ugly to me, sufficiently so that I really do not want to work on that codebase. I guess I am biased towards K&R C (which I use to drive my C++ style), Java and Go.
 4. it's simply nice and convenient!
Working with Go in Emacs or LiteIDE is a bit of a joy as you get full reformatting on save. Fortunately I only have two main gripes with the Go formatting style, but no-one has a choice, use the Go style as dictated by the Go team at Google or do not use Go.
 5. it's a great first step when you're faced with fixing someone 
 else's crap code
Having just taken on a C/C++ codebase, I can agree that manually reformatting is a great way of "getting in" to the code. Then an automated formatting would contribute as well.
 I don't think it'll be hard to do as a builtin feature of dmd.
It should be a separate tool not a part of one of the three compilers.
 My only concern about it is if dfmt is changed, then we get faced 
 with a
 blizzard of changes in the D github repositories.
Tough ;-) -- Russel. ============================================================================= Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.net 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Jan 11 2015
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/11/2015 4:47 AM, Russel Winder via Digitalmars-d wrote:
 Next question - standalone tool, or built in to dmd (like Ddoc)?
Also why in DMD and not in LDC or GDC?
It would be in the DMD front end, so LDC and GDC would it automatically.
 1. people expect this sort of thing these days
Thanks to Python PEP-8 and Go, yes. Personalized code formatting is increasingly a thing of the past, programming languages are now opinionated and fascist when it comes to formatting. Even if the rules are wrong.
The rules are always going to be wrong. But in this case, that's ok.
 2. it tends to end bikeshedding arguments about the right way to
 format things
Except when the tool implements the wrong style.
Sometimes it's better to just conform.
 3. it'll help standardize the format of D code in the D repositories
Even if it is the wrong formatting standard? I have to admit that the Phobos format rules make the code look appallingly ugly to me, sufficiently so that I really do not want to work on that codebase. I guess I am biased towards K&R C (which I use to drive my C++ style), Java and Go.
Really? Like what? After many years of arguing about formatting myself, I decided that I had better things to waste my time on.
 4. it's simply nice and convenient!
Working with Go in Emacs or LiteIDE is a bit of a joy as you get full reformatting on save. Fortunately I only have two main gripes with the Go formatting style, but no-one has a choice, use the Go style as dictated by the Go team at Google or do not use Go.
So you decided to conform rather than not use Go. (Anyhow, I suggest that gofmt is only mandatory if you wish to contribute to offical Go code, not for your own projects.)
Jan 11 2015
parent Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sun, 2015-01-11 at 11:19 -0800, Walter Bright via Digitalmars-d wrote:
 
[…]
 It would be in the DMD front end, so LDC and GDC would it 
 automatically.
s/it/get it/ ? What wouldn't be automatic would be the command line options and other surrounding bits and pieces.
 
[…]
 The rules are always going to be wrong. But in this case, that's ok.
For someone, not everyone. For some the rules will be right or at least acceptable.
 
[…]
 Sometimes it's better to just conform.
It depends if the grievances are small enough to ignore and/or the benefits outweight. Sometimes conformance is a "bridge too far" and non-use, or use of an alternative is the right course of action.
 
[…]
 Really? Like what? After many years of arguing about formatting 
 myself, I
 decided that I had better things to waste my time on.
I do not see a point in opening up the debate. An official style exists, it is that which is required for Phobos. This is the style that will be implemented by dfmt. […]
 
 So you decided to conform rather than not use Go.
Yes. My grievances against the enforced style are not sufficient to abandon, and the need to deal with error codes at all times is not a sufficient pain to stop me using Go. What is the counterbalance? Go routines, the use of channels, and the eshewing of any form of shared mutable memory. Go has a lot of crap, a lot of positive and negative hype, and dataflow built into the language. I find it surprising easy to ignore the crap.
 (Anyhow, I suggest that gofmt is only mandatory if you wish to 
 contribute to
 offical Go code, not for your own projects.)
I suspect applying gofmt to Phobos code would have interesting results ;-) PEP-8 allows for some minor variations. It can be applied as is for official PEP-8 compliant code or you can have variation. Of course the pep8 code is only an advisor, gofmt rewrites your code. Originally the gc compiler automatically rewrote your code, but that was seen as too fascist, so gofmt was split off as a separate tool. Now though everyone uses gofmt anyway adhering to the "one true style". Having gofmt separate is though still the right thing as there is gc and gccgo which share no code. -- Russel. ============================================================================= Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.net 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Jan 12 2015
prev sibling next sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Saturday, 10 January 2015 at 22:11:55 UTC, Walter Bright wrote:
 On 1/10/2015 12:17 PM, Walter Bright wrote:
 Has someone made a dfmt, like http://gofmt.com/ ?
Next question - standalone tool, or built in to dmd (like Ddoc)?
I think best approach is akin to git subcommands - you can call stuff via `git command` but it in fact runs separate `git-command` binary behind the scene (which can also be used stand-alone). I would love to see DDOC implemented that way too.
Jan 11 2015
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/11/2015 5:11 AM, Dicebot wrote:
 I would love to see DDOC implemented that way too.
Ddoc makes use of semantic info, not just an AST. For semantic info, you pretty much need a real compiler.
Jan 11 2015
parent Jacob Carlborg <doob me.com> writes:
On 2015-01-11 20:20, Walter Bright wrote:

 Ddoc makes use of semantic info, not just an AST. For semantic info, you
 pretty much need a real compiler.
I've been thinking of that the last couple of days. It should be pretty straightforward to copy-paste the driver part of DMD, i.e the part contains the main function and handling of the command line arguments. Then remove everything that's not needed for Ddoc. -- /Jacob Carlborg
Jan 11 2015
prev sibling next sibling parent ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sun, 11 Jan 2015 12:47:41 +0000
Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 I don't think it'll be hard to do as a builtin feature of dmd.
=20 It should be a separate tool not a part of one of the three compilers.=20
i can't see anything wrong with built-in tool. even if it can't be configured to one's tastes, it's still good for occasional contributors, who can stop worrying about code formatting and just write that code. and if it will be configurable (which is not that hard to do -- to some extent), it will be usable for many more people. besides, being part of the compiler this tool has a good chances of being always up-to-date. as there is already .di emitter, i believe that is can be customised further to do full code formatting. the only thing left to solve is comments (not a small one, though). i.e. compiler already has most of the work done, why not reuse it?
Jan 11 2015
prev sibling parent reply Bruno Medeiros <bruno.do.medeiros+dng gmail.com> writes:
On 10/01/2015 22:11, Walter Bright wrote:
 On 1/10/2015 12:17 PM, Walter Bright wrote:

 2. it tends to end bikeshedding arguments about the right way to format
 things
No, it wouldn't. The tool would have to support formatting options, not just one style (or the tool would be crap). And even then, people could argue about what the default formatting should be... -- Bruno Medeiros https://twitter.com/brunodomedeiros
Jan 13 2015
parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, 13 Jan 2015 12:37:22 +0000
Bruno Medeiros via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 On 10/01/2015 22:11, Walter Bright wrote:
 On 1/10/2015 12:17 PM, Walter Bright wrote:

 2. it tends to end bikeshedding arguments about the right way to format
 things
=20 No, it wouldn't. The tool would have to support formatting options, not=20 just one style (or the tool would be crap). And even then, people could=20 argue about what the default formatting should be...
default should be one that used in Phobos.
Jan 13 2015
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 13 January 2015 at 12:43:05 UTC, ketmar via 
Digitalmars-d wrote:
 On Tue, 13 Jan 2015 12:37:22 +0000
 Bruno Medeiros via Digitalmars-d <digitalmars-d puremagic.com> 
 wrote:

 On 10/01/2015 22:11, Walter Bright wrote:
 On 1/10/2015 12:17 PM, Walter Bright wrote:

 2. it tends to end bikeshedding arguments about the right 
 way to format
 things
No, it wouldn't. The tool would have to support formatting options, not just one style (or the tool would be crap). And even then, people could argue about what the default formatting should be...
default should be one that used in Phobos.
phobos is very APIish, most of business logic do not look like that.
Jan 15 2015
parent ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Fri, 16 Jan 2015 02:05:09 +0000
deadalnix via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 On Tuesday, 13 January 2015 at 12:43:05 UTC, ketmar via=20
 Digitalmars-d wrote:
 On Tue, 13 Jan 2015 12:37:22 +0000
 Bruno Medeiros via Digitalmars-d <digitalmars-d puremagic.com>=20
 wrote:

 On 10/01/2015 22:11, Walter Bright wrote:
 On 1/10/2015 12:17 PM, Walter Bright wrote:

 2. it tends to end bikeshedding arguments about the right=20
 way to format
 things
=20 No, it wouldn't. The tool would have to support formatting=20 options, not just one style (or the tool would be crap). And=20 even then, people could argue about what the default=20 formatting should be...
default should be one that used in Phobos.
=20 phobos is very APIish, most of business logic do not look like=20 that.
that's why formatter must have alot of options to tune. but *default* set must be one that used in Phobos. just to end any converstion like this with "'cause it's used in Phobos. period."
Jan 16 2015
prev sibling next sibling parent "Brian Schott" <briancschott gmail.com> writes:
On Saturday, 10 January 2015 at 20:18:03 UTC, Walter Bright wrote:
 Has someone made a dfmt, like http://gofmt.com/ ?
I have a module in libdparse that takes an AST and formats it to an output range, but it is still fairly primitive at the moment. It could probably be made into a real formatter with a few days of work.
Jan 10 2015
prev sibling next sibling parent Shammah Chancellor <anonymous coward.com> writes:
On 2015-01-10 20:17:34 +0000, Walter Bright said:

 Has someone made a dfmt, like http://gofmt.com/ ?
No, I was planning on working on one if we ever got libd parsing 100% of the code. Unfortunately I haven't had a lot of time to work on these sorts of things lately. -Shammah
Jan 10 2015
prev sibling next sibling parent reply "Brian Schott" <briancschott gmail.com> writes:
On Saturday, 10 January 2015 at 20:18:03 UTC, Walter Bright wrote:
 Has someone made a dfmt, like http://gofmt.com/ ?
https://github.com/Hackerpilot/dfmt The above is the work of one afternoon and not well tested.
Jan 11 2015
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 1/11/2015 5:53 PM, Brian Schott wrote:
 On Saturday, 10 January 2015 at 20:18:03 UTC, Walter Bright wrote:
 Has someone made a dfmt, like http://gofmt.com/ ?
https://github.com/Hackerpilot/dfmt The above is the work of one afternoon and not well tested.
That was quick!
Jan 11 2015
prev sibling parent "Martin Nowak" <code dawg.eu> writes:
On Monday, 12 January 2015 at 01:53:20 UTC, Brian Schott wrote:
 On Saturday, 10 January 2015 at 20:18:03 UTC, Walter Bright 
 wrote:
 Has someone made a dfmt, like http://gofmt.com/ ?
https://github.com/Hackerpilot/dfmt The above is the work of one afternoon and not well tested.
Thanks Brian, looks promising.
Jan 12 2015
prev sibling next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Saturday, 10 January 2015 at 20:18:03 UTC, Walter Bright wrote:
 Has someone made a dfmt, like http://gofmt.com/ ?
That is amongst the plans for libd. I'd be happy to support anyone that want to work on it :)
Jan 11 2015
parent reply Shammah Chancellor <anonymous coward.com> writes:
On 2015-01-12 03:23:28 +0000, deadalnix said:

 On Saturday, 10 January 2015 at 20:18:03 UTC, Walter Bright wrote:
 Has someone made a dfmt, like http://gofmt.com/ ?
That is amongst the plans for libd. I'd be happy to support anyone that want to work on it :)
This is part of the reason I wanted to de-couple libd's parsing from with libd-llvm can currently handle and have separate tests for that libd. Is this something we might be able to consider doing? We can easily have some code and do Code->ast->fmt Code->ast as tests for libd. -Shammah
Jan 12 2015
parent "deadalnix" <deadalnix gmail.com> writes:
On Monday, 12 January 2015 at 15:20:24 UTC, Shammah Chancellor 
wrote:
 On 2015-01-12 03:23:28 +0000, deadalnix said:

 On Saturday, 10 January 2015 at 20:18:03 UTC, Walter Bright 
 wrote:
 Has someone made a dfmt, like http://gofmt.com/ ?
That is amongst the plans for libd. I'd be happy to support anyone that want to work on it :)
This is part of the reason I wanted to de-couple libd's parsing from with libd-llvm can currently handle and have separate tests for that libd. Is this something we might be able to consider doing? We can easily have some code and do Code->ast->fmt Code->ast as tests for libd. -Shammah
The lexer/parser can be used without the semantic analysis part. It should be alright for now.
Jan 12 2015
prev sibling next sibling parent Bruno Medeiros <bruno.do.medeiros+dng gmail.com> writes:
On 10/01/2015 20:17, Walter Bright wrote:
 Has someone made a dfmt, like http://gofmt.com/ ?
With the DDT parser, it would be fairly easy to add such functionality, but it never has been high priority. -- Bruno Medeiros https://twitter.com/brunodomedeiros
Jan 13 2015
prev sibling parent "HaraldZealot" <harald_zealot tut.by> writes:
On Saturday, 10 January 2015 at 20:18:03 UTC, Walter Bright wrote:
 Has someone made a dfmt, like http://gofmt.com/ ?
I with one my good student have started project coDewife 1,5 month ago. It based on some interesting theoretical approach (something like FSM, but with essential differences). It is being in very raw state by now, but has possibility to configure many good configuration. For example we can manipulate with underscore in decimal literal. I have benchmarked this decimal formatter a little, and it shows nod bad result: approximately 5 usec by one codeunit. We in active codding phase by now, but I can't predict terms of first useful release yet. And by the way I have a question. When I read http://dlang.org/lex.html#integerliteral I understand that 0b, 0x or 0x_ isn't valid, but that compiles. Is that bug or spoiled documentations?
Jan 15 2015