www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Is anyone working on a D source code formatting tool?

reply Walter Bright <newshound2 digitalmars.com> writes:
Has someone made a dfmt, like http://gofmt.com/ ?
Jan 10 2015
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-01-10 21:17, Walter Bright wrote:

 Has someone made a dfmt, like http://gofmt.com/ ?


I have thought about it a couple of times but never started. It would be 
really nice to have.

-- 
/Jacob Carlborg
Jan 10 2015
parent reply "Meta" <jared771 gmail.com> writes:
On Saturday, 10 January 2015 at 20:54:56 UTC, Jacob Carlborg 
wrote:

 On 2015-01-10 21:17, Walter Bright wrote:

 Has someone made a dfmt, like http://gofmt.com/ ?


 I have thought about it a couple of times but never started. It 
 would be really nice to have.


https://github.com/Hackerpilot/dfix
Jan 10 2015
parent reply "Meta" <jared771 gmail.com> writes:
On Saturday, 10 January 2015 at 21:17:29 UTC, Meta wrote:

 On Saturday, 10 January 2015 at 20:54:56 UTC, Jacob Carlborg 
 wrote:

 On 2015-01-10 21:17, Walter Bright wrote:

 Has someone made a dfmt, like http://gofmt.com/ ?


 I have thought about it a couple of times but never started. 
 It would be really nice to have.


 https://github.com/Hackerpilot/dfix


Ah, never mind me. Although the functionality between dfix and a 
dfmt is probably quite similar.
Jan 10 2015
parent "Brian Schott" <briancschott gmail.com> writes:
On Saturday, 10 January 2015 at 21:18:58 UTC, Meta wrote:

 https://github.com/Hackerpilot/dfix


 Ah, never mind me. Although the functionality between dfix and 
 a dfmt is probably quite similar.


dfix is more complicated than it could be because it tries to 
avoid changing the formatting of the file that it's operating on.
Jan 10 2015
prev sibling next sibling parent reply "weaselcat" <weaselcat gmail.com> writes:
On Saturday, 10 January 2015 at 20:18:03 UTC, Walter Bright wrote:

 Has someone made a dfmt, like http://gofmt.com/ ?


Uncrustify claims D support.
http://uncrustify.sourceforge.net/
Jan 10 2015
parent Andrej Mitrovic via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 1/10/15, weaselcat via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 On Saturday, 10 January 2015 at 20:18:03 UTC, Walter Bright wrote:

 Has someone made a dfmt, like http://gofmt.com/ ?


 Uncrustify claims D support.
 http://uncrustify.sourceforge.net/


Yes, and the author is responsive whenever a new D-related bug is filed.

I've used Uncrustify successfully with D for many years. It would be
nice to have an implementation in D (purely to be able to more easily
hack on the tool and configure it to our own liking), but as far as
having a tool that works right now - uncrustify is exactly that tool.
Jan 11 2015
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/10/2015 12:17 PM, Walter Bright wrote:

 Has someone made a dfmt, like http://gofmt.com/ ?


Next question - standalone tool, or built in to dmd (like Ddoc)?

BTW, I think dfmt would be a significant win for D:

1. people expect this sort of thing these days
2. it tends to end bikeshedding arguments about the right way to format things
3. it'll help standardize the format of D code in the D repositories
4. it's simply nice and convenient!
5. it's a great first step when you're faced with fixing someone else's crap
code

I don't think it'll be hard to do as a builtin feature of dmd.

My only concern about it is if dfmt is changed, then we get faced with a 
blizzard of changes in the D github repositories.
Jan 10 2015
next sibling parent "Tobias Pankrath" <tobias pankrath.net> writes:
On Saturday, 10 January 2015 at 22:11:55 UTC, Walter Bright wrote:

 On 1/10/2015 12:17 PM, Walter Bright wrote:

 Has someone made a dfmt, like http://gofmt.com/ ?


 Next question - standalone tool, or built in to dmd (like Ddoc)?

 My only concern about it is if dfmt is changed, then we get 
 faced with a blizzard of changes in the D github repositories.


Build it into DMD with an option to change files in place or dump 
them to stdout. I do think though, this would require a pragma or 
a special comment to disable it for parts of code. There will 
always be the case where the standard is wrong.
Jan 10 2015
prev sibling next sibling parent reply "Joakim" <dlang joakim.fea.st> writes:
On Saturday, 10 January 2015 at 22:11:55 UTC, Walter Bright wrote:

 On 1/10/2015 12:17 PM, Walter Bright wrote:

 Has someone made a dfmt, like http://gofmt.com/ ?


 Next question - standalone tool, or built in to dmd (like Ddoc)?

 BTW, I think dfmt would be a significant win for D:

 1. people expect this sort of thing these days
 2. it tends to end bikeshedding arguments about the right way 
 to format things
 3. it'll help standardize the format of D code in the D 
 repositories
 4. it's simply nice and convenient!
 5. it's a great first step when you're faced with fixing 
 someone else's crap code

 I don't think it'll be hard to do as a builtin feature of dmd.

 My only concern about it is if dfmt is changed, then we get 
 faced with a blizzard of changes in the D github repositories.


I agree that there should be a tool like this for D, don't care 
if it's stand-alone or part of dmd.

In my experience submitting PRs on github, 2. and 3. would 
definitely be alleviated by dfmt, as there's sometimes review 
comments about formatting and they'd go away if you just told all 
contributors to run dfmt first, to get the patches in a standard 
format.

As for your concern about github formatting commits, inevitable 
cost of standardizing and then changing your mind.
Jan 10 2015
parent "Suliman" <evermind live.ru> writes:
Can this problem be solve on Text Editor level?

I use Sublime, and I need some tool for code formating. Maybe 
there is any ready to use addons for D?
Jan 10 2015
prev sibling next sibling parent "ponce" <contact gam3sfrommars.fr> writes:
On Saturday, 10 January 2015 at 22:11:55 UTC, Walter Bright wrote:

 On 1/10/2015 12:17 PM, Walter Bright wrote:

 Has someone made a dfmt, like http://gofmt.com/ ?


 Next question - standalone tool, or built in to dmd (like Ddoc)?

 BTW, I think dfmt would be a significant win for D:

 1. people expect this sort of thing these days
 2. it tends to end bikeshedding arguments about the right way 
 to format things
 3. it'll help standardize the format of D code in the D 
 repositories
 4. it's simply nice and convenient!
 5. it's a great first step when you're faced with fixing 
 someone else's crap code

 I don't think it'll be hard to do as a builtin feature of dmd.

 My only concern about it is if dfmt is changed, then we get 
 faced with a blizzard of changes in the D github repositories.


Since Brian Schott already wrote a dfix, it seems easier to make 
dfmt the same program.
Jan 11 2015
prev sibling next sibling parent reply "NVolcz" <niklas.volcz gmail.com> writes:
On Saturday, 10 January 2015 at 22:11:55 UTC, Walter Bright wrote:

 On 1/10/2015 12:17 PM, Walter Bright wrote:

 Has someone made a dfmt, like http://gofmt.com/ ?


 Next question - standalone tool, or built in to dmd (like Ddoc)?

 BTW, I think dfmt would be a significant win for D:

 1. people expect this sort of thing these days
 2. it tends to end bikeshedding arguments about the right way 
 to format things
 3. it'll help standardize the format of D code in the D 
 repositories
 4. it's simply nice and convenient!
 5. it's a great first step when you're faced with fixing 
 someone else's crap code

 I don't think it'll be hard to do as a builtin feature of dmd.

 My only concern about it is if dfmt is changed, then we get 
 faced with a blizzard of changes in the D github repositories.


I guess dfmt is to small for GSOC but could we bundle it with 
some other important tool to create a bigger task?

Best regards,
NVolcz
Jan 11 2015
parent Walter Bright <newshound2 digitalmars.com> writes:
On 1/11/2015 2:00 AM, NVolcz wrote:

 I guess dfmt is to small for GSOC


I'm not so sure. If the code has no comments, it's trivial. Handling comments, 
though, can be a bit of a challenge.
Jan 11 2015
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-01-10 23:11, Walter Bright wrote:


 Next question - standalone tool, or built in to dmd (like Ddoc)?


Ideally the dmd front end should be available as a library then dfmt 
should be a separate tool that uses the same front end library as dmd. 
But I know that we're not there yet.

It's always nice if a tool is built as a library with a with executable 
on top. This allows for other tools to be built using this library.

-- 
/Jacob Carlborg
Jan 11 2015
next sibling parent Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sun, 2015-01-11 at 11:04 +0100, Jacob Carlborg via Digitalmars-d wrote:

 On 2015-01-10 23:11, Walter Bright wrote:
 

 Next question - standalone tool, or built in to dmd (like Ddoc)?


 
 Ideally the dmd front end should be available as a library then dfmt 
 should be a separate tool that uses the same front end library as 
 dmd. But I know that we're not there yet.


And written in D :-)


 It's always nice if a tool is built as a library with a with 
 executable on top. This allows for other tools to be built using 
 this library.


dmd, ldc, gdc, dfmt, ddoc, lots of programs one parsing library 
required.

-- 
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Jan 11 2015
prev sibling parent reply "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Jacob Carlborg"  wrote in message news:m8thr2$pag$1 digitalmars.com...


 Ideally the dmd front end should be available as a library then dfmt 
 should be a separate tool that uses the same front end library as dmd. But 
 I know that we're not there yet.


Well, some of it is.  If somebody wants to work on adding the necessary 
lexer/parser/etc features on the C++ side I'm happy to help get ddmd ready 
to be used for this.


 It's always nice if a tool is built as a library with a with executable on 
 top. This allows for other tools to be built using this library.


Agreed.
Jan 11 2015
parent reply "Stefan Koch" <uplink.coder googlemail.com> writes:
I'm  powerful writing a parser-generator, that will be able to 
transform the generated parse-tree back into source automatically.
writing a rule-based formatter should be pretty doable.
Jan 11 2015
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/11/2015 9:45 AM, Stefan Koch wrote:

 I'm  powerful writing a parser-generator, that will be able to transform the
 generated parse-tree back into source automatically.
 writing a rule-based formatter should be pretty doable.


Formatting the AST into text is straightforward, dmd already does that for .di 
file generation.

The main problem is what to do about comments, which don't fit into the grammar.

A secondary problem is what to do when the line length limit is exceeded, such 
as for long expressions.
Jan 11 2015
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-01-11 19:48, Walter Bright wrote:


 The main problem is what to do about comments, which don't fit into the
 grammar.

 A secondary problem is what to do when the line length limit is
 exceeded, such as for long expressions.


clang-format seems to do a pretty good job with both of these. Comments 
seem to be intact unless they're too long, then they're wrapped. It 
seems to wrap at a space or other non-identifier character. Same thing 
with expressions that are too long.

You should download it [1] a give it a try on some C++ code.

[1] http://llvm.org/releases/download.html

-- 
/Jacob Carlborg
Jan 11 2015
parent "Ahmed Fasih" <wuzzyview gmail.com> writes:
Hi there, I'm a C++/Python refugee, new to D.


 clang-format seems to do a pretty good job with both of these. 
 Comments seem to be intact unless they're too long, then 
 they're wrapped. It seems to wrap at a space or other 
 non-identifier character. Same thing with expressions that are 
 too long.


I would love such a tool for D, especially based on the ideas of 
clang-format. I first heard about clang-format from a talk by 
Google's Chandler Carruth: the way he said it, the LLVM guys 
looked at what their C++ programmers wasted the most time on, and 
it turned out to be whitespace, surprisingly enough. So they 
implemented was is essentially LaTeX for source code (optimal 
placement of spaces and line breaks using Djikstra's algorithm, 
the works). And he said it changed the way he codes, and at that 
point I had to go get it, and I 100% agree with him: it's awesome 
to have.

It works with Vim, Emacs, Visual Studio, Sublime, etc., because 
it provides a simple Python wrapper to the executable that 
anything can call any time to format any code. It works with 
C/C++, Objective-C, and Javascript; when I discovered it could do 
JavaScript, my use of that language rose substantially (could be 
coincidence :).

Reading some of the discussion here about whether to integrate it 
into the compiler, etc., makes me realize that one of the nice 
things about clang-format is real-time interactivity, i.e., I can 
type-type-type code quickly and without bothering with 
whitespace, just getting the idea out of my head and into the 
editor, then when I reach a breathing space I can hit a keycombo, 
and my editor reformats the block I just typed. I find that the 
hits to my flow are much smaller than having to do this manually, 
and I think that's what Carruth meant when he said it changed his 
coding.

Another nice thing about it is that it's fully parameterized (how 
many spaces, whether to indent this, what penalty to assign that, 
etc., example at [1]). It can search the current and parent 
directories for a .clang-format file with those parameters. It 
can also generate stock .clang-format files conforming to various 
coding styles, viz., LLVM, Google, Mozilla, WebKit, & Chromium. I 
throw this into my Git repos; happiness ensues.

Hope this helps outline the paths others have taken!

[1] Example Javascript .clang-format file: 
https://github.com/fasiha/kanjiwild/blob/gh-pages/.clang-format
Jan 16 2015
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/11/15 10:48 AM, Walter Bright wrote:

 On 1/11/2015 9:45 AM, Stefan Koch wrote:

 I'm  powerful writing a parser-generator, that will be able to
 transform the
 generated parse-tree back into source automatically.
 writing a rule-based formatter should be pretty doable.


 Formatting the AST into text is straightforward, dmd already does that
 for .di file generation.

 The main problem is what to do about comments, which don't fit into the
 grammar.


In the first version comments might go through unchanged.


 A secondary problem is what to do when the line length limit is
 exceeded, such as for long expressions.





Andrei
Jan 11 2015
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/11/2015 11:50 AM, Andrei Alexandrescu wrote:

 On 1/11/15 10:48 AM, Walter Bright wrote:

 The main problem is what to do about comments, which don't fit into the
 grammar.


 In the first version comments might go through unchanged.


Consider:

     for /*comment*/ (a;
                      b;
                      c)

Do what with that?
Jan 11 2015
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/11/15 1:15 PM, Walter Bright wrote:

 On 1/11/2015 11:50 AM, Andrei Alexandrescu wrote:

 On 1/11/15 10:48 AM, Walter Bright wrote:

 The main problem is what to do about comments, which don't fit into the
 grammar.


 In the first version comments might go through unchanged.


 Consider:

      for /*comment*/ (a;
                       b;
                       c)

 Do what with that?


I don't know. Simplest would be to punt for now - such rare embedded 
comments should not be blockers. -- Andrei
Jan 11 2015
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/11/2015 1:31 PM, Andrei Alexandrescu wrote:

 On 1/11/15 1:15 PM, Walter Bright wrote:

 On 1/11/2015 11:50 AM, Andrei Alexandrescu wrote:

 On 1/11/15 10:48 AM, Walter Bright wrote:

 The main problem is what to do about comments, which don't fit into the
 grammar.


 In the first version comments might go through unchanged.


 Consider:

      for /*comment*/ (a;
                       b;
                       c)

 Do what with that?


 I don't know. Simplest would be to punt for now - such rare embedded comments
 should not be blockers. -- Andrei


Normally I would agree, but deleting peoples' comments from their source code
is 
not a good plan. It'll make them justifiably angry.
Jan 11 2015
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/11/15 4:37 PM, Walter Bright wrote:

 On 1/11/2015 1:31 PM, Andrei Alexandrescu wrote:

 On 1/11/15 1:15 PM, Walter Bright wrote:

 On 1/11/2015 11:50 AM, Andrei Alexandrescu wrote:

 On 1/11/15 10:48 AM, Walter Bright wrote:

 The main problem is what to do about comments, which don't fit into
 the
 grammar.


 In the first version comments might go through unchanged.


 Consider:

      for /*comment*/ (a;
                       b;
                       c)

 Do what with that?


 I don't know. Simplest would be to punt for now - such rare embedded
 comments
 should not be blockers. -- Andrei


 Normally I would agree, but deleting peoples' comments from their source
 code is not a good plan. It'll make them justifiably angry.


By punt i mean leave as is. -- Andrei
Jan 11 2015
prev sibling parent reply "Zach the Mystic" <reachzach gggmail.com> writes:
On Monday, 12 January 2015 at 00:38:20 UTC, Walter Bright wrote:

 On 1/11/15 10:48 AM, Walter Bright wrote:

 The main problem is what to do about comments, which don't 
 fit into the
 grammar.


 In the first version comments might go through unchanged.


 Consider:

     for /*comment*/ (a;
                      b;
                      c)

 Do what with that?


 I don't know. Simplest would be to punt for now - such rare 
 embedded comments
 should not be blockers. -- Andrei


 Normally I would agree, but deleting peoples' comments from 
 their source code is not a good plan. It'll make them 
 justifiably angry.


This conversation reminds me of something I've thought about ever 
since I first studied D. D takes the C preprocessor and folds it 
into the regular AST. But comments still seemed like the outlier. 
I looked through a bunch of source code and tried to figure out 
the most specific place anyone could possibly put a comment. The 
most detailed I found were something like:

enum X {
   ONE,
   TWO, // We need a TWO here
   THREE
}

So I started conceiving of a language in which even the 
*comments* were part of the AST. For, me this would be the 
aesthetic ideal. It just seemed like the next step in total AST 
integration.
Jan 11 2015
next sibling parent reply "qznc" <qznc web.de> writes:
On Monday, 12 January 2015 at 05:43:33 UTC, Zach the Mystic wrote:

 On Monday, 12 January 2015 at 00:38:20 UTC, Walter Bright wrote:

 On 1/11/15 10:48 AM, Walter Bright wrote:

 The main problem is what to do about comments, which don't 
 fit into the
 grammar.


 In the first version comments might go through unchanged.


 Consider:

    for /*comment*/ (a;
                     b;
                     c)

 Do what with that?


 I don't know. Simplest would be to punt for now - such rare 
 embedded comments
 should not be blockers. -- Andrei


 Normally I would agree, but deleting peoples' comments from 
 their source code is not a good plan. It'll make them 
 justifiably angry.


 This conversation reminds me of something I've thought about 
 ever since I first studied D. D takes the C preprocessor and 
 folds it into the regular AST. But comments still seemed like 
 the outlier. I looked through a bunch of source code and tried 
 to figure out the most specific place anyone could possibly put 
 a comment. The most detailed I found were something like:

 enum X {
   ONE,
   TWO, // We need a TWO here
   THREE
 }

 So I started conceiving of a language in which even the 
 *comments* were part of the AST. For, me this would be the 
 aesthetic ideal. It just seemed like the next step in total AST 
 integration.


The clang-format approach is to make decisions based on the AST, 
but edit the byte array. AST nodes contain exact positions: line 
and column numbers. Sometimes more of them. For example, an 
if-statement needs to know the position of 'else' as well.

Here is another example: void setX(int position /* inches */);
However, I think it should really be wrapped into a struct 
instead to get type checker's approval.

Initially, clang-format only did whitespace changes. I am not 
sure if they weakened this already. For example, stripping parens 
in expressions (((((x)))) => x) would be acceptable for me.


btw +1 for https://github.com/Hackerpilot/dfmt
Jan 12 2015
parent "Brian Schott" <briancschott gmail.com> writes:
On Monday, 12 January 2015 at 15:30:34 UTC, qznc wrote:

 The clang-format approach is to make decisions based on the 
 AST, but edit the byte array.


dfix uses a similar approach. It uses the AST location 
information to make decisions while iterating through the token 
array. I think I will end up doing the same thing with dfmt.
Jan 12 2015
prev sibling parent reply "qznc" <qznc web.de> writes:
On Monday, 12 January 2015 at 05:43:33 UTC, Zach the Mystic wrote:

 On Monday, 12 January 2015 at 00:38:20 UTC, Walter Bright wrote:

 On 1/11/15 10:48 AM, Walter Bright wrote:

 The main problem is what to do about comments, which don't 
 fit into the
 grammar.


 In the first version comments might go through unchanged.


 Consider:

    for /*comment*/ (a;
                     b;
                     c)

 Do what with that?


 I don't know. Simplest would be to punt for now - such rare 
 embedded comments
 should not be blockers. -- Andrei


 Normally I would agree, but deleting peoples' comments from 
 their source code is not a good plan. It'll make them 
 justifiably angry.


 This conversation reminds me of something I've thought about 
 ever since I first studied D. D takes the C preprocessor and 
 folds it into the regular AST. But comments still seemed like 
 the outlier. I looked through a bunch of source code and tried 
 to figure out the most specific place anyone could possibly put 
 a comment. The most detailed I found were something like:

 enum X {
   ONE,
   TWO, // We need a TWO here
   THREE
 }

 So I started conceiving of a language in which even the 
 *comments* were part of the AST. For, me this would be the 
 aesthetic ideal. It just seemed like the next step in total AST 
 integration.


The clang-format approach is to make decisions based on the AST,
but edit the byte array. AST nodes contain exact positions: line
and column numbers. Sometimes more of them. For example, an
if-statement needs to know the position of 'else' as well.

Here is another example: void setX(int position /* inches */);
However, I think it should really be wrapped into a struct
instead to get type checker's approval.

Initially, clang-format only did whitespace changes. I am not
sure if they weakened this already. For example, stripping parens
in expressions (((((x)))) => x) would be acceptable for me.


btw +1 for https://github.com/Hackerpilot/dfmt
Jan 12 2015
parent "deadalnix" <deadalnix gmail.com> writes:
On Monday, 12 January 2015 at 15:30:42 UTC, qznc wrote:

 So I started conceiving of a language in which even the 
 *comments* were part of the AST. For, me this would be the 
 aesthetic ideal. It just seemed like the next step in total 
 AST integration.


 The clang-format approach is to make decisions based on the AST,
 but edit the byte array. AST nodes contain exact positions: line
 and column numbers. Sometimes more of them. For example, an
 if-statement needs to know the position of 'else' as well.

 Here is another example: void setX(int position /* inches */);
 However, I think it should really be wrapped into a struct
 instead to get type checker's approval.

 Initially, clang-format only did whitespace changes. I am not
 sure if they weakened this already. For example, stripping 
 parens
 in expressions (((((x)))) => x) would be acceptable for me.


 btw +1 for https://github.com/Hackerpilot/dfmt


Last time I checked, it work on the lexer rather than the AST.
Jan 12 2015
prev sibling next sibling parent "weaselcat" <weaselcat gmail.com> writes:
On Sunday, 11 January 2015 at 21:16:41 UTC, Walter Bright wrote:

 On 1/11/2015 11:50 AM, Andrei Alexandrescu wrote:

 On 1/11/15 10:48 AM, Walter Bright wrote:

 The main problem is what to do about comments, which don't 
 fit into the
 grammar.


 In the first version comments might go through unchanged.


 Consider:

     for /*comment*/ (a;
                      b;
                      c)

 Do what with that?


Why not just move the comment to the end of the expression?
Jan 11 2015
prev sibling parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sun, 11 Jan 2015 13:15:22 -0800
Walter Bright via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 On 1/11/2015 11:50 AM, Andrei Alexandrescu wrote:

 On 1/11/15 10:48 AM, Walter Bright wrote:

 The main problem is what to do about comments, which don't fit into the
 grammar.


 In the first version comments might go through unchanged.


=20
 Consider:
=20
      for /*comment*/ (a;
                       b;
                       c)
=20
 Do what with that?
=20


it's easy: put it before `for`.

  /*comment*/
  for (...)

or just ignore it. and i must confess that i've never seen comment like
this in my lifetime.
Jan 12 2015
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2015-01-12 09:11, ketmar via Digitalmars-d wrote:


 it's easy: put it before `for`.

    /*comment*/
    for (...)

 or just ignore it. and i must confess that i've never seen comment like
 this in my lifetime.


I agree. clang-format manage to keep it in the same place.

-- 
/Jacob Carlborg
Jan 12 2015
prev sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Monday, 12 January 2015 at 08:11:27 UTC, ketmar via 
Digitalmars-d wrote:

 it's easy: put it before `for`.

   /*comment*/
   for (...)

 or just ignore it. and i must confess that i've never seen 
 comment like
 this in my lifetime.


You can't ignore. That is why building such tool in not that easy.
Jan 12 2015
parent ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Mon, 12 Jan 2015 19:17:47 +0000
deadalnix via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 On Monday, 12 January 2015 at 08:11:27 UTC, ketmar via=20
 Digitalmars-d wrote:

 it's easy: put it before `for`.

   /*comment*/
   for (...)

 or just ignore it. and i must confess that i've never seen=20
 comment like
 this in my lifetime.


=20
 You can't ignore. That is why building such tool in not that easy.


by "ignore" i meant "live it where it was". if somebody is so smart
that he writes such comments... oh, well, then he is smart enough to
reformat the source manually. and if this is some other person's
source... i'd say "don't use it, that person is sick".
Jan 12 2015
prev sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 11 January 2015 at 19:50:51 UTC, Andrei Alexandrescu 
wrote:

 A secondary problem is what to do when the line length limit is
 exceeded, such as for long expressions.





The usual way is to associate cost with various cesures and run 
something Dijkstra (or A*) like to find the best cesures.
Jan 11 2015
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2015-01-11 19:48, Walter Bright wrote:


 The main problem is what to do about comments, which don't fit into the
 grammar.


Both Clang and Eclipse JDT provide API's for accessing comments.

-- 
/Jacob Carlborg
Jan 12 2015
prev sibling next sibling parent Bruno Medeiros <bruno.do.medeiros+dng gmail.com> writes:
On 11/01/2015 18:48, Walter Bright wrote:

 On 1/11/2015 9:45 AM, Stefan Koch wrote:

 I'm  powerful writing a parser-generator, that will be able to
 transform the
 generated parse-tree back into source automatically.
 writing a rule-based formatter should be pretty doable.


 Formatting the AST into text is straightforward, dmd already does that
 for .di file generation.

 The main problem is what to do about comments, which don't fit into the
 grammar.


It's not that hard to do that once you have a parser that preserves the 
source range of all AST nodes. With that you can write a formatting 
algorithm than *modifies* the original source, assisted by AST info, 
instead of trying to recreate the whole source from the AST info alone. 
This way you can reformat without losing comment data, even if the AST 
doesn't have store comments.


-- 
Bruno Medeiros
https://twitter.com/brunodomedeiros
Jan 13 2015
prev sibling parent reply Ary Borenszweig <ary esperanto.org.ar> writes:
On 1/11/15 3:48 PM, Walter Bright wrote:

 On 1/11/2015 9:45 AM, Stefan Koch wrote:

 I'm  powerful writing a parser-generator, that will be able to
 transform the
 generated parse-tree back into source automatically.
 writing a rule-based formatter should be pretty doable.


 Formatting the AST into text is straightforward, dmd already does that
 for .di file generation.

 The main problem is what to do about comments, which don't fit into the
 grammar.

 A secondary problem is what to do when the line length limit is
 exceeded, such as for long expressions.


The way I did it in Descent (I copied the logic from JDT) is to parse 
the code into an AST, and then walk the AST in sync with a lexer. So if 
you have this:

void /* comment /* foo() {}

the AST would be a FunctionDecl (whatever the name is) so you'd expect a 
type (consume that AST node, in sync with the lexer), then check for 
comments/newlines/etc., skip/print them, then consume the name, check 
for comments/newlines/etc.

That way the AST doesn't have to know anything about comments, but 
comments need to be known by the lexer (via a flag, probably).

Considering how flexible is JDT's formatter, I think this solution is 
pretty good.
Jan 16 2015
parent reply "Brian Schott" <briancschott gmail.com> writes:
On Friday, 16 January 2015 at 15:06:42 UTC, Ary Borenszweig wrote:

 The way I did it in Descent (I copied the logic from JDT) is to 
 parse the code into an AST, and then walk the AST in sync with 
 a lexer.


My dfmt tool does something similar. The parser runs over the 
code first and makes notes on things like the location of unary 
operators and which braces end function/aggregate declarations. 
Then the formatter iterates over the tokens (including comments) 
and is able to correctly print "a* b;" instead of "a * b;".
Jan 16 2015
parent "qznc" <qznc web.de> writes:
On Friday, 16 January 2015 at 15:55:53 UTC, Brian Schott wrote:

 On Friday, 16 January 2015 at 15:06:42 UTC, Ary Borenszweig 
 wrote:

 The way I did it in Descent (I copied the logic from JDT) is 
 to parse the code into an AST, and then walk the AST in sync 
 with a lexer.


 My dfmt tool does something similar. The parser runs over the 
 code first and makes notes on things like the location of unary 
 operators and which braces end function/aggregate declarations. 
 Then the formatter iterates over the tokens (including 
 comments) and is able to correctly print "a* b;" instead of "a 
 * b;".


This is a short talk about clang-format's design and 
implementation:
https://www.youtube.com/watch?v=s7JmdCfI__c

They have this concept of "unwrapped lines" into which the source 
code is split by a "structural parser". I am not sure, if dfmt 
really needs it, because we don't have cpp macros littered around 
in D code.

While dfmt already works well for some code (e.g. the snippet on 
dlang.org frontpage), the biggest hurdle (imho) is an expression 
formatter, which for example understands operator precedences. 
Currently, dfmt uses a greedy line fill algorithm.
Jan 16 2015
prev sibling next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Saturday, 10 January 2015 at 22:11:55 UTC, Walter Bright wrote:

 On 1/10/2015 12:17 PM, Walter Bright wrote:

 Has someone made a dfmt, like http://gofmt.com/ ?


 Next question - standalone tool, or built in to dmd (like Ddoc)?


Please don't make the compiler-executable modify source code. It 
makes makefile editing more "dangerous".

A standalone tool is appropriate.
Jan 11 2015
prev sibling next sibling parent reply Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sat, 2015-01-10 at 14:11 -0800, Walter Bright via Digitalmars-d wrote:

 On 1/10/2015 12:17 PM, Walter Bright wrote:

 Has someone made a dfmt, like http://gofmt.com/ ?


 
 Next question - standalone tool, or built in to dmd (like Ddoc)?


Why in the compiler, source code formatting is not a compilation 
issue. Of course the issue is parsing to AST, which the compiler has, 
but doesn't provide as a library. In Go, the parsing code is separated 
from the compiler so that different tools can be constructed using 
different combinations of the same code.

Also why in DMD and not in LDC or GDC?


 BTW, I think dfmt would be a significant win for D:
 
 1. people expect this sort of thing these days


Thanks to Python PEP-8 and Go, yes. Personalized code formatting is 
increasingly a thing of the past, programming languages are now 
opinionated and fascist when it comes to formatting. Even if the rules 
are wrong.


 2. it tends to end bikeshedding arguments about the right way to 
 format things


Except when the tool implements the wrong style.


 3. it'll help standardize the format of D code in the D repositories 



Even if it is the wrong formatting standard?

I have to admit that the Phobos format rules make the code look 
appallingly ugly to me, sufficiently so that I really do not want to 
work on that codebase. I guess I am biased towards K&R C (which I use 
to drive my C++ style), Java and Go.


 4. it's simply nice and convenient!


Working with Go in Emacs or LiteIDE is a bit of a joy as you get full 
reformatting on save. Fortunately I only have two main gripes with the 
Go formatting style, but no-one has a choice, use the Go style as 
dictated by the Go team at Google or do not use Go.


 5. it's a great first step when you're faced with fixing someone 
 else's crap code


Having just taken on a C/C++ codebase, I can agree that manually 
reformatting is a great way of "getting in" to the code. Then an 
automated formatting would contribute as well.


 I don't think it'll be hard to do as a builtin feature of dmd.


It should be a separate tool not a part of one of the three compilers. 


 My only concern about it is if dfmt is changed, then we get faced 
 with a
 blizzard of changes in the D github repositories.


Tough ;-)

-- 
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Jan 11 2015
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/11/2015 4:47 AM, Russel Winder via Digitalmars-d wrote:

 Next question - standalone tool, or built in to dmd (like Ddoc)?


 Also why in DMD and not in LDC or GDC?


It would be in the DMD front end, so LDC and GDC would it automatically.



 1. people expect this sort of thing these days


 Thanks to Python PEP-8 and Go, yes. Personalized code formatting is
 increasingly a thing of the past, programming languages are now
 opinionated and fascist when it comes to formatting. Even if the rules
 are wrong.


The rules are always going to be wrong. But in this case, that's ok.



 2. it tends to end bikeshedding arguments about the right way to
 format things


 Except when the tool implements the wrong style.


Sometimes it's better to just conform.



 3. it'll help standardize the format of D code in the D repositories


 Even if it is the wrong formatting standard?

 I have to admit that the Phobos format rules make the code look
 appallingly ugly to me, sufficiently so that I really do not want to
 work on that codebase. I guess I am biased towards K&R C (which I use
 to drive my C++ style), Java and Go.


Really? Like what? After many years of arguing about formatting myself, I 
decided that I had better things to waste my time on.



 4. it's simply nice and convenient!


 Working with Go in Emacs or LiteIDE is a bit of a joy as you get full
 reformatting on save. Fortunately I only have two main gripes with the
 Go formatting style, but no-one has a choice, use the Go style as
 dictated by the Go team at Google or do not use Go.


So you decided to conform rather than not use Go.

(Anyhow, I suggest that gofmt is only mandatory if you wish to contribute to 
offical Go code, not for your own projects.)
Jan 11 2015
parent Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sun, 2015-01-11 at 11:19 -0800, Walter Bright via Digitalmars-d wrote:

 


[…]

 It would be in the DMD front end, so LDC and GDC would it 
 automatically.


s/it/get it/ ?

What wouldn't be automatic would be the command line options and other 
surrounding bits and pieces.

 


[…]

 The rules are always going to be wrong. But in this case, that's ok.


For someone, not everyone. For some the rules will be right or at 
least acceptable.

 


[…]

 Sometimes it's better to just conform.


It depends if the grievances are small enough to ignore and/or the 
benefits outweight. Sometimes conformance is a "bridge too far" and 
non-use, or use of an alternative is the right course of action.


 


[…]

 Really? Like what? After many years of arguing about formatting 
 myself, I
 decided that I had better things to waste my time on.


I do not see a point in opening up the debate. An official style 
exists, it is that which is required for Phobos. This is the style 
that will be implemented by dfmt.

[…]


 
 So you decided to conform rather than not use Go.


Yes. My grievances against the enforced style are not sufficient to 
abandon, and the need to deal with error codes at all times is not a 
sufficient pain to stop me using Go. What is the counterbalance? Go 
routines, the use of channels, and the eshewing of any form of shared 
mutable memory. Go has a lot of crap, a lot of positive and negative 
hype, and dataflow built into the language. I find it surprising easy 
to ignore the crap.


 (Anyhow, I suggest that gofmt is only mandatory if you wish to 
 contribute to
 offical Go code, not for your own projects.)


I suspect applying gofmt to Phobos code would have interesting results 
;-)

PEP-8 allows for some minor variations. It can be applied as is for 
official PEP-8 compliant code or you can have variation. Of course the 
pep8 code is only an advisor, gofmt rewrites your code. Originally the 
gc compiler automatically rewrote your code,  but that was seen as too 
fascist, so gofmt was split off as a separate tool. Now though 
everyone uses gofmt anyway adhering to the "one true style". Having 
gofmt separate is though still the right thing as there is gc and 
gccgo which share no code.

-- 
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Jan 12 2015
prev sibling next sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Saturday, 10 January 2015 at 22:11:55 UTC, Walter Bright wrote:

 On 1/10/2015 12:17 PM, Walter Bright wrote:

 Has someone made a dfmt, like http://gofmt.com/ ?


 Next question - standalone tool, or built in to dmd (like Ddoc)?


I think best approach is akin to git subcommands - you can call 
stuff via `git command` but it in fact runs separate 
`git-command` binary behind the scene (which can also be used 
stand-alone). I would love to see DDOC implemented that way too.
Jan 11 2015
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/11/2015 5:11 AM, Dicebot wrote:

 I would love to see DDOC implemented that way too.


Ddoc makes use of semantic info, not just an AST. For semantic info, you pretty 
much need a real compiler.
Jan 11 2015
parent Jacob Carlborg <doob me.com> writes:
On 2015-01-11 20:20, Walter Bright wrote:


 Ddoc makes use of semantic info, not just an AST. For semantic info, you
 pretty much need a real compiler.


I've been thinking of that the last couple of days. It should be pretty 
straightforward to copy-paste the driver part of DMD, i.e the part 
contains the main function and handling of the command line arguments. 
Then remove everything that's not needed for Ddoc.

-- 
/Jacob Carlborg
Jan 11 2015
prev sibling next sibling parent ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sun, 11 Jan 2015 12:47:41 +0000
Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 I don't think it'll be hard to do as a builtin feature of dmd.


=20
 It should be a separate tool not a part of one of the three compilers.=20


i can't see anything wrong with built-in tool. even if it can't be
configured to one's tastes, it's still good for occasional
contributors, who can stop worrying about code formatting and just
write that code.

and if it will be configurable (which is not that hard to do -- to some
extent), it will be usable for many more people. besides, being part of
the compiler this tool has a good chances of being always up-to-date.

as there is already .di emitter, i believe that is can be customised
further to do full code formatting. the only thing left to solve is
comments (not a small one, though). i.e. compiler already has most of
the work done, why not reuse it?
Jan 11 2015
prev sibling parent reply Bruno Medeiros <bruno.do.medeiros+dng gmail.com> writes:
On 10/01/2015 22:11, Walter Bright wrote:

 On 1/10/2015 12:17 PM, Walter Bright wrote:

 2. it tends to end bikeshedding arguments about the right way to format
 things


No, it wouldn't. The tool would have to support formatting options, not 
just one style (or the tool would be crap). And even then, people could 
argue about what the default formatting should be...

-- 
Bruno Medeiros
https://twitter.com/brunodomedeiros
Jan 13 2015
parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, 13 Jan 2015 12:37:22 +0000
Bruno Medeiros via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 On 10/01/2015 22:11, Walter Bright wrote:

 On 1/10/2015 12:17 PM, Walter Bright wrote:

 2. it tends to end bikeshedding arguments about the right way to format
 things


=20
 No, it wouldn't. The tool would have to support formatting options, not=20
 just one style (or the tool would be crap). And even then, people could=20
 argue about what the default formatting should be...


default should be one that used in Phobos.
Jan 13 2015
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 13 January 2015 at 12:43:05 UTC, ketmar via 
Digitalmars-d wrote:

 On Tue, 13 Jan 2015 12:37:22 +0000
 Bruno Medeiros via Digitalmars-d <digitalmars-d puremagic.com> 
 wrote:


 On 10/01/2015 22:11, Walter Bright wrote:

 On 1/10/2015 12:17 PM, Walter Bright wrote:

 2. it tends to end bikeshedding arguments about the right 
 way to format
 things


 
 No, it wouldn't. The tool would have to support formatting 
 options, not just one style (or the tool would be crap). And 
 even then, people could argue about what the default 
 formatting should be...


 default should be one that used in Phobos.


phobos is very APIish, most of business logic do not look like 
that.
Jan 15 2015
parent ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Fri, 16 Jan 2015 02:05:09 +0000
deadalnix via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 On Tuesday, 13 January 2015 at 12:43:05 UTC, ketmar via=20
 Digitalmars-d wrote:

 On Tue, 13 Jan 2015 12:37:22 +0000
 Bruno Medeiros via Digitalmars-d <digitalmars-d puremagic.com>=20
 wrote:


 On 10/01/2015 22:11, Walter Bright wrote:

 On 1/10/2015 12:17 PM, Walter Bright wrote:

 2. it tends to end bikeshedding arguments about the right=20
 way to format
 things


=20
 No, it wouldn't. The tool would have to support formatting=20
 options, not just one style (or the tool would be crap). And=20
 even then, people could argue about what the default=20
 formatting should be...


 default should be one that used in Phobos.


=20
 phobos is very APIish, most of business logic do not look like=20
 that.


that's why formatter must have alot of options to tune. but *default*
set must be one that used in Phobos. just to end any converstion like
this with "'cause it's used in Phobos. period."
Jan 16 2015
prev sibling next sibling parent "Brian Schott" <briancschott gmail.com> writes:
On Saturday, 10 January 2015 at 20:18:03 UTC, Walter Bright wrote:

 Has someone made a dfmt, like http://gofmt.com/ ?


I have a module in libdparse that takes an AST and formats it to 
an output range, but it is still fairly primitive at the moment. 
It could probably be made into a real formatter with a few days 
of work.
Jan 10 2015
prev sibling next sibling parent Shammah Chancellor <anonymous coward.com> writes:
On 2015-01-10 20:17:34 +0000, Walter Bright said:


 Has someone made a dfmt, like http://gofmt.com/ ?


No, I was planning on working on one if we ever got libd parsing 100% 
of the code.    Unfortunately I haven't had a lot of time to work on 
these sorts of things lately.

-Shammah
Jan 10 2015
prev sibling next sibling parent reply "Brian Schott" <briancschott gmail.com> writes:
On Saturday, 10 January 2015 at 20:18:03 UTC, Walter Bright wrote:

 Has someone made a dfmt, like http://gofmt.com/ ?


https://github.com/Hackerpilot/dfmt

The above is the work of one afternoon and not well tested.
Jan 11 2015
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 1/11/2015 5:53 PM, Brian Schott wrote:

 On Saturday, 10 January 2015 at 20:18:03 UTC, Walter Bright wrote:

 Has someone made a dfmt, like http://gofmt.com/ ?


 https://github.com/Hackerpilot/dfmt

 The above is the work of one afternoon and not well tested.


That was quick!
Jan 11 2015
prev sibling parent "Martin Nowak" <code dawg.eu> writes:
On Monday, 12 January 2015 at 01:53:20 UTC, Brian Schott wrote:

 On Saturday, 10 January 2015 at 20:18:03 UTC, Walter Bright 
 wrote:

 Has someone made a dfmt, like http://gofmt.com/ ?


 https://github.com/Hackerpilot/dfmt

 The above is the work of one afternoon and not well tested.


Thanks Brian, looks promising.
Jan 12 2015
prev sibling next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Saturday, 10 January 2015 at 20:18:03 UTC, Walter Bright wrote:

 Has someone made a dfmt, like http://gofmt.com/ ?


That is amongst the plans for libd. I'd be happy to support 
anyone that want to work on it :)
Jan 11 2015
parent reply Shammah Chancellor <anonymous coward.com> writes:
On 2015-01-12 03:23:28 +0000, deadalnix said:


 On Saturday, 10 January 2015 at 20:18:03 UTC, Walter Bright wrote:

 Has someone made a dfmt, like http://gofmt.com/ ?


 
 That is amongst the plans for libd. I'd be happy to support anyone that 
 want to work on it :)


This is part of the reason I wanted to de-couple libd's parsing from 
with libd-llvm can currently handle and have separate tests for that 
libd.

Is this something we might be able to consider doing?  We can easily 
have some code and do Code->ast->fmt Code->ast as tests for libd.

-Shammah
Jan 12 2015
parent "deadalnix" <deadalnix gmail.com> writes:
On Monday, 12 January 2015 at 15:20:24 UTC, Shammah Chancellor 
wrote:

 On 2015-01-12 03:23:28 +0000, deadalnix said:


 On Saturday, 10 January 2015 at 20:18:03 UTC, Walter Bright 
 wrote:

 Has someone made a dfmt, like http://gofmt.com/ ?


 
 That is amongst the plans for libd. I'd be happy to support 
 anyone that want to work on it :)


 This is part of the reason I wanted to de-couple libd's parsing 
 from with libd-llvm can currently handle and have separate 
 tests for that libd.

 Is this something we might be able to consider doing?  We can 
 easily have some code and do Code->ast->fmt Code->ast as tests 
 for libd.

 -Shammah


The lexer/parser can be used without the semantic analysis part. 
It should be alright for now.
Jan 12 2015
prev sibling next sibling parent Bruno Medeiros <bruno.do.medeiros+dng gmail.com> writes:
On 10/01/2015 20:17, Walter Bright wrote:

 Has someone made a dfmt, like http://gofmt.com/ ?


With the DDT parser, it would be fairly easy to add such functionality, 
but it never has been high priority.

-- 
Bruno Medeiros
https://twitter.com/brunodomedeiros
Jan 13 2015
prev sibling parent "HaraldZealot" <harald_zealot tut.by> writes:
On Saturday, 10 January 2015 at 20:18:03 UTC, Walter Bright wrote:

 Has someone made a dfmt, like http://gofmt.com/ ?


I with one my good student have started project coDewife 1,5 
month ago. It based on some interesting theoretical approach 
(something like FSM, but with essential differences). It is being 
in very raw state by now, but has possibility to configure many 
good configuration. For example we can manipulate with underscore 
in decimal literal. I have benchmarked this decimal formatter a 
little, and it shows nod bad result: approximately 5 usec by one 
codeunit. We in active codding phase by now, but I can't predict 
terms of first useful release yet.


And by the way I have a question. When I read 
http://dlang.org/lex.html#integerliteral I understand that 0b, 0x 
or 0x_ isn't valid, but that compiles. Is that bug or spoiled 
documentations?
Jan 15 2015