www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - D Compiler as a Library

reply Ary Manzana <ary esperanto.org.ar> writes:
Having a D compiler available as a library will (at least) give these 
benefits:

   1. Can be used by an IDE: D is statically typed and so an IDE can 
benefit a lot from this. The features Descent had, as far as I remember, 
were:
     1.1. Outline
     1.2. Autocompletion
     1.3. Type Hierarchy
     1.4. Syntax and semantic errors, showing not only the line number 
but also column numbers if it makes sense
     1.5. Automatic import inclusion (say, typing writefln and getting a 
list of modules that provide that symbol)
     1.6. Compile-time view: replace auto with the inferred type, insert 
mixins into scope, rewrite operator overloads and other lowerings (but 
I'm not sure this point is really useful)
     1.7. Determine, given a set of versions and flags, which branches 
of static ifs are used/unused
     1.8. Open declaration
     1.9. Show implementations (of an interface, of interface's method 
or, abstract methods, or method overrides).
     1.10. Propose to override a method (you type some letters and then 
hit some key combination and get a list of methods to override)
     1.11. Get the code of a template when instantiated.
  2. Can be used to build better doc generators: one that shows known 
subclasses or interface implementation, shows inherited methods, type 
hierarchy.
  3. Can be used for lints and other such tools.

As you can see, a simple lexer/parser built into an IDE, doc generator 
or lint will just give basic features but will never achieve something 
exceptionally good if it lacks the full semantic knowledge of the code.

I'll write a list of things I'd like this compiler-as-library to have, 
but please help me make it bigger :-)

  * Don't use global variables (DMD is just thought to be run once, so 
when used as a library it can just be used, well, once)
  * Provide a lexer which gives line numbers and column numbers 
(beginning, end)
  * Provide a parser with the same features
  * The semantic phase should not discard any information found while 
parsing. For example when DMD resolves a type it recursively resolves 
aliasing and keeps the last one. An example:

   alias int foo;
   alias foo* bar;

   bar something() { ... }

   It would be nice if "bar", after semantic analysis is done, carries 
the information that bar is "foo*" and that "foo" is "int". Also that 
something's return type is "bar", not "int*".
  * Provide errors and warnings that have line numbers as well as column 
numbers.
  * Allow to parse the top-level definitions of a module. Whit this I 
mean skipping function bodies. At least Descent first built a the 
outline of the whole project by doing this. This mode should also allow 
specifying a location as a target, and if that location falls inside a 
function body then it's contents are returned (useful when editing a 
file, so you can get the outline as well as semantic info of the 
function currently being edited, which will never affect semantic in 
other parts of the module). This will dramatically speed up the editor.
  * Don't stop parsing on errors (I think DMD already does this).
  * Provide a visitor class. If possible, use visitors to implement 
semantic analysis. The visitor will make it super easy to implement 
lints and to generate documentation.
Apr 13 2012
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
Le 13/04/2012 11:58, Ary Manzana a écrit :
 Having a D compiler available as a library will (at least) give these
 benefits:

 1. Can be used by an IDE: D is statically typed and so an IDE can
 benefit a lot from this. The features Descent had, as far as I remember,
 were:
 1.1. Outline
 1.2. Autocompletion
 1.3. Type Hierarchy
 1.4. Syntax and semantic errors, showing not only the line number but
 also column numbers if it makes sense
 1.5. Automatic import inclusion (say, typing writefln and getting a list
 of modules that provide that symbol)
 1.6. Compile-time view: replace auto with the inferred type, insert
 mixins into scope, rewrite operator overloads and other lowerings (but
 I'm not sure this point is really useful)
 1.7. Determine, given a set of versions and flags, which branches of
 static ifs are used/unused
 1.8. Open declaration
 1.9. Show implementations (of an interface, of interface's method or,
 abstract methods, or method overrides).
 1.10. Propose to override a method (you type some letters and then hit
 some key combination and get a list of methods to override)
 1.11. Get the code of a template when instantiated.
 2. Can be used to build better doc generators: one that shows known
 subclasses or interface implementation, shows inherited methods, type
 hierarchy.
 3. Can be used for lints and other such tools.

 As you can see, a simple lexer/parser built into an IDE, doc generator
 or lint will just give basic features but will never achieve something
 exceptionally good if it lacks the full semantic knowledge of the code.

 I'll write a list of things I'd like this compiler-as-library to have,
 but please help me make it bigger :-)

 * Don't use global variables (DMD is just thought to be run once, so
 when used as a library it can just be used, well, once)
 * Provide a lexer which gives line numbers and column numbers
 (beginning, end)
 * Provide a parser with the same features
 * The semantic phase should not discard any information found while
 parsing. For example when DMD resolves a type it recursively resolves
 aliasing and keeps the last one. An example:

 alias int foo;
 alias foo* bar;

 bar something() { ... }

 It would be nice if "bar", after semantic analysis is done, carries the
 information that bar is "foo*" and that "foo" is "int". Also that
 something's return type is "bar", not "int*".
 * Provide errors and warnings that have line numbers as well as column
 numbers.
 * Allow to parse the top-level definitions of a module. Whit this I mean
 skipping function bodies. At least Descent first built a the outline of
 the whole project by doing this. This mode should also allow specifying
 a location as a target, and if that location falls inside a function
 body then it's contents are returned (useful when editing a file, so you
 can get the outline as well as semantic info of the function currently
 being edited, which will never affect semantic in other parts of the
 module). This will dramatically speed up the editor.
 * Don't stop parsing on errors (I think DMD already does this).
 * Provide a visitor class. If possible, use visitors to implement
 semantic analysis. The visitor will make it super easy to implement
 lints and to generate documentation.

SDC have a lot of theses, and I proposed a similar stuff for its evolution. I think it is easier for SDC than it is for dmd considering the codebase of both.
Apr 13 2012
next sibling parent reply "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Friday, 13 April 2012 at 13:08:51 UTC, deadalnix wrote:
 SDC have a lot of theses, and I proposed a similar stuff for 
 its evolution. I think it is easier for SDC than it is for dmd 
 considering the codebase of both.

I think we've got the lexer and parser completely separate from most of the rest of the codebase (like the codegen), due to repeated requests from people who wanted to use these parts for IDEs and other tools. I've yet to see anyone actually go through with using it though, possibly because there is no documentation for a lot of it. Documenting these parts fully into something of a public API and then putting it online is definitely on the todo list. Perhaps there would be more motivation to do this rather than work on something else if someone actually tried using SDC in their project instead of just talking about it, so it's kind of a catch-22. That said, the parser is currently evolving alongside the codegen. When we want to start implementing new parts of the language, we iteratively add it to the parser, hence it's not complete. It's very easy to work with though and it's mostly a menial task (although it's kind of fun to produce beautiful parser errors :P). Anyway, for anyone interested, you can find us on Github and #d.sdc on FreeNode.
Apr 13 2012
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-04-13 17:25, Jakob Ovrum wrote:

 I think we've got the lexer and parser completely separate from
 most of the rest of the codebase (like the codegen), due to
 repeated requests from people who wanted to use these parts for
 IDEs and other tools.

Cool.
 I've yet to see anyone actually go through with using it though,
 possibly because there is no documentation for a lot of it.
 Documenting these parts fully into something of a public API and
 then putting it online is definitely on the todo list. Perhaps
 there would be more motivation to do this rather than work on
 something else if someone actually tried using SDC in their
 project instead of just talking about it, so it's kind of a
 catch-22.

That's always a problem.
 That said, the parser is currently evolving alongside the
 codegen. When we want to start implementing new parts of the
 language, we iteratively add it to the parser, hence it's not
 complete. It's very easy to work with though and it's mostly a
 menial task (although it's kind of fun to produce beautiful
 parser errors :P).

 Anyway, for anyone interested, you can find us on Github and
 #d.sdc on FreeNode.

How does it compare to DMD, does it implement the whole language yet? -- /Jacob Carlborg
Apr 13 2012
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-04-14 12:35, Manu wrote:
 On 13 April 2012 18:25, Jakob Ovrum <jakobovrum gmail.com
     That said, the parser is currently evolving alongside the
     codegen. When we want to start implementing new parts of the
     language, we iteratively add it to the parser, hence it's not
     complete. It's very easy to work with though and it's mostly a
     menial task (although it's kind of fun to produce beautiful
     parser errors :P).

     Anyway, for anyone interested, you can find us on Github and
     #d.sdc on FreeNode.


 Just out of curiosity, why would anyone write a code gen these days?
 With projects like LLVM, which can perform great codegen to basically
 any architecture, you'd be crazy not to use that...
 Surely 90% of the value of writing your own D compiler would be the
 unique front end, which may have different design principles (like use
 as a lib, usable on tools and stuff as discussed here) ?

 I'm sorry to say, if you write your own codegen, I would never use your
 compiler. If you use LLVM in the back end, I'll definitely give it a
 look, then merit will depend entirely on the front end (speed,
 flexibility, quality of error messages, runtime error detection, etc).
 I know it's a fun exercise to write a codegen, so from an educational
 perspective, sure, it's valuable to you as a programmer, but I don't
 think it helps your project at all.

I don't know what Jakob means with "codegen" in this case but SDC depends on LLVM. Have a look at the bottom of: https://github.com/bhelyer/SDC -- /Jacob Carlborg
Apr 14 2012
prev sibling next sibling parent reply deadalnix <deadalnix gmail.com> writes:
Le 13/04/2012 22:31, Trass3r a Ă©crit :
 I think we've got the lexer and parser completely separate from
 most of the rest of the codebase (like the codegen), due to
 repeated requests from people who wanted to use these parts for
 IDEs and other tools.

Still some things to learn from Clang though. e.g. it still directly builds an AST instead of using some kind of interface.

I'm very interested in what you mean here. Do you have a link or can you write an explaination ?
 btw, why do you use exceptions for compiler errors instead of a cheaper
 and more sophisticated system?

Currently it is exception only, but this is a known issue.
Apr 14 2012
parent deadalnix <deadalnix gmail.com> writes:
Le 14/04/2012 18:26, Trass3r a Ă©crit :
 Still some things to learn from Clang though.
 e.g. it still directly builds an AST instead of using some kind of
 interface.

I'm very interested in what you mean here. Do you have a link or can you write an explaination ?

Clang decouples the parser from AST construction by letting the parser take an interface class that has a virtual method for everything that is parsed along the lines of ActOnStartOfFunctionDef(). So a client that doesn't need an AST doesn't have to build one. Don't know if it's properly implemented in Clang now though.

Ho I see ! That is brilliant !
Apr 14 2012
prev sibling parent deadalnix <deadalnix gmail.com> writes:
Le 14/04/2012 12:35, Manu a Ă©crit :
 On 13 April 2012 18:25, Jakob Ovrum <jakobovrum gmail.com
 <mailto:jakobovrum gmail.com>> wrote:

     On Friday, 13 April 2012 at 13:08:51 UTC, deadalnix wrote:

         SDC have a lot of theses, and I proposed a similar stuff for its
         evolution. I think it is easier for SDC than it is for dmd
         considering the codebase of both.


     I think we've got the lexer and parser completely separate from
     most of the rest of the codebase (like the codegen), due to
     repeated requests from people who wanted to use these parts for
     IDEs and other tools.

     I've yet to see anyone actually go through with using it though,
     possibly because there is no documentation for a lot of it.
     Documenting these parts fully into something of a public API and
     then putting it online is definitely on the todo list. Perhaps
     there would be more motivation to do this rather than work on
     something else if someone actually tried using SDC in their
     project instead of just talking about it, so it's kind of a
     catch-22.

     That said, the parser is currently evolving alongside the
     codegen. When we want to start implementing new parts of the
     language, we iteratively add it to the parser, hence it's not
     complete. It's very easy to work with though and it's mostly a
     menial task (although it's kind of fun to produce beautiful
     parser errors :P).

     Anyway, for anyone interested, you can find us on Github and
     #d.sdc on FreeNode.


 Just out of curiosity, why would anyone write a code gen these days?
 With projects like LLVM, which can perform great codegen to basically
 any architecture, you'd be crazy not to use that...
 Surely 90% of the value of writing your own D compiler would be the
 unique front end, which may have different design principles (like use
 as a lib, usable on tools and stuff as discussed here) ?

 I'm sorry to say, if you write your own codegen, I would never use your
 compiler. If you use LLVM in the back end, I'll definitely give it a
 look, then merit will depend entirely on the front end (speed,
 flexibility, quality of error messages, runtime error detection, etc).
 I know it's a fun exercise to write a codegen, so from an educational
 perspective, sure, it's valuable to you as a programmer, but I don't
 think it helps your project at all.

Codegen is done by LLVM in SDC, but you still need some codegen glue.
Apr 14 2012
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-04-13 15:10, deadalnix wrote:

 SDC have a lot of theses, and I proposed a similar stuff for its
 evolution. I think it is easier for SDC than it is for dmd considering
 the codebase of both.

I would guess so as well, although I have no experience of SDC. -- /Jacob Carlborg
Apr 13 2012
prev sibling next sibling parent "F i L" <witte2008 gmail.com> writes:
Jacob Carlborg wrote:
 How does it compare to DMD, does it implement the whole 
 language yet?

https://github.com/bhelyer/SDC It lists feature support.
Apr 13 2012
prev sibling next sibling parent Trass3r <un known.com> writes:
 I think we've got the lexer and parser completely separate from
 most of the rest of the codebase (like the codegen), due to
 repeated requests from people who wanted to use these parts for
 IDEs and other tools.

Still some things to learn from Clang though. e.g. it still directly builds an AST instead of using some kind of interface. btw, why do you use exceptions for compiler errors instead of a cheaper and more sophisticated system?
 I've yet to see anyone actually go through with using it though,
 possibly because there is no documentation for a lot of it.

Yep, no comments anywhere :/
Apr 13 2012
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--00248c76910a8c031804bda124ac
Content-Type: text/plain; charset=UTF-8

On 13 April 2012 18:25, Jakob Ovrum <jakobovrum gmail.com> wrote:

 On Friday, 13 April 2012 at 13:08:51 UTC, deadalnix wrote:

 SDC have a lot of theses, and I proposed a similar stuff for its
 evolution. I think it is easier for SDC than it is for dmd considering the
 codebase of both.

I think we've got the lexer and parser completely separate from most of the rest of the codebase (like the codegen), due to repeated requests from people who wanted to use these parts for IDEs and other tools. I've yet to see anyone actually go through with using it though, possibly because there is no documentation for a lot of it. Documenting these parts fully into something of a public API and then putting it online is definitely on the todo list. Perhaps there would be more motivation to do this rather than work on something else if someone actually tried using SDC in their project instead of just talking about it, so it's kind of a catch-22. That said, the parser is currently evolving alongside the codegen. When we want to start implementing new parts of the language, we iteratively add it to the parser, hence it's not complete. It's very easy to work with though and it's mostly a menial task (although it's kind of fun to produce beautiful parser errors :P). Anyway, for anyone interested, you can find us on Github and #d.sdc on FreeNode.

Just out of curiosity, why would anyone write a code gen these days? With projects like LLVM, which can perform great codegen to basically any architecture, you'd be crazy not to use that... Surely 90% of the value of writing your own D compiler would be the unique front end, which may have different design principles (like use as a lib, usable on tools and stuff as discussed here) ? I'm sorry to say, if you write your own codegen, I would never use your compiler. If you use LLVM in the back end, I'll definitely give it a look, then merit will depend entirely on the front end (speed, flexibility, quality of error messages, runtime error detection, etc). I know it's a fun exercise to write a codegen, so from an educational perspective, sure, it's valuable to you as a programmer, but I don't think it helps your project at all. --00248c76910a8c031804bda124ac Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div class=3D"gmail_quote">On 13 April 2012 18:25, Jakob Ovrum <span dir=3D= "ltr">&lt;<a href=3D"mailto:jakobovrum gmail.com">jakobovrum gmail.com</a>&= gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 = 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <div class=3D"im">On Friday, 13 April 2012 at 13:08:51 UTC, deadalnix wrote= :<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> SDC have a lot of theses, and I proposed a similar stuff for its evolution.= I think it is easier for SDC than it is for dmd considering the codebase o= f both.<br> </blockquote> <br></div> I think we&#39;ve got the lexer and parser completely separate from<br> most of the rest of the codebase (like the codegen), due to<br> repeated requests from people who wanted to use these parts for<br> IDEs and other tools.<br> <br> I&#39;ve yet to see anyone actually go through with using it though,<br> possibly because there is no documentation for a lot of it.<br> Documenting these parts fully into something of a public API and<br> then putting it online is definitely on the todo list. Perhaps<br> there would be more motivation to do this rather than work on<br> something else if someone actually tried using SDC in their<br> project instead of just talking about it, so it&#39;s kind of a<br> catch-22.<br> <br> That said, the parser is currently evolving alongside the<br> codegen. When we want to start implementing new parts of the<br> language, we iteratively add it to the parser, hence it&#39;s not<br> complete. It&#39;s very easy to work with though and it&#39;s mostly a<br> menial task (although it&#39;s kind of fun to produce beautiful<br> parser errors :P).<br> <br> Anyway, for anyone interested, you can find us on Github and<br> #d.sdc on FreeNode.<br> </blockquote></div><br><div>Just out of curiosity, why would anyone write a= code gen these days? With projects like LLVM, which can perform great code= gen to basically any architecture, you&#39;d be crazy not to use that...</d= iv> <div>Surely 90% of the value of writing your own D compiler would be the un= ique front end, which may have different design principles (like use as a l= ib, usable on tools and stuff as discussed here) ?</div><div><br></div> <div>I&#39;m sorry to say, if you write your own codegen, I would never use= your compiler. If you use LLVM in the back end, I&#39;ll definitely give i= t a look, then merit will depend entirely on the front end (speed, flexibil= ity, quality of error messages, runtime error detection, etc).</div> <div>I know it&#39;s a fun exercise to write a codegen, so from an educatio= nal perspective, sure, it&#39;s valuable to you as a programmer, but I don&= #39;t think it helps your project at all.</div> --00248c76910a8c031804bda124ac--
Apr 14 2012
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--20cf300e4de7c6e7fd04bda5b33b
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On 14 April 2012 16:51, deadalnix <deadalnix gmail.com> wrote:

 Le 14/04/2012 12:35, Manu a =C3=A9crit :

 On 13 April 2012 18:25, Jakob Ovrum <jakobovrum gmail.com
 <mailto:jakobovrum gmail.com>> wrote:

    On Friday, 13 April 2012 at 13:08:51 UTC, deadalnix wrote:

        SDC have a lot of theses, and I proposed a similar stuff for its
        evolution. I think it is easier for SDC than it is for dmd
        considering the codebase of both.


    I think we've got the lexer and parser completely separate from
    most of the rest of the codebase (like the codegen), due to
    repeated requests from people who wanted to use these parts for
    IDEs and other tools.

    I've yet to see anyone actually go through with using it though,
    possibly because there is no documentation for a lot of it.
    Documenting these parts fully into something of a public API and
    then putting it online is definitely on the todo list. Perhaps
    there would be more motivation to do this rather than work on
    something else if someone actually tried using SDC in their
    project instead of just talking about it, so it's kind of a
    catch-22.

    That said, the parser is currently evolving alongside the
    codegen. When we want to start implementing new parts of the
    language, we iteratively add it to the parser, hence it's not
    complete. It's very easy to work with though and it's mostly a
    menial task (although it's kind of fun to produce beautiful
    parser errors :P).

    Anyway, for anyone interested, you can find us on Github and
    #d.sdc on FreeNode.


 Just out of curiosity, why would anyone write a code gen these days?
 With projects like LLVM, which can perform great codegen to basically
 any architecture, you'd be crazy not to use that...
 Surely 90% of the value of writing your own D compiler would be the
 unique front end, which may have different design principles (like use
 as a lib, usable on tools and stuff as discussed here) ?

 I'm sorry to say, if you write your own codegen, I would never use your
 compiler. If you use LLVM in the back end, I'll definitely give it a
 look, then merit will depend entirely on the front end (speed,
 flexibility, quality of error messages, runtime error detection, etc).
 I know it's a fun exercise to write a codegen, so from an educational
 perspective, sure, it's valuable to you as a programmer, but I don't
 think it helps your project at all.

Codegen is done by LLVM in SDC, but you still need some codegen glue.

Ah okay, cool. NM me then :P --20cf300e4de7c6e7fd04bda5b33b Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div class=3D"gmail_quote">On 14 April 2012 16:51, deadalnix <span dir=3D"l= tr">&lt;<a href=3D"mailto:deadalnix gmail.com">deadalnix gmail.com</a>&gt;<= /span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8= ex;border-left:1px #ccc solid;padding-left:1ex"> Le 14/04/2012 12:35, Manu a =C3=A9crit :<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"><div class=3D"im"> On 13 April 2012 18:25, Jakob Ovrum &lt;<a href=3D"mailto:jakobovrum gmail.= com" target=3D"_blank">jakobovrum gmail.com</a><br></div><div><div class=3D= "h5"> &lt;mailto:<a href=3D"mailto:jakobovrum gmail.com" target=3D"_blank">jakobo= vrum gmail.com</a>&gt;&gt; wrote:<br> <br> =C2=A0 =C2=A0On Friday, 13 April 2012 at 13:08:51 UTC, deadalnix wrote:<br=

=C2=A0 =C2=A0 =C2=A0 =C2=A0SDC have a lot of theses, and I proposed a simi= lar stuff for its<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0evolution. I think it is easier for SDC than it= is for dmd<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0considering the codebase of both.<br> <br> <br> =C2=A0 =C2=A0I think we&#39;ve got the lexer and parser completely separat= e from<br> =C2=A0 =C2=A0most of the rest of the codebase (like the codegen), due to<b= r> =C2=A0 =C2=A0repeated requests from people who wanted to use these parts f= or<br> =C2=A0 =C2=A0IDEs and other tools.<br> <br> =C2=A0 =C2=A0I&#39;ve yet to see anyone actually go through with using it = though,<br> =C2=A0 =C2=A0possibly because there is no documentation for a lot of it.<b= r> =C2=A0 =C2=A0Documenting these parts fully into something of a public API = and<br> =C2=A0 =C2=A0then putting it online is definitely on the todo list. Perhap= s<br> =C2=A0 =C2=A0there would be more motivation to do this rather than work on= <br> =C2=A0 =C2=A0something else if someone actually tried using SDC in their<b= r> =C2=A0 =C2=A0project instead of just talking about it, so it&#39;s kind of= a<br> =C2=A0 =C2=A0catch-22.<br> <br> =C2=A0 =C2=A0That said, the parser is currently evolving alongside the<br> =C2=A0 =C2=A0codegen. When we want to start implementing new parts of the<= br> =C2=A0 =C2=A0language, we iteratively add it to the parser, hence it&#39;s= not<br> =C2=A0 =C2=A0complete. It&#39;s very easy to work with though and it&#39;s= mostly a<br> =C2=A0 =C2=A0menial task (although it&#39;s kind of fun to produce beautif= ul<br> =C2=A0 =C2=A0parser errors :P).<br> <br> =C2=A0 =C2=A0Anyway, for anyone interested, you can find us on Github and<= br> =C2=A0 =C2=A0#d.sdc on FreeNode.<br> <br> <br></div></div><div class=3D"im"> Just out of curiosity, why would anyone write a code gen these days?<br> With projects like LLVM, which can perform great codegen to basically<br> any architecture, you&#39;d be crazy not to use that...<br> Surely 90% of the value of writing your own D compiler would be the<br> unique front end, which may have different design principles (like use<br> as a lib, usable on tools and stuff as discussed here) ?<br> <br> I&#39;m sorry to say, if you write your own codegen, I would never use your= <br> compiler. If you use LLVM in the back end, I&#39;ll definitely give it a<br=

flexibility, quality of error messages, runtime error detection, etc).<br> I know it&#39;s a fun exercise to write a codegen, so from an educational<b= r> perspective, sure, it&#39;s valuable to you as a programmer, but I don&#39;= t<br> think it helps your project at all.<br> </div></blockquote> <br> Codegen is done by LLVM in SDC, but you still need some codegen glue.<br> </blockquote></div><br><div>Ah okay, cool. NM me then :P</div> --20cf300e4de7c6e7fd04bda5b33b--
Apr 14 2012
prev sibling next sibling parent Trass3r <un known.com> writes:
 Still some things to learn from Clang though.
 e.g. it still directly builds an AST instead of using some kind of
 interface.

I'm very interested in what you mean here. Do you have a link or can you write an explaination ?

Clang decouples the parser from AST construction by letting the parser take an interface class that has a virtual method for everything that is parsed along the lines of ActOnStartOfFunctionDef(). So a client that doesn't need an AST doesn't have to build one. Don't know if it's properly implemented in Clang now though.
Apr 14 2012
prev sibling next sibling parent reply Ary Manzana <ary esperanto.org.ar> writes:
On 4/13/12 9:10 PM, deadalnix wrote:
 Le 13/04/2012 11:58, Ary Manzana a écrit :
 Having a D compiler available as a library will (at least) give these
 benefits:

 1. Can be used by an IDE: D is statically typed and so an IDE can
 benefit a lot from this. The features Descent had, as far as I remember,
 were:
 1.1. Outline
 1.2. Autocompletion
 1.3. Type Hierarchy
 1.4. Syntax and semantic errors, showing not only the line number but
 also column numbers if it makes sense
 1.5. Automatic import inclusion (say, typing writefln and getting a list
 of modules that provide that symbol)
 1.6. Compile-time view: replace auto with the inferred type, insert
 mixins into scope, rewrite operator overloads and other lowerings (but
 I'm not sure this point is really useful)
 1.7. Determine, given a set of versions and flags, which branches of
 static ifs are used/unused
 1.8. Open declaration
 1.9. Show implementations (of an interface, of interface's method or,
 abstract methods, or method overrides).
 1.10. Propose to override a method (you type some letters and then hit
 some key combination and get a list of methods to override)
 1.11. Get the code of a template when instantiated.
 2. Can be used to build better doc generators: one that shows known
 subclasses or interface implementation, shows inherited methods, type
 hierarchy.
 3. Can be used for lints and other such tools.

 As you can see, a simple lexer/parser built into an IDE, doc generator
 or lint will just give basic features but will never achieve something
 exceptionally good if it lacks the full semantic knowledge of the code.

 I'll write a list of things I'd like this compiler-as-library to have,
 but please help me make it bigger :-)

 * Don't use global variables (DMD is just thought to be run once, so
 when used as a library it can just be used, well, once)
 * Provide a lexer which gives line numbers and column numbers
 (beginning, end)
 * Provide a parser with the same features
 * The semantic phase should not discard any information found while
 parsing. For example when DMD resolves a type it recursively resolves
 aliasing and keeps the last one. An example:

 alias int foo;
 alias foo* bar;

 bar something() { ... }

 It would be nice if "bar", after semantic analysis is done, carries the
 information that bar is "foo*" and that "foo" is "int". Also that
 something's return type is "bar", not "int*".
 * Provide errors and warnings that have line numbers as well as column
 numbers.
 * Allow to parse the top-level definitions of a module. Whit this I mean
 skipping function bodies. At least Descent first built a the outline of
 the whole project by doing this. This mode should also allow specifying
 a location as a target, and if that location falls inside a function
 body then it's contents are returned (useful when editing a file, so you
 can get the outline as well as semantic info of the function currently
 being edited, which will never affect semantic in other parts of the
 module). This will dramatically speed up the editor.
 * Don't stop parsing on errors (I think DMD already does this).
 * Provide a visitor class. If possible, use visitors to implement
 semantic analysis. The visitor will make it super easy to implement
 lints and to generate documentation.

SDC have a lot of theses, and I proposed a similar stuff for its evolution. I think it is easier for SDC than it is for dmd considering the codebase of both.

Cool! SDC is the way to go. Let's focus our efforts on that project. :-) One thing I saw in the code is that global variables are used in it (specially in the sdc.aglobal module). Also, the lexer returns a TokenStream which contains the full array of tokens. This is very slow compared to returning them as they are scanned, when requested. But I guess this is easy to fix as it's hidden behind the TokenStream interface. I tried to compile a simple file: void main() {}
 ./bin/sdc main.d

---------------- 5 sdc 0x000000010b87aa2a _d_assertm + 42 6 sdc 0x000000010b4e2daa void sdc.gen.sdcmodule.__assert(int) + 26 7 sdc 0x000000010b55d19c void sdc.gen.sdcmodule.Store.addFunction(sdc.gen.sdcfunction.Function) + 92 8 sdc 0x000000010b55d75b void sdc.gen.sdcmodule.Scope.add(immutable(char)[], sdc.gen.sdcfunction.Function) + 123 9 sdc 0x000000010b54ee36 void sdc.gen.declaration.declareFunctionDeclaration(sdc.ast.declaration. unctionDeclaration, sdc.ast.sdcmodule.DeclarationDefinition, sdc.gen.sdcmodule.Module) + 1278 10 sdc 0x000000010b54e4b2 void sdc.gen.declaration.declareDeclaration(sdc.ast.declaration.Declaration, sdc.ast.sdcmodule.DeclarationDefinition, sdc.gen.sdcmodule.Module) + 146 11 sdc 0x000000010b54d12e void sdc.gen.base.genDeclarationDefinition(sdc.ast.sdcmodule.DeclarationDefinition, sdc.gen.sdcmodule.Module, ulong) + 374 12 sdc 0x000000010b54c8d0 void sdc.gen.base.resolveDeclarationDefinitionList(sdc.ast.sdcmodule.Decl rationDefinition[], sdc.gen.sdcmodule.Module, sdc.gen.type.Type) + 564 13 sdc 0x000000010b54c439 sdc.gen.sdcmodule.Module sdc.gen.base.genModule(sdc.ast.sdcmodule.Module, sdc.aglobal.TranslationUnit) + 157 14 sdc 0x000000010b53432d int sdc.sdc.realmain(immutable(char)[][]) + 3081 15 sdc 0x000000010b4e283e _Dmain + 54 16 sdc 0x000000010b87b415 extern (C) int rt.dmain2.main(int, char**).void runMain() + 29 17 sdc 0x000000010b87adc5 extern (C) int rt.dmain2.main(int, char**).void tryExec(scope void delegate()) + 45 18 sdc 0x000000010b87b467 extern (C) int rt.dmain2.main(int, char**).void runAll() + 63 19 sdc 0x000000010b87adc5 extern (C) int rt.dmain2.main(int, char**).void tryExec(scope void delegate()) + 45 20 sdc 0x000000010b87ad48 main + 232 21 sdc 0x000000010b4e2694 start + 52 22 ??? 0x0000000000000002 0x0 + 2 ---------------- I couldn't find the line of the assertion. What are all those "+146" and "+54" about?
Apr 15 2012
parent reply Jacob Carlborg <doob me.com> writes:
On 2012-04-16 04:35, Ary Manzana wrote:
 On 4/13/12 9:10 PM, deadalnix wrote:
 SDC have a lot of theses, and I proposed a similar stuff for its
 evolution. I think it is easier for SDC than it is for dmd considering
 the codebase of both.

Cool! SDC is the way to go. Let's focus our efforts on that project. :-)

I noticed that SDC is licensed under the GPL license :( That is not compatible with, for example, EPL used by Eclipse. -- /Jacob Carlborg
Apr 15 2012
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-04-16 09:00, Jakob Ovrum wrote:

 We are changing the license soon, with BSD/MIT in mind. I am really just
 waiting for Bernard to make the change, we've gotten permission from all
 contributors as far as I know.

Ok, that sounds good. I would prefer Boost but I guess BSD/MIT is also ok. -- /Jacob Carlborg
Apr 16 2012
prev sibling next sibling parent reply deadalnix <deadalnix gmail.com> writes:
Le 16/04/2012 09:00, Jakob Ovrum a Ă©crit :
 On Monday, 16 April 2012 at 06:37:00 UTC, Jacob Carlborg wrote:
 On 2012-04-16 04:35, Ary Manzana wrote:
 On 4/13/12 9:10 PM, deadalnix wrote:
 SDC have a lot of theses, and I proposed a similar stuff for its
 evolution. I think it is easier for SDC than it is for dmd considering
 the codebase of both.

Cool! SDC is the way to go. Let's focus our efforts on that project. :-)

I noticed that SDC is licensed under the GPL license :( That is not compatible with, for example, EPL used by Eclipse.

We are changing the license soon, with BSD/MIT in mind. I am really just waiting for Bernard to make the change, we've gotten permission from all contributors as far as I know.

You didn't asked me, but now you have mine. (Anyway, I'm not a big contributor, and only have submitted some small patches).
Apr 16 2012
parent Somedude <lovelydear mailmetrash.com> writes:
Le 16/04/2012 13:58, Bernard Helyer a Ă©crit :
 
 I recall mentioning it via IRC. Which is hardly going to stand up
 in a court. :V
 

has to be taken seriously, I'm affraid....
Apr 19 2012
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-04-16 14:14, Bernard Helyer wrote:

 https://github.com/bhelyer/SDC/commit/2002ec27e8df9144f0d1051032abbfe24d2ad4d2


 Go nuts.

Awesome :) -- /Jacob Carlborg
Apr 16 2012
prev sibling parent reply "Nick Sabalausky" <SeeWebsiteToContactMe semitwist.com> writes:
"David Nadlinger" <see klickverbot.at> wrote in message 
news:voymctvtskltfzhslhkp forum.dlang.org...
 On Monday, 16 April 2012 at 11:57:29 UTC, Bernard Helyer wrote:
 Oh god, what did we decide on? Boost?

Imho, MIT is just fine for a compiler. It's not like a line of attribution is much to ask when you are getting an entire compiler fronend for free - in fact, applications should probably mention the compiler frontend they use in the docs anyway in case there are incompatibilities, .

There's no attribution in MIT.
Apr 19 2012
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
Le 19/04/2012 17:23, Roman D. Boiko a Ă©crit :
 On Thursday, 19 April 2012 at 15:11:50 UTC, David Nadlinger wrote:
 On Thursday, 19 April 2012 at 10:15:36 UTC, Roman D. Boiko wrote:
 Actually, I prefer Boost only because it is slightly more popular […]

Not to argue about the Boost license being popular in the D community, and not that the question would really matter, but what leads you to this general conclusion? I couldn't find any credible statistics on a quick Google search, but a numer of well known projects use the/a MIT license (X, Ruby on Rails, Mono, Lua, …). David

I wish I could delete that post :) My claim is not based on any research. However, I prefer Boost because: "The Boost Software License is based upon the MIT license, but differs from the MIT license in that it: (i) makes clear that licenses can be granted to organizations as well as individuals; (ii) does not require that the license appear with executables or other binary uses of the library; (iii) expressly disclaims -- on behalf of the author and copyright holders of the software only -- the warranty of title (a warranty that, under the Uniform Commercial Code, is separate from the warranty of non-infringement) (iv) does not extend the disclaimer of warranties to licensees, so that they may, if they choose, undertake such warranties (e.g., in exchange for payment)." http://ideas.opensource.org/ticket/45

Very good point. Is it too late to change again ? By the way, what is the status of the attribution clause ?
Apr 19 2012
parent Mirko Pilger <pilger cymotec.de> writes:
 If it will be decided to change the license, please pay attention to how
 should programmers apply the license to source and header files:

or do it the way it is done in phobos.
Apr 19 2012
prev sibling next sibling parent reply "Nick Sabalausky" <SeeWebsiteToContactMe semitwist.com> writes:
"Roman D. Boiko" <rb d-coding.com> wrote in message 
news:gyhkcrhkaedsjzoohrvp forum.dlang.org...
 Motivation for Boost would be reducing the number of licenses that the 
 code author must know. Also, here are some differences between these 
 licenses, which I consider as Boost advantages:

MIT's *much* easier to understand though. Boost has some real goofy, obfuscated wordings. Although it's *worlds* better in that regard than the completely impenatrable GPL or Creative Commons.
 "The Boost Software License is based upon the MIT license, but differs 
 from the MIT license in that it:

 (i) makes clear that licenses can be granted to organizations as well as 
 individuals;

 (ii) does not require that the license appear with executables or other 
 binary uses of the library;

My favorite license, zlib ( http://www.opensource.org/licenses/Zlib ) doesn't have this #2 issue, and it's even easier to read and understand than MIT. Plus it doesn't say anything like "to any person", so it should take care of #1, too. (Although personally, I think I like MIT better in that regard: It's a deterrent against corporations, which gives it a little bit of the benefit of the GPL, but without all the bullshit.)
 (iii) expressly disclaims -- on behalf of the author and copyright holders 
 of the software only -- the warranty of title (a warranty that, under the 
 Uniform Commercial Code, is separate from the warranty of 
 non-infringement)

 (iv) does not extend the disclaimer of warranties to licensees, so that 
 they may, if they choose, undertake such warranties (e.g., in exchange for 
 payment)."

 http://ideas.opensource.org/ticket/45 

Apr 19 2012
parent Jacob Carlborg <doob me.com> writes:
On 2012-04-19 21:54, Nick Sabalausky wrote:

 MIT's *much* easier to understand though. Boost has some real goofy,
 obfuscated wordings. Although it's *worlds* better in that regard than the
 completely impenatrable GPL or Creative Commons.

What I like about the Boost license is that it says explicitly that no attribution is needed for binary distribution. The zlib/libpng license on the other hand says so implicitly. -- /Jacob Carlborg
Apr 19 2012
prev sibling parent "Nick Sabalausky" <SeeWebsiteToContactMe semitwist.com> writes:
"David Nadlinger" <see klickverbot.at> wrote in message 
news:ebhaxqxtjuccyqfyapfo forum.dlang.org...
 On Thursday, 19 April 2012 at 09:58:52 UTC, Nick Sabalausky wrote:
 There's no attribution in MIT.

»The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software« - I am not a lawyer, but I what makes you so sure that binaries don't fall under this requirement as well? The Boost people also seem to interpret MIT as requiring binary attribution; Dave Abrahams mentioned the binary clause in Boost as a difference to MIT in his OSI approval request at least ([1]).

I'm pretty sure that "attribution" typically refers to the requirement that software X which *uses* software Y must include notice that software Y is being used. But you're right that MIT doesn't make it clear whether binary-only redistributions of software Y must contain the license and copyright notice.
Apr 19 2012
prev sibling next sibling parent "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Monday, 16 April 2012 at 06:37:00 UTC, Jacob Carlborg wrote:
 On 2012-04-16 04:35, Ary Manzana wrote:
 On 4/13/12 9:10 PM, deadalnix wrote:
 SDC have a lot of theses, and I proposed a similar stuff for 
 its
 evolution. I think it is easier for SDC than it is for dmd 
 considering
 the codebase of both.

Cool! SDC is the way to go. Let's focus our efforts on that project. :-)

I noticed that SDC is licensed under the GPL license :( That is not compatible with, for example, EPL used by Eclipse.

We are changing the license soon, with BSD/MIT in mind. I am really just waiting for Bernard to make the change, we've gotten permission from all contributors as far as I know.
Apr 16 2012
prev sibling next sibling parent "Bernard Helyer" <b.helyer gmail.com> writes:
On Monday, 16 April 2012 at 07:00:19 UTC, Jakob Ovrum wrote:
 On Monday, 16 April 2012 at 06:37:00 UTC, Jacob Carlborg wrote:
 On 2012-04-16 04:35, Ary Manzana wrote:
 On 4/13/12 9:10 PM, deadalnix wrote:
 SDC have a lot of theses, and I proposed a similar stuff for 
 its
 evolution. I think it is easier for SDC than it is for dmd 
 considering
 the codebase of both.

Cool! SDC is the way to go. Let's focus our efforts on that project. :-)

I noticed that SDC is licensed under the GPL license :( That is not compatible with, for example, EPL used by Eclipse.

We are changing the license soon, with BSD/MIT in mind. I am really just waiting for Bernard to make the change, we've gotten permission from all contributors as far as I know.

Oh god, what did we decide on? Boost?
Apr 16 2012
prev sibling next sibling parent "Bernard Helyer" <b.helyer gmail.com> writes:
On Monday, 16 April 2012 at 11:34:14 UTC, deadalnix wrote:
 Le 16/04/2012 09:00, Jakob Ovrum a Ă©crit :
 On Monday, 16 April 2012 at 06:37:00 UTC, Jacob Carlborg wrote:
 On 2012-04-16 04:35, Ary Manzana wrote:
 On 4/13/12 9:10 PM, deadalnix wrote:
 SDC have a lot of theses, and I proposed a similar stuff 
 for its
 evolution. I think it is easier for SDC than it is for dmd 
 considering
 the codebase of both.

Cool! SDC is the way to go. Let's focus our efforts on that project. :-)

I noticed that SDC is licensed under the GPL license :( That is not compatible with, for example, EPL used by Eclipse.

We are changing the license soon, with BSD/MIT in mind. I am really just waiting for Bernard to make the change, we've gotten permission from all contributors as far as I know.

You didn't asked me, but now you have mine. (Anyway, I'm not a big contributor, and only have submitted some small patches).

I recall mentioning it via IRC. Which is hardly going to stand up in a court. :V
Apr 16 2012
prev sibling next sibling parent "Bernard Helyer" <b.helyer gmail.com> writes:
On Monday, 16 April 2012 at 06:37:00 UTC, Jacob Carlborg wrote:
 On 2012-04-16 04:35, Ary Manzana wrote:
 On 4/13/12 9:10 PM, deadalnix wrote:
 SDC have a lot of theses, and I proposed a similar stuff for 
 its
 evolution. I think it is easier for SDC than it is for dmd 
 considering
 the codebase of both.

Cool! SDC is the way to go. Let's focus our efforts on that project. :-)

I noticed that SDC is licensed under the GPL license :( That is not compatible with, for example, EPL used by Eclipse.

https://github.com/bhelyer/SDC/commit/2002ec27e8df9144f0d1051032abbfe24d2ad4d2 Go nuts.
Apr 16 2012
prev sibling next sibling parent "Bernard Helyer" <b.helyer gmail.com> writes:
On Monday, 16 April 2012 at 02:34:29 UTC, Ary Manzana wrote:
 On 4/13/12 9:10 PM, deadalnix wrote:
 Le 13/04/2012 11:58, Ary Manzana a Ă©crit :
 Having a D compiler available as a library will (at least) 
 give these
 benefits:

 1. Can be used by an IDE: D is statically typed and so an IDE 
 can
 benefit a lot from this. The features Descent had, as far as 
 I remember,
 were:
 1.1. Outline
 1.2. Autocompletion
 1.3. Type Hierarchy
 1.4. Syntax and semantic errors, showing not only the line 
 number but
 also column numbers if it makes sense
 1.5. Automatic import inclusion (say, typing writefln and 
 getting a list
 of modules that provide that symbol)
 1.6. Compile-time view: replace auto with the inferred type, 
 insert
 mixins into scope, rewrite operator overloads and other 
 lowerings (but
 I'm not sure this point is really useful)
 1.7. Determine, given a set of versions and flags, which 
 branches of
 static ifs are used/unused
 1.8. Open declaration
 1.9. Show implementations (of an interface, of interface's 
 method or,
 abstract methods, or method overrides).
 1.10. Propose to override a method (you type some letters and 
 then hit
 some key combination and get a list of methods to override)
 1.11. Get the code of a template when instantiated.
 2. Can be used to build better doc generators: one that shows 
 known
 subclasses or interface implementation, shows inherited 
 methods, type
 hierarchy.
 3. Can be used for lints and other such tools.

 As you can see, a simple lexer/parser built into an IDE, doc 
 generator
 or lint will just give basic features but will never achieve 
 something
 exceptionally good if it lacks the full semantic knowledge of 
 the code.

 I'll write a list of things I'd like this compiler-as-library 
 to have,
 but please help me make it bigger :-)

 * Don't use global variables (DMD is just thought to be run 
 once, so
 when used as a library it can just be used, well, once)
 * Provide a lexer which gives line numbers and column numbers
 (beginning, end)
 * Provide a parser with the same features
 * The semantic phase should not discard any information found 
 while
 parsing. For example when DMD resolves a type it recursively 
 resolves
 aliasing and keeps the last one. An example:

 alias int foo;
 alias foo* bar;

 bar something() { ... }

 It would be nice if "bar", after semantic analysis is done, 
 carries the
 information that bar is "foo*" and that "foo" is "int". Also 
 that
 something's return type is "bar", not "int*".
 * Provide errors and warnings that have line numbers as well 
 as column
 numbers.
 * Allow to parse the top-level definitions of a module. Whit 
 this I mean
 skipping function bodies. At least Descent first built a the 
 outline of
 the whole project by doing this. This mode should also allow 
 specifying
 a location as a target, and if that location falls inside a 
 function
 body then it's contents are returned (useful when editing a 
 file, so you
 can get the outline as well as semantic info of the function 
 currently
 being edited, which will never affect semantic in other parts 
 of the
 module). This will dramatically speed up the editor.
 * Don't stop parsing on errors (I think DMD already does 
 this).
 * Provide a visitor class. If possible, use visitors to 
 implement
 semantic analysis. The visitor will make it super easy to 
 implement
 lints and to generate documentation.

SDC have a lot of theses, and I proposed a similar stuff for its evolution. I think it is easier for SDC than it is for dmd considering the codebase of both.

Cool! SDC is the way to go. Let's focus our efforts on that project. :-) One thing I saw in the code is that global variables are used in it (specially in the sdc.aglobal module). Also, the lexer returns a TokenStream which contains the full array of tokens. This is very slow compared to returning them as they are scanned, when requested. But I guess this is easy to fix as it's hidden behind the TokenStream interface. I tried to compile a simple file: void main() {}
 ./bin/sdc main.d

failure ---------------- 5 sdc 0x000000010b87aa2a _d_assertm + 42 6 sdc 0x000000010b4e2daa void sdc.gen.sdcmodule.__assert(int) + 26 7 sdc 0x000000010b55d19c void sdc.gen.sdcmodule.Store.addFunction(sdc.gen.sdcfunction.Function) + 92 8 sdc 0x000000010b55d75b void sdc.gen.sdcmodule.Scope.add(immutable(char)[], sdc.gen.sdcfunction.Function) + 123 9 sdc 0x000000010b54ee36 void sdc.gen.declaration.declareFunctionDeclaration(sdc.ast.declaration. unctionDeclaration, sdc.ast.sdcmodule.DeclarationDefinition, sdc.gen.sdcmodule.Module) + 1278 10 sdc 0x000000010b54e4b2 void sdc.gen.declaration.declareDeclaration(sdc.ast.declaration.Declaration, sdc.ast.sdcmodule.DeclarationDefinition, sdc.gen.sdcmodule.Module) + 146 11 sdc 0x000000010b54d12e void sdc.gen.base.genDeclarationDefinition(sdc.ast.sdcmodule.DeclarationDefinition, sdc.gen.sdcmodule.Module, ulong) + 374 12 sdc 0x000000010b54c8d0 void sdc.gen.base.resolveDeclarationDefinitionList(sdc.ast.sdcmodule.Decl rationDefinition[], sdc.gen.sdcmodule.Module, sdc.gen.type.Type) + 564 13 sdc 0x000000010b54c439 sdc.gen.sdcmodule.Module sdc.gen.base.genModule(sdc.ast.sdcmodule.Module, sdc.aglobal.TranslationUnit) + 157 14 sdc 0x000000010b53432d int sdc.sdc.realmain(immutable(char)[][]) + 3081 15 sdc 0x000000010b4e283e _Dmain + 54 16 sdc 0x000000010b87b415 extern (C) int rt.dmain2.main(int, char**).void runMain() + 29 17 sdc 0x000000010b87adc5 extern (C) int rt.dmain2.main(int, char**).void tryExec(scope void delegate()) + 45 18 sdc 0x000000010b87b467 extern (C) int rt.dmain2.main(int, char**).void runAll() + 63 19 sdc 0x000000010b87adc5 extern (C) int rt.dmain2.main(int, char**).void tryExec(scope void delegate()) + 45 20 sdc 0x000000010b87ad48 main + 232 21 sdc 0x000000010b4e2694 start + 52 22 ??? 0x0000000000000002 0x0 + 2 ----------------

https://github.com/bhelyer/SDC/issues/30 Rename main.d to something else and it should work. Gotta fix that. >_<
Apr 16 2012
prev sibling next sibling parent "Roman D. Boiko" <rb d-coding.com> writes:
On Monday, 16 April 2012 at 11:57:29 UTC, Bernard Helyer wrote:
 On Monday, 16 April 2012 at 07:00:19 UTC, Jakob Ovrum wrote:
 On Monday, 16 April 2012 at 06:37:00 UTC, Jacob Carlborg wrote:
 On 2012-04-16 04:35, Ary Manzana wrote:
 On 4/13/12 9:10 PM, deadalnix wrote:
 SDC have a lot of theses, and I proposed a similar stuff 
 for its
 evolution. I think it is easier for SDC than it is for dmd 
 considering
 the codebase of both.

Cool! SDC is the way to go. Let's focus our efforts on that project. :-)

I noticed that SDC is licensed under the GPL license :( That is not compatible with, for example, EPL used by Eclipse.

We are changing the license soon, with BSD/MIT in mind. I am really just waiting for Bernard to make the change, we've gotten permission from all contributors as far as I know.

Oh god, what did we decide on? Boost?

MIT is great, it is a huge step forward with respect to commercial reuse of code. However, Boost would allow inclusion of sources (e.g., lexer/parser) into D standard library (I saw it has been mentioned somewhere that such functionality is planned, probably by Andrei Alexandrescu). Of course, before inclusion it might be rewritten significantly, but at least it could be based on SDC or whatever other implementation with Boost license. What was the motivation behind selecting MIT?
Apr 19 2012
prev sibling next sibling parent "David Nadlinger" <see klickverbot.at> writes:
On Monday, 16 April 2012 at 11:57:29 UTC, Bernard Helyer wrote:
 Oh god, what did we decide on? Boost?

Imho, MIT is just fine for a compiler. It's not like a line of attribution is much to ask when you are getting an entire compiler fronend for free – in fact, applications should probably mention the compiler frontend they use in the docs anyway in case there are incompatibilities, … David
Apr 19 2012
prev sibling next sibling parent "Roman D. Boiko" <rb d-coding.com> writes:
On Thursday, 19 April 2012 at 09:58:52 UTC, Nick Sabalausky wrote:
 "David Nadlinger" <see klickverbot.at> wrote in message
 news:voymctvtskltfzhslhkp forum.dlang.org...
 On Monday, 16 April 2012 at 11:57:29 UTC, Bernard Helyer wrote:
 Oh god, what did we decide on? Boost?

Imho, MIT is just fine for a compiler. It's not like a line of attribution is much to ask when you are getting an entire compiler fronend for free - in fact, applications should probably mention the compiler frontend they use in the docs anyway in case there are incompatibilities, .

There's no attribution in MIT.

Yes, there is no attribution: https://github.com/roman-d-boiko/SDC/blob/master/LICENCE. It is very similar to Boost. Actually, I prefer Boost only because it is slightly more popular and because the D standard library uses it. I'm perfectly happy with MIT, though, and already cloned SDC :) Just interested about motivation behind choosing MIT.
Apr 19 2012
prev sibling next sibling parent "Roman D. Boiko" <rb d-coding.com> writes:
On Thursday, 19 April 2012 at 10:15:36 UTC, Roman D. Boiko wrote:
 On Thursday, 19 April 2012 at 09:58:52 UTC, Nick Sabalausky 
 wrote:
 "David Nadlinger" <see klickverbot.at> wrote in message
 news:voymctvtskltfzhslhkp forum.dlang.org...
 On Monday, 16 April 2012 at 11:57:29 UTC, Bernard Helyer 
 wrote:
 Oh god, what did we decide on? Boost?

Imho, MIT is just fine for a compiler. It's not like a line of attribution is much to ask when you are getting an entire compiler fronend for free - in fact, applications should probably mention the compiler frontend they use in the docs anyway in case there are incompatibilities, .

There's no attribution in MIT.

Yes, there is no attribution: https://github.com/roman-d-boiko/SDC/blob/master/LICENCE. It is very similar to Boost. Actually, I prefer Boost only because it is slightly more popular and because the D standard library uses it. I'm perfectly happy with MIT, though, and already cloned SDC :) Just interested about motivation behind choosing MIT.

Motivation for Boost would be reducing the number of licenses that the code author must know. Also, here are some differences between these licenses, which I consider as Boost advantages: "The Boost Software License is based upon the MIT license, but differs from the MIT license in that it: (i) makes clear that licenses can be granted to organizations as well as individuals; (ii) does not require that the license appear with executables or other binary uses of the library; (iii) expressly disclaims -- on behalf of the author and copyright holders of the software only -- the warranty of title (a warranty that, under the Uniform Commercial Code, is separate from the warranty of non-infringement) (iv) does not extend the disclaimer of warranties to licensees, so that they may, if they choose, undertake such warranties (e.g., in exchange for payment)." http://ideas.opensource.org/ticket/45
Apr 19 2012
prev sibling next sibling parent "Roman D. Boiko" <rb d-coding.com> writes:
On Thursday, 19 April 2012 at 11:01:10 UTC, Somedude wrote:
 Le 16/04/2012 13:58, Bernard Helyer a Ă©crit :
 
 I recall mentioning it via IRC. Which is hardly going to stand 
 up
 in a court. :V
 

license, this has to be taken seriously, I'm affraid....

Definitely. There are actually many stories to learn from. E.g.: http://en.wikipedia.org/wiki/SCO_v._IBM (and more at http://en.wikipedia.org/wiki/SCO_Group), http://blogs.computerworlduk.com/open-enterprise/2010/09/could-this-lawsuit-undermine-the-gnu-gpl/index.htm
Apr 19 2012
prev sibling next sibling parent "David Nadlinger" <see klickverbot.at> writes:
On Thursday, 19 April 2012 at 09:58:52 UTC, Nick Sabalausky wrote:
 There's no attribution in MIT.

»The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software« – I am not a lawyer, but I what makes you so sure that binaries don't fall under this requirement as well? The Boost people also seem to interpret MIT as requiring binary attribution; Dave Abrahams mentioned the binary clause in Boost as a difference to MIT in his OSI approval request at least ([1]). David [1] http://ideas.opensource.org/ticket/45
Apr 19 2012
prev sibling next sibling parent "David Nadlinger" <see klickverbot.at> writes:
On Thursday, 19 April 2012 at 10:15:36 UTC, Roman D. Boiko wrote:
 Actually, I prefer Boost only because it is slightly more 
 popular […]

Not to argue about the Boost license being popular in the D community, and not that the question would really matter, but what leads you to this general conclusion? I couldn't find any credible statistics on a quick Google search, but a numer of well known projects use the/a MIT license (X, Ruby on Rails, Mono, Lua, …). David
Apr 19 2012
prev sibling next sibling parent "Roman D. Boiko" <rb d-coding.com> writes:
On Thursday, 19 April 2012 at 15:11:50 UTC, David Nadlinger wrote:
 On Thursday, 19 April 2012 at 10:15:36 UTC, Roman D. Boiko 
 wrote:
 Actually, I prefer Boost only because it is slightly more 
 popular […]

Not to argue about the Boost license being popular in the D community, and not that the question would really matter, but what leads you to this general conclusion? I couldn't find any credible statistics on a quick Google search, but a numer of well known projects use the/a MIT license (X, Ruby on Rails, Mono, Lua, …). David

I wish I could delete that post :) My claim is not based on any research. However, I prefer Boost because: "The Boost Software License is based upon the MIT license, but differs from the MIT license in that it: (i) makes clear that licenses can be granted to organizations as well as individuals; (ii) does not require that the license appear with executables or other binary uses of the library; (iii) expressly disclaims -- on behalf of the author and copyright holders of the software only -- the warranty of title (a warranty that, under the Uniform Commercial Code, is separate from the warranty of non-infringement) (iv) does not extend the disclaimer of warranties to licensees, so that they may, if they choose, undertake such warranties (e.g., in exchange for payment)." http://ideas.opensource.org/ticket/45
Apr 19 2012
prev sibling next sibling parent "Roman D. Boiko" <rb d-coding.com> writes:
On Thursday, 19 April 2012 at 15:30:21 UTC, deadalnix wrote:

 "The Boost Software License is based upon the MIT license, but 
 differs
 from the MIT license in that it:

 (i) makes clear that licenses can be granted to organizations 
 as well as
 individuals;

 (ii) does not require that the license appear with executables 
 or other
 binary uses of the library;

 (iii) expressly disclaims -- on behalf of the author and 
 copyright
 holders of the software only -- the warranty of title (a 
 warranty that,
 under the Uniform Commercial Code, is separate from the 
 warranty of
 non-infringement)

 (iv) does not extend the disclaimer of warranties to 
 licensees, so that
 they may, if they choose, undertake such warranties (e.g., in 
 exchange
 for payment)."

 http://ideas.opensource.org/ticket/45

Very good point. Is it too late to change again ? By the way, what is the status of the attribution clause ?

If it will be decided to change the license, please pay attention to how should programmers apply the license to source and header files: "Add a comment based on the following template, substituting appropriate text for the italicized portion: // Copyright Joe Coder 2004 - 2006. // Distributed under the Boost Software License, Version 1.0. // (See accompanying file LICENSE_1_0.txt or copy at // http://www.boost.org/LICENSE_1_0.txt) Please leave an empty line before and after the above comment block. It is fine if the copyright and license messages are not on different lines; in no case there should be other intervening text. Do not include "All rights reserved" anywhere. Other ways of licensing source files have been considered, but some of them turned out to unintentionally nullify legal elements of the license. Having fixed language for referring to the license helps corporate legal departments evaluate the boost distribution. Creativity in license reference language is strongly discouraged, but judicious changes in the use of whitespace are fine." http://www.boost.org/users/license.html Alternatively, a license copy may be referenced at http://opensource.org/licenses/bsl1.0.html, which looks nicer. :)
Apr 19 2012
prev sibling parent "Roman D. Boiko" <rb d-coding.com> writes:
On Thursday, 19 April 2012 at 19:54:48 UTC, Nick Sabalausky wrote:
 MIT's *much* easier to understand though. Boost has some real 
 goofy, obfuscated wordings. Although it's *worlds* better in 
 that regard than the completely impenatrable GPL or Creative 
 Commons.

Comparing http://www.opensource.org/licenses/MIT and http://opensource.org/licenses/bsl1.0.html, I would not conclude that Boost is more difficult to understand.
 My favorite license, zlib ( 
 http://www.opensource.org/licenses/Zlib )
 doesn't have this #2 issue, and it's even easier to read and 
 understand than
 MIT.

 Plus it doesn't say anything like "to any person", so it should 
 take care of
 #1, too. (Although personally, I think I like MIT better in 
 that regard:
 It's a deterrent against corporations, which gives it a little 
 bit of the
 benefit of the GPL, but without all the bullshit.)

Here I do not agree either. Assuming that we would *want* such "benefit", MIT does not provide it. As for zlib, it is very different from MIT/Boost, thus it is difficult to compare them (for non-lawyers). License is a tool, and its up to SDC authors to select one. I just wanted to provide some information so that another option (Boost) is considered.
Apr 19 2012
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2012-04-13 11:58, Ary Manzana wrote:
 Having a D compiler available as a library will (at least) give these
 benefits:

 1. Can be used by an IDE: D is statically typed and so an IDE can
 benefit a lot from this. The features Descent had, as far as I remember,
 were:
 1.1. Outline
 1.2. Autocompletion
 1.3. Type Hierarchy
 1.4. Syntax and semantic errors, showing not only the line number but
 also column numbers if it makes sense
 1.5. Automatic import inclusion (say, typing writefln and getting a list
 of modules that provide that symbol)
 1.6. Compile-time view: replace auto with the inferred type, insert
 mixins into scope, rewrite operator overloads and other lowerings (but
 I'm not sure this point is really useful)

Sure it is, it's very usable.
 1.7. Determine, given a set of versions and flags, which branches of
 static ifs are used/unused
 1.8. Open declaration
 1.9. Show implementations (of an interface, of interface's method or,
 abstract methods, or method overrides).
 1.10. Propose to override a method (you type some letters and then hit
 some key combination and get a list of methods to override)
 1.11. Get the code of a template when instantiated.
 2. Can be used to build better doc generators: one that shows known
 subclasses or interface implementation, shows inherited methods, type
 hierarchy.
 3. Can be used for lints and other such tools.

 As you can see, a simple lexer/parser built into an IDE, doc generator
 or lint will just give basic features but will never achieve something
 exceptionally good if it lacks the full semantic knowledge of the code.

 I'll write a list of things I'd like this compiler-as-library to have,
 but please help me make it bigger :-)

* Show generated documentation for symbols both on hover and in the autocompletion list * Show source for symbols * Import organizer (or what to call it). Remove unused imports and add missing ones * Code formatting * Fix it/quick fix. Button for automatically fixing simple errors * Syntax and semantic highlighting :) * Refactoring I can also think about a lot of features that is usable if you also have a GUI builder in the IDE.
 * Don't use global variables (DMD is just thought to be run once, so
 when used as a library it can just be used, well, once)
 * Provide a lexer which gives line numbers and column numbers
 (beginning, end)
 * Provide a parser with the same features
 * The semantic phase should not discard any information found while
 parsing. For example when DMD resolves a type it recursively resolves
 aliasing and keeps the last one. An example:

 alias int foo;
 alias foo* bar;

 bar something() { ... }

 It would be nice if "bar", after semantic analysis is done, carries the
 information that bar is "foo*" and that "foo" is "int". Also that
 something's return type is "bar", not "int*".
 * Provide errors and warnings that have line numbers as well as column
 numbers.
 * Allow to parse the top-level definitions of a module. Whit this I mean
 skipping function bodies. At least Descent first built a the outline of
 the whole project by doing this. This mode should also allow specifying
 a location as a target, and if that location falls inside a function
 body then it's contents are returned (useful when editing a file, so you
 can get the outline as well as semantic info of the function currently
 being edited, which will never affect semantic in other parts of the
 module). This will dramatically speed up the editor.
 * Don't stop parsing on errors (I think DMD already does this).
 * Provide a visitor class. If possible, use visitors to implement
 semantic analysis. The visitor will make it super easy to implement
 lints and to generate documentation.

This is a great start and as I've said elsewhere, this would be so cool and usable to have. -- /Jacob Carlborg
Apr 13 2012
next sibling parent reply "Nick Sabalausky" <SeeWebsiteToContactMe semitwist.com> writes:
"Jacob Carlborg" <doob me.com> wrote in message 
news:jm9ulg$ui3$1 digitalmars.com...
 * Refactoring

* Make DustMite even more awesomer by attempting to factor out functions, etc.
Apr 14 2012
parent reply Jacob Carlborg <doob me.com> writes:
On 2012-04-18 14:49, Marco Leise wrote:

 I want refactoring to be as simple as Foo.renameSymbol("std.path.sep",
"std.path.dirSeperator"); if the connection between module- and filename allows
"std.path" to be traced back to the original file.

I'm not sure but I don't think that is enough. In Clang you do something like this: 1. Get cursor of source location 2. Get a unique global ID of the cursor the corresponds to the symbol (unified symbol resolution) 3. Walk the AST to find all matches of this ID 4. Get the source location of the cursors which match 5. Rename the symbol at the source location -- /Jacob Carlborg
Apr 18 2012
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2012-04-19 10:48, Roman D. Boiko wrote:

 Convenience interface on top of low-level core functionality can be
 provided and would suffice for many typical situations.

In that case I think something like this might work: Foo.renameSymbol("file.d:16:4", "bar"); Where "file.d" is the file in which the symbol is located. ":16:4" would be the row and column number. "bar" would be the new name of the symbol. -- /Jacob Carlborg
Apr 19 2012
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-04-19 18:38, Marco Leise wrote:
 Am Thu, 19 Apr 2012 11:24:34 +0200
 schrieb Jacob Carlborg<doob me.com>:

 On 2012-04-19 10:48, Roman D. Boiko wrote:

 Convenience interface on top of low-level core functionality can be
 provided and would suffice for many typical situations.

In that case I think something like this might work: Foo.renameSymbol("file.d:16:4", "bar"); Where "file.d" is the file in which the symbol is located. ":16:4" would be the row and column number. "bar" would be the new name of the symbol.

Doesn't the D module system make symbols unique? I thought with a set of open/relevant files (call it a project if you will) a symbol name can be given without ambiguities. This might go into bike shedding, but it looks like dropping one of D's strengths here. It is of course still necessary to have code to look up symbols under the cursor, no question.

Hmm, I guess so. -- /Jacob Carlborg
Apr 19 2012
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-04-21 15:32, Marco Leise wrote:
 Am Thu, 19 Apr 2012 18:58:26 +0200
 schrieb "Roman D. Boiko"<rb d-coding.com>:

 I doubt specifying symbol it would be any better than location.
 For example, you would not be able to rename a local variable or
 any other symbol nested in a function.

Yes you can! Try to compile this: void x() { int i = 1; void y() { int i = 2; { int i = 3; } } } Prints: Error: shadowing declaration x.y.i is deprecated Local variables have unique names and creating duplicate names is deprecated since a while.

It's possible to have an instance variable and a local variable with the same name. -- /Jacob Carlborg
Apr 21 2012
prev sibling parent reply Ary Manzana <ary esperanto.org.ar> writes:
On 4/19/12 12:48 AM, Jacob Carlborg wrote:
 On 2012-04-18 14:49, Marco Leise wrote:

 I want refactoring to be as simple as Foo.renameSymbol("std.path.sep",
 "std.path.dirSeperator"); if the connection between module- and
 filename allows "std.path" to be traced back to the original file.

I'm not sure but I don't think that is enough. In Clang you do something like this: 1. Get cursor of source location 2. Get a unique global ID of the cursor the corresponds to the symbol (unified symbol resolution) 3. Walk the AST to find all matches of this ID 4. Get the source location of the cursors which match 5. Rename the symbol at the source location

Unfortunately rename can't be perfect in D because you can't apply it inside templates.
Apr 19 2012
next sibling parent Ary Manzana <ary esperanto.org.ar> writes:
On 4/19/12 7:25 PM, Roman D. Boiko wrote:
 On Thursday, 19 April 2012 at 11:04:20 UTC, Ary Manzana wrote:
 On 4/19/12 12:48 AM, Jacob Carlborg wrote:
 On 2012-04-18 14:49, Marco Leise wrote:

 I want refactoring to be as simple as Foo.renameSymbol("std.path.sep",
 "std.path.dirSeperator"); if the connection between module- and
 filename allows "std.path" to be traced back to the original file.

I'm not sure but I don't think that is enough. In Clang you do something like this: 1. Get cursor of source location 2. Get a unique global ID of the cursor the corresponds to the symbol (unified symbol resolution) 3. Walk the AST to find all matches of this ID 4. Get the source location of the cursors which match 5. Rename the symbol at the source location

Unfortunately rename can't be perfect in D because you can't apply it inside templates.

In general, there is nothing preventing renaming in templates or mixins, if you do renaming after semantic analysis. However, there can be some troubles, e.g., if mixin string is generated on the fly from some function. Compiler error should suffice in this case, or even better, refactoring tool should give a warning.

T foo(T)(T x) { return x.something(); } int something(int x) { return 1; } float something(float x) { return 1.0; } Now... go and rename the first function named "something". What do you do with "x.something()" inside the template... rename it or not?
Apr 19 2012
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2012-04-19 13:05, Ary Manzana wrote:

 Unfortunately rename can't be perfect in D because you can't apply it
 inside templates.

Could you elaborate a bit on why this is? -- /Jacob Carlborg
Apr 19 2012
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-04-19 16:55, Roman D. Boiko wrote:
 On Thursday, 19 April 2012 at 14:46:26 UTC, Jacob Carlborg wrote:
 On 2012-04-19 13:05, Ary Manzana wrote:

 Unfortunately rename can't be perfect in D because you can't apply it
 inside templates.

Could you elaborate a bit on why this is?

Please see several last messages for this discussion.

Yeah, I saw that now, sorry. -- /Jacob Carlborg
Apr 19 2012
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-04-19 17:00, Roman D. Boiko wrote:
 On Thursday, 19 April 2012 at 14:55:50 UTC, David Nadlinger wrote:
 The question is, though, whether »perfect« renaming support is really
 required, or if a defensive solution warning on potential ambiguities
 isn't already useful enough for most cases.

 David

"Perfect" support is impossible, I guess. For example, if a library has been released, a rename would cause API breaking changes. But for code under control of user which is doing the raname it is possible to provide customization what to rename (ReSharper does that, for example).

That's not up to the IDE to decide. If the user wants to break the API the IDE should let him/her. -- /Jacob Carlborg
Apr 19 2012
prev sibling next sibling parent "Roman D. Boiko" <rb d-coding.com> writes:
On Wednesday, 18 April 2012 at 16:48:46 UTC, Jacob Carlborg wrote:
 On 2012-04-18 14:49, Marco Leise wrote:

 I want refactoring to be as simple as 
 Foo.renameSymbol("std.path.sep", "std.path.dirSeperator"); if 
 the connection between module- and filename allows "std.path" 
 to be traced back to the original file.

I'm not sure but I don't think that is enough. In Clang you do something like this: 1. Get cursor of source location 2. Get a unique global ID of the cursor the corresponds to the symbol (unified symbol resolution) 3. Walk the AST to find all matches of this ID 4. Get the source location of the cursors which match 5. Rename the symbol at the source location

Convenience interface on top of low-level core functionality can be provided and would suffice for many typical situations.
Apr 19 2012
prev sibling next sibling parent "Roman D. Boiko" <rb d-coding.com> writes:
On Thursday, 19 April 2012 at 11:04:20 UTC, Ary Manzana wrote:
 On 4/19/12 12:48 AM, Jacob Carlborg wrote:
 On 2012-04-18 14:49, Marco Leise wrote:

 I want refactoring to be as simple as 
 Foo.renameSymbol("std.path.sep",
 "std.path.dirSeperator"); if the connection between module- 
 and
 filename allows "std.path" to be traced back to the original 
 file.

I'm not sure but I don't think that is enough. In Clang you do something like this: 1. Get cursor of source location 2. Get a unique global ID of the cursor the corresponds to the symbol (unified symbol resolution) 3. Walk the AST to find all matches of this ID 4. Get the source location of the cursors which match 5. Rename the symbol at the source location

Unfortunately rename can't be perfect in D because you can't apply it inside templates.

In general, there is nothing preventing renaming in templates or mixins, if you do renaming after semantic analysis. However, there can be some troubles, e.g., if mixin string is generated on the fly from some function. Compiler error should suffice in this case, or even better, refactoring tool should give a warning.
Apr 19 2012
prev sibling next sibling parent "Roman D. Boiko" <rb d-coding.com> writes:
On Thursday, 19 April 2012 at 11:04:20 UTC, Ary Manzana wrote:
 On 4/19/12 12:48 AM, Jacob Carlborg wrote:
 On 2012-04-18 14:49, Marco Leise wrote:

 I want refactoring to be as simple as 
 Foo.renameSymbol("std.path.sep",
 "std.path.dirSeperator"); if the connection between module- 
 and
 filename allows "std.path" to be traced back to the original 
 file.

I'm not sure but I don't think that is enough. In Clang you do something like this: 1. Get cursor of source location 2. Get a unique global ID of the cursor the corresponds to the symbol (unified symbol resolution) 3. Walk the AST to find all matches of this ID 4. Get the source location of the cursors which match 5. Rename the symbol at the source location

Unfortunately rename can't be perfect in D because you can't apply it inside templates.

In general, there is nothing preventing renaming in templates or mixins, if you do renaming after semantic analysis. However, there can be some troubles, e.g., if mixin string is generated on the fly from some function. Compiler error should suffice in this case, or even better, refactoring tool should give a warning.
Apr 19 2012
prev sibling next sibling parent "Tobias Pankrath" <tobias pankrath.net> writes:
On Thursday, 19 April 2012 at 12:16:05 UTC, Ary Manzana wrote:
 On 4/19/12 7:25 PM, Roman D. Boiko wrote:
 On Thursday, 19 April 2012 at 11:04:20 UTC, Ary Manzana wrote:
 On 4/19/12 12:48 AM, Jacob Carlborg wrote:
 On 2012-04-18 14:49, Marco Leise wrote:

 I want refactoring to be as simple as 
 Foo.renameSymbol("std.path.sep",
 "std.path.dirSeperator"); if the connection between module- 
 and
 filename allows "std.path" to be traced back to the 
 original file.

I'm not sure but I don't think that is enough. In Clang you do something like this: 1. Get cursor of source location 2. Get a unique global ID of the cursor the corresponds to the symbol (unified symbol resolution) 3. Walk the AST to find all matches of this ID 4. Get the source location of the cursors which match 5. Rename the symbol at the source location

Unfortunately rename can't be perfect in D because you can't apply it inside templates.

In general, there is nothing preventing renaming in templates or mixins, if you do renaming after semantic analysis. However, there can be some troubles, e.g., if mixin string is generated on the fly from some function. Compiler error should suffice in this case, or even better, refactoring tool should give a warning.

T foo(T)(T x) { return x.something(); } int something(int x) { return 1; } float something(float x) { return 1.0; } Now... go and rename the first function named "something". What do you do with "x.something()" inside the template... rename it or not?

1. Ask the user or 2. Check if there is an instance of foo!int which refers to something. If so and there is no other instance, that refers to another something: rename it. 3. If there in another instance: Ask the user to rename one, all, none.
Apr 19 2012
prev sibling next sibling parent "Roman D. Boiko" <rb d-coding.com> writes:
On Thursday, 19 April 2012 at 12:31:24 UTC, Tobias Pankrath wrote:

 T foo(T)(T x) {
  return x.something();
 }

 int something(int x) {
  return 1;
 }

 float something(float x) {
  return 1.0;
 }

 Now... go and rename the first function named "something". 
 What do you do with "x.something()" inside the template... 
 rename it or not?

1. Ask the user or 2. Check if there is an instance of foo!int which refers to something. If so and there is no other instance, that refers to another something: rename it. 3. If there in another instance: Ask the user to rename one, all, none.

I would go ReSharper way: present user a tree with all occurrences of related symbols and checkboxes to include or exclude each. Or, in this particular case, display a list of affected types and checkboxes for each. This isn't VERY user friendly, though, but very flexible and reasonable. Give warnings if necessary. Semantic analysis phase would provide information whether symbol is actually related, or only has the same name (but different semantics and scope).
Apr 19 2012
prev sibling next sibling parent "Roman D. Boiko" <rb d-coding.com> writes:
On Thursday, 19 April 2012 at 12:55:50 UTC, Roman D. Boiko wrote:
 On Thursday, 19 April 2012 at 12:31:24 UTC, Tobias Pankrath 
 wrote:

 T foo(T)(T x) {
 return x.something();
 }

 int something(int x) {
 return 1;
 }

 float something(float x) {
 return 1.0;
 }

 Now... go and rename the first function named "something". 
 What do you do with "x.something()" inside the template... 
 rename it or not?



 I would go ReSharper way: present user a tree with all 
 occurrences of related symbols and checkboxes to include or 
 exclude each. Or, in this particular case, display a list of 
 affected types and checkboxes for each. This isn't VERY user 
 friendly, though, but very flexible and reasonable. Give 
 warnings if necessary.

 Semantic analysis phase would provide information whether 
 symbol is actually related, or only has the same name (but 
 different semantics and scope).

Respective convenience API would simply provide overloads and reasonable defaults for similar cases. My point is that there is nothing in language itself preventing such functionality, if we have semantic analysis. The problem is related to user intentions, and decision should be made by the user (but the tool should provide good defaults and enough information).
Apr 19 2012
prev sibling next sibling parent "Roman D. Boiko" <rb d-coding.com> writes:
On Thursday, 19 April 2012 at 14:46:26 UTC, Jacob Carlborg wrote:
 On 2012-04-19 13:05, Ary Manzana wrote:

 Unfortunately rename can't be perfect in D because you can't 
 apply it
 inside templates.

Could you elaborate a bit on why this is?

Please see several last messages for this discussion.
Apr 19 2012
prev sibling next sibling parent "David Nadlinger" <see klickverbot.at> writes:
On Thursday, 19 April 2012 at 14:46:26 UTC, Jacob Carlborg wrote:
 On 2012-04-19 13:05, Ary Manzana wrote:

 Unfortunately rename can't be perfect in D because you can't 
 apply it
 inside templates.

Could you elaborate a bit on why this is?

A template can be instantiated in user code with parameters that can't be known at library compile/refactor time (or mixed into, again unknown, scopes), and this can OTOH lead to problems like: --- int doSomething(int a) { return a; } auto foo(T)(T t) { // User renames doSomething(int) above – should this be // changed as well? return t.doSomething(); } unittest { assert(foo(3) == 3); } --- The question is, though, whether »perfect« renaming support is really required, or if a defensive solution warning on potential ambiguities isn't already useful enough for most cases. David
Apr 19 2012
prev sibling next sibling parent "Roman D. Boiko" <rb d-coding.com> writes:
On Thursday, 19 April 2012 at 14:55:50 UTC, David Nadlinger wrote:
 The question is, though, whether »perfect« renaming support 
 is really required, or if a defensive solution warning on 
 potential ambiguities isn't already useful enough for most 
 cases.

 David

"Perfect" support is impossible, I guess. For example, if a library has been released, a rename would cause API breaking changes. But for code under control of user which is doing the raname it is possible to provide customization what to rename (ReSharper does that, for example).
Apr 19 2012
prev sibling next sibling parent "Roman D. Boiko" <rb d-coding.com> writes:
On Thursday, 19 April 2012 at 16:38:31 UTC, Marco Leise wrote:
 Am Thu, 19 Apr 2012 11:24:34 +0200
 schrieb Jacob Carlborg <doob me.com>:

 On 2012-04-19 10:48, Roman D. Boiko wrote:
 
 Convenience interface on top of low-level core functionality 
 can be
 provided and would suffice for many typical situations.

In that case I think something like this might work: Foo.renameSymbol("file.d:16:4", "bar"); Where "file.d" is the file in which the symbol is located. ":16:4" would be the row and column number. "bar" would be the new name of the symbol.

Doesn't the D module system make symbols unique? I thought with a set of open/relevant files (call it a project if you will) a symbol name can be given without ambiguities. This might go into bike shedding, but it looks like dropping one of D's strengths here. It is of course still necessary to have code to look up symbols under the cursor, no question.

I doubt specifying symbol it would be any better than location. For example, you would not be able to rename a local variable or any other symbol nested in a function.
Apr 19 2012
prev sibling next sibling parent "Roman D. Boiko" <rb d-coding.com> writes:
 That's not up to the IDE to decide. If the user wants to break 
 the API the IDE should let him/her.

Yes, that's exactly my point. It is a matter of user goals, not a language issue, and decision should not be made by a tool instead of a user.
Apr 19 2012
prev sibling next sibling parent "Roman D. Boiko" <rb d-coding.com> writes:
On Saturday, 21 April 2012 at 13:32:28 UTC, Marco Leise wrote:
 Am Thu, 19 Apr 2012 18:58:26 +0200
 schrieb "Roman D. Boiko" <rb d-coding.com>:

 I doubt specifying symbol it would be any better than 
 location. For example, you would not be able to rename a local 
 variable or any other symbol nested in a function.

Yes you can! Try to compile this: void x() { int i = 1; void y() { int i = 2; { int i = 3; } } } Prints: Error: shadowing declaration x.y.i is deprecated Local variables have unique names and creating duplicate names is deprecated since a while.

I guess x.y.i is just a convenience naming used by DMD, and it does not appear in specification. Also, Artur Skawina provided a nice counter-example where it would not work.
Apr 21 2012
prev sibling next sibling parent "Roman D. Boiko" <rb d-coding.com> writes:
 It's possible to have an instance variable and a local variable 
 with the same name.

According to Marco's idea, those would be qualified differently: className.x vs className.funcName.x, but he recently provided an example where that is ambiguous. Also it would be a problem for symbols declared inside anonimous members. In general, I like symbol location ("file.d:50:7"). It should always be available to the tool which uses API.
Apr 21 2012
prev sibling parent "David Nadlinger" <see klickverbot.at> writes:
On Thursday, 19 April 2012 at 16:38:31 UTC, Marco Leise wrote:
 Doesn't the D module system make symbols unique?

--- void someFunction() { { int someVariable = 3; // ... } { int someVariable = 1000; // ... } } --- How do you target the someVariable symbol from the second block? David
Apr 21 2012
prev sibling next sibling parent Marco Leise <Marco.Leise gmx.de> writes:
Am Sat, 14 Apr 2012 22:44:07 -0400
schrieb "Nick Sabalausky" <SeeWebsiteToContactMe semitwist.com>:

 "Jacob Carlborg" <doob me.com> wrote in message 
 news:jm9ulg$ui3$1 digitalmars.com...
 * Refactoring

* Make DustMite even more awesomer by attempting to factor out functions, etc.

Something internal: * Keep a view of every opened file in memory. * Allow to manipulate that file with an interface like insert, delete, replace of character ranges. * Reparse only what has changed through that interface (remember datetime.d?) This is to avoid CPU hogging and typing delays. * Manage multiple open projects. That means sharing of the internal representation of druntime/Phobos and libraries. * We probably want two parser interfaces. Thinking of XML: SAX and DOM. One parser iterates over the tokens and calls event handlers (no AST). Example: lazy (line-by-line) editor syntax highlighting. The other parser builds the whole module as an AST to work on. Example: editing code multiple times in an IDE/refactorings * If necessary, some sort of transaction, so a refactoring that fails in the middle of the operation doesn't leave the code in a broken state. * DDoc awareness. Cross-linked symbols etc. must be included when that symbol is renamed. I want refactoring to be as simple as Foo.renameSymbol("std.path.sep", "std.path.dirSeperator"); if the connection between module- and filename allows "std.path" to be traced back to the original file. -- Marco
Apr 18 2012
prev sibling next sibling parent "Roman D. Boiko" <rb d-coding.com> writes:
On Friday, 13 April 2012 at 09:57:49 UTC, Ary Manzana wrote:
 Having a D compiler available as a library will (at least) give 
 these benefits:

   1. Can be used by an IDE: D is statically typed and so an IDE 
 can benefit a lot from this. The features Descent had, as far 
 as I remember, were:
     1.1. Outline
     1.2. Autocompletion
     1.3. Type Hierarchy
     1.4. Syntax and semantic errors, showing not only the line 
 number but also column numbers if it makes sense
     1.5. Automatic import inclusion (say, typing writefln and 
 getting a list of modules that provide that symbol)
     1.6. Compile-time view: replace auto with the inferred 
 type, insert mixins into scope, rewrite operator overloads and 
 other lowerings (but I'm not sure this point is really useful)
     1.7. Determine, given a set of versions and flags, which 
 branches of static ifs are used/unused
     1.8. Open declaration
     1.9. Show implementations (of an interface, of interface's 
 method or, abstract methods, or method overrides).
     1.10. Propose to override a method (you type some letters 
 and then hit some key combination and get a list of methods to 
 override)
     1.11. Get the code of a template when instantiated.
  2. Can be used to build better doc generators: one that shows 
 known subclasses or interface implementation, shows inherited 
 methods, type hierarchy.
  3. Can be used for lints and other such tools.

 As you can see, a simple lexer/parser built into an IDE, doc 
 generator or lint will just give basic features but will never 
 achieve something exceptionally good if it lacks the full 
 semantic knowledge of the code.

 I'll write a list of things I'd like this compiler-as-library 
 to have, but please help me make it bigger :-)

  * Don't use global variables (DMD is just thought to be run 
 once, so when used as a library it can just be used, well, once)
  * Provide a lexer which gives line numbers and column numbers 
 (beginning, end)
  * Provide a parser with the same features
  * The semantic phase should not discard any information found 
 while parsing. For example when DMD resolves a type it 
 recursively resolves aliasing and keeps the last one. An 
 example:

   alias int foo;
   alias foo* bar;

   bar something() { ... }

   It would be nice if "bar", after semantic analysis is done, 
 carries the information that bar is "foo*" and that "foo" is 
 "int". Also that something's return type is "bar", not "int*".
  * Provide errors and warnings that have line numbers as well 
 as column numbers.
  * Allow to parse the top-level definitions of a module. Whit 
 this I mean skipping function bodies. At least Descent first 
 built a the outline of the whole project by doing this. This 
 mode should also allow specifying a location as a target, and 
 if that location falls inside a function body then it's 
 contents are returned (useful when editing a file, so you can 
 get the outline as well as semantic info of the function 
 currently being edited, which will never affect semantic in 
 other parts of the module). This will dramatically speed up the 
 editor.
  * Don't stop parsing on errors (I think DMD already does this).
  * Provide a visitor class. If possible, use visitors to 
 implement semantic analysis. The visitor will make it super 
 easy to implement lints and to generate documentation.

By the way, I also started a project called <a href="https://github.com/roman-d-boiko/DCT">The D Compiler Tools (DCT)</a> about a month ago. It is provided under the Boost license, and has the goal to enable building third-party tools with functionality that would include described above. I'm trying to build LLVM-based codegen and also reuse frontend in a separate project with basic IDE functionality for D. I have never implemented compilers before, and probably should have called my project SDC if only that name had not been taken before ;) Goals are very similar to those of SDC (especially now, after its re-licensing). But I don't commit to ever finish the project, because my free time is very limited :(. There is SIGNIFICANTLY less functionality implemented at this moment than in SDC. Currently, only primitive lexing is in place (I follow the KISS principle where possible) and parsing of auto declarations (auto i = 3 * (2 + 8), etc.) with stubs for most other cases. (Please note that project Readme file is outdated.) Parser is top-down recursive descent, and it follows specification very closely, except some differences needed to simplify implementation (like using loops to implement left-recursion in specification). Anyone interested in discussing DCT or participating in development would be welcome!
Apr 19 2012
prev sibling next sibling parent "Tobias Pankrath" <tobias pankrath.net> writes:
On Thursday, 19 April 2012 at 09:24:23 UTC, Roman D. Boiko wrote:
 On Friday, 13 April 2012 at 09:57:49 UTC, Ary Manzana wrote:
 Having a D compiler available as a library will (at least) 
 give these benefits:


What about joining forces with sdc then?
Apr 19 2012
prev sibling next sibling parent "Roman D. Boiko" <rb d-coding.com> writes:
On Thursday, 19 April 2012 at 10:28:08 UTC, Tobias Pankrath wrote:
 On Thursday, 19 April 2012 at 09:24:23 UTC, Roman D. Boiko 
 wrote:
 On Friday, 13 April 2012 at 09:57:49 UTC, Ary Manzana wrote:
 Having a D compiler available as a library will (at least) 
 give these benefits:


What about joining forces with sdc then?

I think I will use some code from SDC and contibute back to it now that it is liberally licensed. It wasn't an option when it was under GNU, because I generally prefer possibility for commercial usage. However, I believe that the project I'm working on (DCT) has a good potential and the D community will better benefit from two alternatives. There are several design differences, and will be more. This also gives me additional flexibility and more learning opportunities. I will dual-license (Boost+MIT) any part of my code if somebody would like to incorporate such part in SDC. My question about motivation for MIT licence was caused in particular by desire to minimize possible overhead of dealing with two licenses for me.
Apr 19 2012
prev sibling next sibling parent "so" <so so.so> writes:
On Thursday, 19 April 2012 at 10:28:08 UTC, Tobias Pankrath wrote:
 On Thursday, 19 April 2012 at 09:24:23 UTC, Roman D. Boiko 
 wrote:
 On Friday, 13 April 2012 at 09:57:49 UTC, Ary Manzana wrote:
 Having a D compiler available as a library will (at least) 
 give these benefits:


What about joining forces with sdc then?

Wow! Didn't know there was a D compiler with Boost like licence. Awesome job whoever started it.
Apr 19 2012
prev sibling next sibling parent Marco Leise <Marco.Leise gmx.de> writes:
Am Thu, 19 Apr 2012 11:24:34 +0200
schrieb Jacob Carlborg <doob me.com>:

 On 2012-04-19 10:48, Roman D. Boiko wrote:
 
 Convenience interface on top of low-level core functionality can be
 provided and would suffice for many typical situations.

In that case I think something like this might work: Foo.renameSymbol("file.d:16:4", "bar"); Where "file.d" is the file in which the symbol is located. ":16:4" would be the row and column number. "bar" would be the new name of the symbol.

Doesn't the D module system make symbols unique? I thought with a set of open/relevant files (call it a project if you will) a symbol name can be given without ambiguities. This might go into bike shedding, but it looks like dropping one of D's strengths here. It is of course still necessary to have code to look up symbols under the cursor, no question. -- Marco
Apr 19 2012
prev sibling next sibling parent Marco Leise <Marco.Leise gmx.de> writes:
Am Thu, 19 Apr 2012 18:58:26 +0200
schrieb "Roman D. Boiko" <rb d-coding.com>:

 I doubt specifying symbol it would be any better than location. 
 For example, you would not be able to rename a local variable or 
 any other symbol nested in a function.

Yes you can! Try to compile this: void x() { int i = 1; void y() { int i = 2; { int i = 3; } } } Prints: Error: shadowing declaration x.y.i is deprecated Local variables have unique names and creating duplicate names is deprecated since a while. -- Marco
Apr 21 2012
prev sibling next sibling parent Artur Skawina <art.08.09 gmail.com> writes:
On 04/21/12 15:32, Marco Leise wrote:
 Am Thu, 19 Apr 2012 18:58:26 +0200
 schrieb "Roman D. Boiko" <rb d-coding.com>:
 
 I doubt specifying symbol it would be any better than location. 
 For example, you would not be able to rename a local variable or 
 any other symbol nested in a function.

Yes you can! Try to compile this: void x() { int i = 1; void y() { int i = 2; { int i = 3; } } } Prints: Error: shadowing declaration x.y.i is deprecated Local variables have unique names and creating duplicate names is deprecated since a while.

void x() { int i = 1; { int i = 2; writeln(i); } writeln(i); } artur
Apr 21 2012
prev sibling parent Marco Leise <Marco.Leise gmx.de> writes:
I'll answer you both here...

Am Sat, 21 Apr 2012 16:11:02 +0200
schrieb Artur Skawina <art.08.09 gmail.com>:

 On 04/21/12 15:32, Marco Leise wrote:
 Am Thu, 19 Apr 2012 18:58:26 +0200
 schrieb "Roman D. Boiko" <rb d-coding.com>:
 
 I doubt specifying symbol it would be any better than location. 
 For example, you would not be able to rename a local variable or 
 any other symbol nested in a function.

Yes you can! Try to compile this: void x() { int i = 1; void y() { int i = 2; { int i = 3; } } } Prints: Error: shadowing declaration x.y.i is deprecated Local variables have unique names and creating duplicate names is deprecated since a while.

void x() { int i = 1; { int i = 2; writeln(i); } writeln(i); } artur

That gives the same error message as before for me. I'm using GDC with DMD 2.057 frontend. Maybe the behavoir has changed. :-( Am Sat, 21 Apr 2012 16:28:15 +0200 schrieb "Roman D. Boiko" <rb d-coding.com>:
 On Saturday, 21 April 2012 at 13:32:28 UTC, Marco Leise wrote:
 Am Thu, 19 Apr 2012 18:58:26 +0200
 schrieb "Roman D. Boiko" <rb d-coding.com>:

 I doubt specifying symbol it would be any better than 
 location. For example, you would not be able to rename a local 
 variable or any other symbol nested in a function.

Yes you can! Try to compile this: void x() { int i = 1; void y() { int i = 2; { int i = 3; } } } Prints: Error: shadowing declaration x.y.i is deprecated Local variables have unique names and creating duplicate names is deprecated since a while.

I guess x.y.i is just a convenience naming used by DMD, and it does not appear in specification. Also, Artur Skawina provided a nice counter-example where it would not work.

I just returned from the toilet where I flushed down this idea, because it doesn't work with overloads: void x(int i) { ... } void x(float i) { ... } "x.i" is ambiguous to the refactoring tool. It would only work on the mangled names and before I go into the depths of what to do with extern(C) and so on... we can just go with the line/column lookup of symbols. -- Marco
Apr 21 2012