www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Bug 14666, can we prioritize ?

reply "deadalnix" <deadalnix gmail.com> writes:
https://issues.dlang.org/show_bug.cgi?id=14666

This bug has become a blocker for SDC. I was able to work around 
it in various places so far, but this becomes harder and harder 
to do so.
Jun 09 2015
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/9/2015 6:43 PM, deadalnix wrote:

 https://issues.dlang.org/show_bug.cgi?id=14666

 This bug has become a blocker for SDC. I was able to work around it in various
 places so far, but this becomes harder and harder to do so.


I suggest as a workaround to restructure the code so it is not doing circular 
imports.
Jun 09 2015
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 10 June 2015 at 02:01:14 UTC, Walter Bright wrote:

 On 6/9/2015 6:43 PM, deadalnix wrote:

 https://issues.dlang.org/show_bug.cgi?id=14666

 This bug has become a blocker for SDC. I was able to work 
 around it in various
 places so far, but this becomes harder and harder to do so.


 I suggest as a workaround to restructure the code so it is not 
 doing circular imports.


The actual code is far more complex than the presented sample :)

That what I'm trying to do in general, but here I'm faced with 
cases that are very hard to untangle.

Also, I do think this kind of weird edge cases are what give D 
this immage of immature language (it has gotten much better over 
the past few years, so there is that).
Jun 09 2015
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/9/2015 10:39 PM, deadalnix wrote:

 The actual code is far more complex than the presented sample :)

 That what I'm trying to do in general, but here I'm faced with cases that are
 very hard to untangle.


The 'tangle' of cyclical import graphs, as you so appropriately put, is not
only 
hard on the compiler, it makes it much harder for human readers to make sense
of 
code. Even reading your little code snippet hurts my brain.

Go, in one of their better decisions, decided "Thou Shalt Not Have Import 
Cycles". This forces users to untangle their code, and I believe that is A Good 
Thing.

I'd like to do that for D, but of course it is far too late for that.
Jun 09 2015
next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 10 June 2015 at 06:55:43 UTC, Walter Bright wrote:

 On 6/9/2015 10:39 PM, deadalnix wrote:

 The actual code is far more complex than the presented sample 
 :)

 That what I'm trying to do in general, but here I'm faced with 
 cases that are
 very hard to untangle.


 The 'tangle' of cyclical import graphs, as you so appropriately 
 put, is not only hard on the compiler, it makes it much harder 
 for human readers to make sense of code. Even reading your 
 little code snippet hurts my brain.

 Go, in one of their better decisions, decided "Thou Shalt Not 
 Have Import Cycles". This forces users to untangle their code, 
 and I believe that is A Good Thing.

 I'd like to do that for D, but of course it is far too late for 
 that.


That wouldn't be possible. For the most part, the cases where I 
have these kind of thing are due to mixin being non hygienic, 
forcing to import thing I should have been passing via template 
argument.

A typical example of that in SDC is the way types are handled.

SDC has AST and IR, which share some common feature when it comes 
to type, but are also different. There is a module that implement 
the common feature for IR types and AST types.

Problem is, this base uses bitfield from the standard lib, which 
in turn uses mixin. Meaning I can't use the base with importing 
all modules where it is going to be instantiated. This in turn 
cause the forward reference thing and forces me to bypass the 
type system in various places to avoid it.

This means either make Liskov very sad and create a 'tangle' with 
imports, or just not use the standard lib and duplicate a ton of 
code.
Jun 10 2015
prev sibling next sibling parent reply Kenji Hara via Digitalmars-d <digitalmars-d puremagic.com> writes:
2015-06-10 15:55 GMT+09:00 Walter Bright via Digitalmars-d <
digitalmars-d puremagic.com>:


 On 6/9/2015 10:39 PM, deadalnix wrote:


 The actual code is far more complex than the presented sample :)

 That what I'm trying to do in general, but here I'm faced with cases that
 are
 very hard to untangle.


 The 'tangle' of cyclical import graphs, as you so appropriately put, is
 not only hard on the compiler, it makes it much harder for human readers to
 make sense of code. Even reading your little code snippet hurts my brain.

 Go, in one of their better decisions, decided "Thou Shalt Not Have Import
 Cycles". This forces users to untangle their code, and I believe that is A
 Good Thing.

 I'd like to do that for D, but of course it is far too late for that.


D has true forward reference resolution mechanism. In that case modules are
cyclic imported, but the declared symbols don't have actual cyclic
references. Therefore the snippet should work.

https://github.com/D-Programming-Language/dmd/pull/4735

And the regression was introduced by your unrelated change. While fixing
the issue, I couldn't understand why it was necessary.

https://github.com/D-Programming-Language/dmd/commit/a04cf864b932061ad7b72e7cad8b16fabc6a825a

Kenji Hara
Jun 10 2015
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/10/2015 5:30 AM, Kenji Hara via Digitalmars-d wrote:

 D has true forward reference resolution mechanism. In that case modules are
 cyclic imported, but the declared symbols don't have actual cyclic references.
 Therefore the snippet should work.


I agree the snippet should work. And I still maintain that cyclical imports are 
something worth significant effort to avoid.

But also, dmd doesn't quite have a proper mechanism for resolving forward 
references - it's a series of hacks, which is why it's been such a source of
bugs.

The right way to do it is to go full lazy on running semantic() on imported 
symbols. I did this with enums a while back, and it's been working well.



 https://github.com/D-Programming-Language/dmd/pull/4735


As always, thanks for the quick action!



 And the regression was introduced by your unrelated change. While fixing the
 issue, I couldn't understand why it was necessary.

 https://github.com/D-Programming-Language/dmd/commit/a04cf864b932061ad7b72e7cad8b16fabc6a825a


Yeah, it was caused by a hack I put in as yet another hack to deal with forward 
references.
Jun 10 2015
next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, 10 June 2015 at 17:28:54 UTC, Walter Bright wrote:

 As always, thanks for the quick action!


What would we ever do without Kenji? :)

- Jonathan M Davis
Jun 10 2015
prev sibling parent "Kenji Hara" <k.hara.pg gmail.com> writes:
On Wednesday, 10 June 2015 at 17:28:54 UTC, Walter Bright wrote:

 But also, dmd doesn't quite have a proper mechanism for 
 resolving forward references - it's a series of hacks, which is 
 why it's been such a source of bugs.

 The right way to do it is to go full lazy on running semantic() 
 on imported symbols. I did this with enums a while back, and 
 it's been working well.


Yes, I've continuously improved the hacky code in dmd, and 
finally it's almost completed to realize lazy semantic analysis.

List of current implementation details in dmd code:

- before than all, an imported symbol will be set its 'scope' for 
its lazy semantic analysis.
     Dsymbol::setScope()

- variable types are determined lazily, and it detects 'circular 
reference' in its initializer.
     v->semantic(v->scope); in DsymbolExp::semantic(), and etc
     v->inuse

- forward reference for functions and on-demand return type 
inference is done:
     FuncDeclaration::functionSemantic()
         Determine function signatures (parameter types and 
returnt types)
         If return type is inferred, it will invoke 
functionSemantic3().
     FuncDeclaration::functionSemantic3()
         Determine function full signature (includind deduced 
attributes and inferred return types)

- an alias for the symbols comes from instantiations will be 
resolve lazily
     AliasDeclaration::toAlias()

- an aggregate size (all of field variable types) is analyzed 
independently from other member semantic (member functions and so 
on)
     VarDeclaration::setFieldOffset()
     AggregateDeclaration::finalizeSize()

- tangled class template definitions is analyzed correctly.
     ClassDeclaration::baseok

- declarations in static if is lazily resolved
     StaticIfDeclaration::include()
         It will run addMembers() and setScope() for the members, 
after the condition is correctly evaluated.

- the types and symbols on UDA is lazily resolved
     UserAttributeDeclaration::getAttributes()
         It's called by __traits(getAttributes) to invoke 
on-demand semantic analysis.
     UserAttributeDeclaration::semantic2()
         UDA does not affect the semantic of annotated symbol, so 
it's normally deferred until semantic2() stage.

Anymore I cannot say they're just hacks.

While doing them, unfortunately I've introduced not a few numbers 
of regression issues, but they have been fixed properly.
Remaining issues in HEAD:

https://github.com/D-Programming-Language/dmd/pull/4731
https://github.com/D-Programming-Language/dmd/pull/4738

I'll be happy when you prioritize the PRs.

Kenji Hara
Jun 10 2015
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
I have always been wondering why cyclic imports are allowed in D 
and if those enable certain design that is not possible 
otherwise. That mixin case sounds interesting but doesn't "click" 
:)
Jun 10 2015
prev sibling parent reply "Brian Rogoff" <brogoff gmail.com> writes:
On Wednesday, 10 June 2015 at 06:55:43 UTC, Walter Bright wrote:

 On 6/9/2015 10:39 PM, deadalnix wrote:

 The actual code is far more complex than the presented sample 
 :)

 That what I'm trying to do in general, but here I'm faced with 
 cases that are
 very hard to untangle.


 The 'tangle' of cyclical import graphs, as you so appropriately 
 put, is not only hard on the compiler, it makes it much harder 
 for human readers to make sense of code. Even reading your 
 little code snippet hurts my brain.

 Go, in one of their better decisions, decided "Thou Shalt Not 
 Have Import Cycles". This forces users to untangle their code, 
 and I believe that is A Good Thing.

 I'd like to do that for D, but of course it is far too late for 
 that.


Ada (83 and 95) were also rigid about not allowing any kind of 
import cycles. At some point, probably after Ada 95 when OO was 
added, it was decided after much experience that there were some 
cases where it was better to allow it. If you're interested, in 
the Ada world this issue was dubbed "the with-ing problem" and a 
quick Google search gives

http://www.ada95.ch/doc/within/FAQ.html#what

http://www.adaic.org/resources/add_content/standards/05rat/html/Rat-4-2.html

also in the world of Haskell, this thread

https://mail.haskell.org/pipermail/haskell-cafe/2004-September/006885.html

and I'm sure in many other places too.

I agree with you, Walter, that mutual recursion amongst imports 
is usually bad, but I think you inadvertently made the right 
decision by allowing it in D, and that the bug should be fixed. 
If people overuse it to write messy code then the community 
should tell them to write that code better.
Jun 10 2015
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/10/15 8:41 AM, Brian Rogoff wrote:

 I agree with you, Walter, that mutual recursion amongst imports is
 usually bad, but I think you inadvertently made the right decision by
 allowing it in D, and that the bug should be fixed. If people overuse it
 to write messy code then the community should tell them to write that
 code better.


I agree. I'm one of those folks who think circular dependencies are 
fine, thank you very much, and the few cases in which it causes problems 
should be flagged and diagnosed in a principled way.

Everybody also likes:

* recursion and mutual recursion of any depth (there's a lot more of it 
going in the usual programs than one might think)

* recursive and mutually recursive types

* declaring top level names in any order and they just "work"

But all of a sudden, people get sugar in their gasoline and sand in 
their gears when they hear of the same thing applied to modules. I 
suspect it's because some legacy languages didn't quite do a stellar job 
at it. Figuring that stuff out and pointing out problems is exactly the 
kind of thing computers are good at and humans, not so good.


Andrei
Jun 10 2015
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/10/2015 9:03 AM, Andrei Alexandrescu wrote:

 But all of a sudden, people get sugar in their gasoline and sand in their gears
 when they hear of the same thing applied to modules. I suspect it's because
some
 legacy languages didn't quite do a stellar job at it. Figuring that stuff out
 and pointing out problems is exactly the kind of thing computers are good at
and
 humans, not so good.


I get sand in my gears about it because it's hard to mentally figure out what
is 
happening. Humans, not so good at that.
Jun 10 2015
prev sibling next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 10 June 2015 at 16:03:17 UTC, Andrei Alexandrescu 
wrote:

 On 6/10/15 8:41 AM, Brian Rogoff wrote:

 I agree with you, Walter, that mutual recursion amongst 
 imports is
 usually bad, but I think you inadvertently made the right 
 decision by
 allowing it in D, and that the bug should be fixed. If people 
 overuse it
 to write messy code then the community should tell them to 
 write that
 code better.


 I agree. I'm one of those folks who think circular dependencies 
 are fine, thank you very much, and the few cases in which it 
 causes problems should be flagged and diagnosed in a principled 
 way.

 Everybody also likes:

 * recursion and mutual recursion of any depth (there's a lot 
 more of it going in the usual programs than one might think)

 * recursive and mutually recursive types

 * declaring top level names in any order and they just "work"

 But all of a sudden, people get sugar in their gasoline and 
 sand in their gears when they hear of the same thing applied to 
 modules. I suspect it's because some legacy languages didn't 
 quite do a stellar job at it. Figuring that stuff out and 
 pointing out problems is exactly the kind of thing computers 
 are good at and humans, not so good.


 Andrei


Cyclic imports create problems that do not exists with forward 
declaration. This is surprisingly hard to get a well defined 
behavior for compile time feature. I would argue to make them 
illegal, but that is still a good idea to avoid them when 
possible.
Jun 10 2015
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 06/10/2015 11:09 PM, deadalnix wrote:

 On Wednesday, 10 June 2015 at 16:03:17 UTC, Andrei Alexandrescu wrote:

 On 6/10/15 8:41 AM, Brian Rogoff wrote:

 I agree with you, Walter, that mutual recursion amongst imports is
 usually bad, but I think you inadvertently made the right decision by
 allowing it in D, and that the bug should be fixed. If people overuse it
 to write messy code then the community should tell them to write that
 code better.


 I agree. I'm one of those folks who think circular dependencies are
 fine, thank you very much, and the few cases in which it causes
 problems should be flagged and diagnosed in a principled way.

 Everybody also likes:

 * recursion and mutual recursion of any depth (there's a lot more of
 it going in the usual programs than one might think)

 * recursive and mutually recursive types

 * declaring top level names in any order and they just "work"

 But all of a sudden, people get sugar in their gasoline and sand in
 their gears when they hear of the same thing applied to modules. I
 suspect it's because some legacy languages didn't quite do a stellar
 job at it. Figuring that stuff out and pointing out problems is
 exactly the kind of thing computers are good at and humans, not so good.


 Andrei


 Cyclic imports create problems that do not exists with forward
 declaration.


No, they don't. They just preclude the band-aid fix.


 This is surprisingly hard to get a well defined behavior
 for compile time feature.


This is a good thing because it makes reasonably sure the final 
resolution rules won't take order of declaration into account.


 I would argue to make them illegal, but that
 is still a good idea to avoid them when possible.


That's the lazy way. :o)

I really need to find some time to get back to D frontend implementation.

What does SDC do now?
Jun 10 2015
next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 10 June 2015 at 21:21:35 UTC, Timon Gehr wrote:

 Cyclic imports create problems that do not exists with forward
 declaration.


 No, they don't. They just preclude the band-aid fix.


 This is surprisingly hard to get a well defined behavior
 for compile time feature.


 This is a good thing because it makes reasonably sure the final 
 resolution rules won't take order of declaration into account.


Well I haven't any solution that is both fast and work with 
cyclic import in a deterministic manner.


 I would argue to make them illegal, but that
 is still a good idea to avoid them when possible.


 That's the lazy way. :o)

 I really need to find some time to get back to D frontend 
 implementation.

 What does SDC do now?


Pretty much what is proposed in DIP31
Jun 10 2015
prev sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 10 June 2015 at 21:21:35 UTC, Timon Gehr wrote:

 I would argue to make them illegal, but that
 is still a good idea to avoid them when possible.


 That's the lazy way. :o)


Also, I forgot a not in there. I would NOT prevent it, but still 
would avoid it in code when possible.
Jun 10 2015
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 06/11/2015 12:15 AM, deadalnix wrote:

 On Wednesday, 10 June 2015 at 21:21:35 UTC, Timon Gehr wrote:

 I would argue to make them illegal, but that
 is still a good idea to avoid them when possible.


 That's the lazy way. :o)


 Also, I forgot a not in there. I would NOT prevent it, but still would
 avoid it in code when possible.


Oh, that's quite relevant. I take back my statement. :)
Jun 10 2015
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2015-06-10 18:03, Andrei Alexandrescu wrote:


 * declaring top level names in any order and they just "work"


Even Objective-C has got this feature now.

-- 
/Jacob Carlborg
Jun 10 2015
prev sibling parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Wednesday, 10 June 2015 at 01:43:41 UTC, deadalnix wrote:

 https://issues.dlang.org/show_bug.cgi?id=14666

 This bug has become a blocker for SDC. I was able to work 
 around it in various places so far, but this becomes harder and 
 harder to do so.


Good news! This is a regression in D 2.061.

OK, that's not really good news, but regressions do have a higher 
likelihood of getting fixed.
Jun 09 2015