www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Is it bad for object.di to depend on core.exception?

reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
So I'm still working on fixing issue 5030, which *should* have been a
trivial fix. But I'm running into a bunch of circumstantial problems,
among which is this new method in AssociativeArray(Key,Value):

    Value opIndex(Key key, string file=__FILE__, size_t line=__LINE__)
    {
    	auto p = key in *cast(Value[Key]*)(&p);
	if (p) return *p;
	throw new RangeError(file, line);
    }

Originally it was simply Value opIndex(Key key) without any range check,
which is probably a bad idea, so I added the throw RangeError in there.

However, this introduces a dependency from object.di to core.exception.
So this requires import core.exception in object.di, without which
Phobos doesn't compile (for obvious reasons). But adding the import
causes druntime to fail to compile, because, for example,
core.stdc.stdio is compiled withouth druntime/src in the compiler's
search path, so the compiler can't find core.exception.

So I'm wondering, is it OK to add druntime/src to the compiler's search
path when building druntime? Or is there a reason for the omission?

But perhaps a more pertinent question is, why is there so much
duplication between aaA.d and object.AssociativeArray? For example,
object.AssociativeArray basically copies (with modifications!) a bunch
of implementation-dependent details from aaA.d, and sometimes uses that,
and sometimes uses the other. Is it possible to dispense with this
schizophrenic code duplication (which is very fragile, since changing
the AA implementation in aaA.d will almost certainly break the Range
stuff in AssociativeArray)?


T

-- 
People say I'm arrogant, but they're just ignorant fools.
Mar 03 2012
next sibling parent reply "Daniel Murphy" <yebblies nospamgmail.com> writes:
"H. S. Teoh" <hsteoh quickfur.ath.cx> wrote in message 
news:mailman.370.1330829036.24984.digitalmars-d puremagic.com...
 So I'm still working on fixing issue 5030, which *should* have been a
 trivial fix. But I'm running into a bunch of circumstantial problems,
 among which is this new method in AssociativeArray(Key,Value):

    Value opIndex(Key key, string file=__FILE__, size_t line=__LINE__)
    {
    auto p = key in *cast(Value[Key]*)(&p);
 if (p) return *p;
 throw new RangeError(file, line);
    }

 Originally it was simply Value opIndex(Key key) without any range check,
 which is probably a bad idea, so I added the throw RangeError in there.

 However, this introduces a dependency from object.di to core.exception.
 So this requires import core.exception in object.di, without which
 Phobos doesn't compile (for obvious reasons). But adding the import
 causes druntime to fail to compile, because, for example,
 core.stdc.stdio is compiled withouth druntime/src in the compiler's
 search path, so the compiler can't find core.exception.

 So I'm wondering, is it OK to add druntime/src to the compiler's search
 path when building druntime? Or is there a reason for the omission?

 But perhaps a more pertinent question is, why is there so much
 duplication between aaA.d and object.AssociativeArray? For example,
 object.AssociativeArray basically copies (with modifications!) a bunch
 of implementation-dependent details from aaA.d, and sometimes uses that,
 and sometimes uses the other. Is it possible to dispense with this
 schizophrenic code duplication (which is very fragile, since changing
 the AA implementation in aaA.d will almost certainly break the Range
 stuff in AssociativeArray)?

Ok, let me tell you a story. Once apon a time AAs were part of the language proper, doing anything with them resulted in calls to druntime c linkage functions, which you can see in aaA.d. Then, a few of years ago they were moved into druntime, so that the implementation and interface could be improved without modifying the compiler. While only about three things were ever added to the AA interface, this had the major side effect of introducing dozens of ice bugs and _increasing_ the amount of code needed to support AAs, especially in the interpreter and the glue layer. On top of this, most of the functionality is still done with the compiler emitting druntime calls. In the last release, Andrei made things worse by copying the AA implementation layout into object.di in order to make ranges work. (To be fair there isn't really a better way to do this with the current setup.) We're basically stuck with this until someone comes up with a solution that lets everything work as it should, with AAs completely in the runtime or the compiler.
Mar 03 2012
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 3/3/12 9:19 PM, Daniel Murphy wrote:
 We're basically stuck with this until someone comes up with a solution that
 lets everything work as it should, with AAs completely in the runtime or the
 compiler.

I think we must migrate the arrays into the runtime. Andrei
Mar 03 2012
next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sat, Mar 03, 2012 at 09:47:49PM -0600, Andrei Alexandrescu wrote:
 On 3/3/12 9:19 PM, Daniel Murphy wrote:
We're basically stuck with this until someone comes up with a
solution that lets everything work as it should, with AAs completely
in the runtime or the compiler.

I think we must migrate the arrays into the runtime.

[...] What are the issues that currently prevent this? T -- Blunt statements really don't have a point.
Mar 03 2012
prev sibling parent reply "Daniel Murphy" <yebblies nospamgmail.com> writes:
"Andrei Alexandrescu" <SeeWebsiteForEmail erdani.org> wrote in message 
news:jiuol3$rqb$1 digitalmars.com...
 On 3/3/12 9:19 PM, Daniel Murphy wrote:
 We're basically stuck with this until someone comes up with a solution 
 that
 lets everything work as it should, with AAs completely in the runtime or 
 the
 compiler.

I think we must migrate the arrays into the runtime. Andrei

I know. It was before my time, but I assume you were the one who pushed for this in the first place? The one advantage I can see to having them in druntime gives is that it means they can be fully templated, and therefore do comparisons etc without going through typeinfo. This is it right?
Mar 03 2012
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sun, Mar 04, 2012 at 03:24:47PM +1100, Daniel Murphy wrote:
 "Andrei Alexandrescu" <SeeWebsiteForEmail erdani.org> wrote in message 
 news:jiuol3$rqb$1 digitalmars.com...

[...]
 I think we must migrate the arrays into the runtime.

 Andrei

I know. It was before my time, but I assume you were the one who pushed for this in the first place? The one advantage I can see to having them in druntime gives is that it means they can be fully templated, and therefore do comparisons etc without going through typeinfo. This is it right?

[...] Not just comparisons, but size of values, etc., all need typeinfo currently. Pretty much all the code in aaA.d needs to dig under the hood and do lots of black magic like pointer arithmetic based on typeinfo stuff in order to do stuff. Perhaps _aaLen might be the only exception. While it's cool that D lets you do this when needed, it's also a sign that some cleanup might be in order. :-) How feasible is it to migrate AA's into object_.d, say, and have the stuff in aaA.d simply wrap around AssociativeArray? I suppose this can't really happen without some major changes in the compiler, since I don't know how to instantiate AssociativeArray given only the typeinfo's (not the actual types) of the key/value. T -- It won't be covered in the book. The source code has to be useful for something, after all. -- Larry Wall
Mar 03 2012
prev sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sun, Mar 04, 2012 at 02:19:42PM +1100, Daniel Murphy wrote:
 "H. S. Teoh" <hsteoh quickfur.ath.cx> wrote in message 

[...]
 But perhaps a more pertinent question is, why is there so much
 duplication between aaA.d and object.AssociativeArray? For example,
 object.AssociativeArray basically copies (with modifications!) a
 bunch of implementation-dependent details from aaA.d, and sometimes
 uses that, and sometimes uses the other. Is it possible to dispense
 with this schizophrenic code duplication (which is very fragile,
 since changing the AA implementation in aaA.d will almost certainly
 break the Range stuff in AssociativeArray)?

Ok, let me tell you a story. Once apon a time AAs were part of the language proper, doing anything with them resulted in calls to druntime c linkage functions, which you can see in aaA.d.

That makes sense.
 Then, a few of years ago they were moved into druntime, so that the
 implementation and interface could be improved without modifying the
 compiler.

Is that when struct AssociativeArray came into being? I'm still trying to understand how exactly AssociativeArray is connected with the C linkage implementation. Is the compiler "aware" of AssociativeArray at all? How does it work when you declare, say, int[string] and try to make a range out of it?
 While only about three things were ever added to the AA interface,
 this had the major side effect of introducing dozens of ice bugs and
 _increasing_ the amount of code needed to support AAs, especially in
 the interpreter and the glue layer.  On top of this, most of the
 functionality is still done with the compiler emitting druntime calls.

And what are the issues preventing the compiler from *not* emitting these calls?
 In the last release, Andrei made things worse by copying the AA
 implementation layout into object.di in order to make ranges work.
 (To be fair there isn't really a better way to do this with the
 current setup.)

I see.
 We're basically stuck with this until someone comes up with a solution
 that lets everything work as it should, with AAs completely in the
 runtime or the compiler. 

[...] Heh. I just try to fix a few AA bugs and now I've to rewrite the thing from scratch? Let me think a bit more about that one. :) The question still stands, though, is there a reason for core.stdc.stdio to be compiled without the include path to druntime/src? T -- All men are mortal. Socrates is mortal. Therefore all men are Socrates.
Mar 03 2012
parent reply "Daniel Murphy" <yebblies nospamgmail.com> writes:
"H. S. Teoh" <hsteoh quickfur.ath.cx> wrote in message 
news:mailman.372.1330833051.24984.digitalmars-d puremagic.com...
 Then, a few of years ago they were moved into druntime, so that the
 implementation and interface could be improved without modifying the
 compiler.

Is that when struct AssociativeArray came into being? I'm still trying to understand how exactly AssociativeArray is connected with the C linkage implementation. Is the compiler "aware" of AssociativeArray at all? How does it work when you declare, say, int[string] and try to make a range out of it?

Yeah. The compiler is aware of AssociativeArray in that it gags semantic errors on it, and turns V[K] types into AssociativeArray!(K,V) types in some cases (like for member lookup). It's sortof messy, try grepping for getImpl in dmd.
 While only about three things were ever added to the AA interface,
 this had the major side effect of introducing dozens of ice bugs and
 _increasing_ the amount of code needed to support AAs, especially in
 the interpreter and the glue layer.  On top of this, most of the
 functionality is still done with the compiler emitting druntime calls.

And what are the issues preventing the compiler from *not* emitting these calls?

Well, for example: aa[k] = v; becomes *__d_aaGet(&aa, k, k.typeinfo) = v; // or something like this And iirc there is no way to currently do this through AssociativeArray.
 We're basically stuck with this until someone comes up with a solution
 that lets everything work as it should, with AAs completely in the
 runtime or the compiler.

[...] Heh. I just try to fix a few AA bugs and now I've to rewrite the thing from scratch? Let me think a bit more about that one. :)

This is basically the conclusion I've come to when I've tried to fix AA bugs and run into these problems. Patch the holes (and make things worse) or do a full redesign... neither have really inspired me to spend lots of time working on it.
 The question still stands, though, is there a reason for core.stdc.stdio
 to be compiled without the include path to druntime/src?

I doubt it, though it's probably not desirable to make object.d import anything else. Is there an extern c function you could call that would throw the range error for you? If you make a pull request the people who maintain druntime will look though it.
Mar 03 2012
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sun, Mar 04, 2012 at 03:34:26PM +1100, Daniel Murphy wrote:
 "H. S. Teoh" <hsteoh quickfur.ath.cx> wrote in message 
 news:mailman.372.1330833051.24984.digitalmars-d puremagic.com...

[...]
 Is that when struct AssociativeArray came into being? I'm still
 trying to understand how exactly AssociativeArray is connected with
 the C linkage implementation. Is the compiler "aware" of
 AssociativeArray at all? How does it work when you declare, say,
 int[string] and try to make a range out of it?

Yeah. The compiler is aware of AssociativeArray in that it gags semantic errors on it, and turns V[K] types into AssociativeArray!(K,V) types in some cases (like for member lookup). It's sortof messy, try grepping for getImpl in dmd.

Hmph. Looks like there's no way to avoid massive overhaul? :-/ [...]
 While only about three things were ever added to the AA interface,
 this had the major side effect of introducing dozens of ice bugs
 and _increasing_ the amount of code needed to support AAs,
 especially in the interpreter and the glue layer.  On top of this,
 most of the functionality is still done with the compiler emitting
 druntime calls.

And what are the issues preventing the compiler from *not* emitting these calls?

Well, for example: aa[k] = v; becomes *__d_aaGet(&aa, k, k.typeinfo) = v; // or something like this And iirc there is no way to currently do this through AssociativeArray.

But isn't this just a matter of overloading opIndex & friends? That's what I'm trying to do, at any rate.
 We're basically stuck with this until someone comes up with a
 solution that lets everything work as it should, with AAs
 completely in the runtime or the compiler.

[...] Heh. I just try to fix a few AA bugs and now I've to rewrite the thing from scratch? Let me think a bit more about that one. :)

This is basically the conclusion I've come to when I've tried to fix AA bugs and run into these problems. Patch the holes (and make things worse) or do a full redesign... neither have really inspired me to spend lots of time working on it.

Well, I'm not afraid of redesign per se -- I consider AA's an important enough part of D to be worth the effort to do right. But I just wanted to know what I'm up against. :P
 The question still stands, though, is there a reason for
 core.stdc.stdio to be compiled without the include path to
 druntime/src?

I doubt it, though it's probably not desirable to make object.d import anything else. Is there an extern c function you could call that would throw the range error for you? If you make a pull request the people who maintain druntime will look though it.

[...] I suppose I *could* add a function in aaA.d to throw the exception. Really ugly, but gets around needing an explicit dependency in object.di. I don't know which is worse though -- to make object.di depend on core.exception, or to make aaA.d depend on it. Both are in some ways circular dependencies. T -- Famous last words: I *think* this will work...
Mar 03 2012
prev sibling parent reply "Martin Nowak" <dawg dawgfoto.de> writes:
 T

Wasn't the latest proposal that we add a working AA implementation to the runtime and switch the compiler after that has settled?
Mar 03 2012
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sun, Mar 04, 2012 at 07:31:49AM +0100, Martin Nowak wrote:
 Wasn't the latest proposal that we add a working AA implementation to
 the runtime and switch the compiler after that has settled?

Is somebody working on that? If not, I may take a crack at doing it. And by runtime you mean object_.d, right? aaA.d is already part of druntime, from what I understand. :-) T -- If it breaks, you get to keep both pieces. -- Software disclaimer notice
Mar 07 2012
parent reply "Daniel Murphy" <yebblies nospamgmail.com> writes:
"H. S. Teoh" <hsteoh quickfur.ath.cx> wrote in message 
news:mailman.176.1331147910.4860.digitalmars-d puremagic.com...
 On Sun, Mar 04, 2012 at 07:31:49AM +0100, Martin Nowak wrote:
 Wasn't the latest proposal that we add a working AA implementation to
 the runtime and switch the compiler after that has settled?

Is somebody working on that? If not, I may take a crack at doing it. And by runtime you mean object_.d, right? aaA.d is already part of druntime, from what I understand. :-) T -- If it breaks, you get to keep both pieces. -- Software disclaimer notice

I was the one that recently tried to get AAs rolled back to the D1 implemenation (unsuccessfully) but I'm not working on a reimplementation. The closed pull requests are sitting on github if you'd like to take a look at them. For it to be 'fixed', AAs need to be able to do all the things they can do now, including producing error messages with V[K] instead of AssociativeArray!(K, V), magic initialization and the same reference semantics. They also need to _work_ in ctfe, either because the implementation is available and ctfeable, or because the calls get translated late enough that the interpreter can pick out the IndexExps/DotIdExps and treat them specially. At the moment I'm leaning towards doing the translations in the glue layer, and using ufcs for everything else, but whatever works. There's a thread on the internals mailing list that lists some of the issues with the current implementation.
Mar 07 2012
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Mar 08, 2012 at 12:37:20PM +1100, Daniel Murphy wrote:
 "H. S. Teoh" <hsteoh quickfur.ath.cx> wrote in message 
 news:mailman.176.1331147910.4860.digitalmars-d puremagic.com...
 On Sun, Mar 04, 2012 at 07:31:49AM +0100, Martin Nowak wrote:
 Wasn't the latest proposal that we add a working AA implementation
 to the runtime and switch the compiler after that has settled?

Is somebody working on that? If not, I may take a crack at doing it.


[...]
 I was the one that recently tried to get AAs rolled back to the D1
 implemenation (unsuccessfully) but I'm not working on a
 reimplementation.  The closed pull requests are sitting on github if
 you'd like to take a look at them.
 
 For it to be 'fixed', AAs need to be able to do all the things they
 can do now, including producing error messages with V[K] instead of
 AssociativeArray!(K, V), magic initialization and the same reference
 semantics.  They also need to _work_ in ctfe, either because the
 implementation is available and ctfeable, or because the calls get
 translated late enough that the interpreter can pick out the
 IndexExps/DotIdExps and treat them specially.
 
 At the moment I'm leaning towards doing the translations in the glue
 layer, and using ufcs for everything else, but whatever works.
 There's a thread on the internals mailing list that lists some of the
 issues with the current implementation. 

[...] Hmm. If this is going to involve major changes in *both* druntime and dmd, I may have bitten off a bit more than I can chew. :-/ I was going on the original statement that we build a workable AA implementation in object_.d first, then port the compiler over. Trying to do everything at once seems like too big a project to pull off successfully in one shot. T -- Without geometry, life would be pointless. -- VS
Mar 07 2012