www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - A possible solution for the opIndexXxxAssign morass

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Right now we're in trouble with operators: opIndex and opIndexAssign 
don't seem to be up to snuff because they don't catch operations like

a[b] += c;

with reasonable expressiveness and efficiency.

Last night this idea occurred to me: we could simply use overloading 
with the existing operator names. Consider:

a += b

gets rewritten as

a.opAddAssign(b)

Then how about this - rewrite this:

a[b] += c

as

a.opAddAssign(b, c);

There's no chance of ambiguity because the parameter counts are 
different. Moreover, this scales to multiple indexes:

a[b1, b2, ..., bn] = c

gets rewritten as

a.opAddAssign(b1, b2, ..., bn, c)

What do you think? I may be missing some important cases or threats.


Andrei
Oct 13 2009
next sibling parent reply Don <nospam nospam.com> writes:
Andrei Alexandrescu wrote:
 Right now we're in trouble with operators: opIndex and opIndexAssign 
 don't seem to be up to snuff because they don't catch operations like
 
 a[b] += c;
 
 with reasonable expressiveness and efficiency.
 
 Last night this idea occurred to me: we could simply use overloading 
 with the existing operator names. Consider:
 
 a += b
 
 gets rewritten as
 
 a.opAddAssign(b)
 
 Then how about this - rewrite this:
 
 a[b] += c
 
 as
 
 a.opAddAssign(b, c);
 
 There's no chance of ambiguity because the parameter counts are 
 different. Moreover, this scales to multiple indexes:
 
 a[b1, b2, ..., bn] = c
 
 gets rewritten as
 
 a.opAddAssign(b1, b2, ..., bn, c)
 
 What do you think? I may be missing some important cases or threats.
 
 
 Andrei

Well timed. I just wrote this operator overloading proposal, part 1. http://www.prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7 I concentrated on getting the use cases established. The indexing thing was something I didn't have a solution for. BTW we need to deal with slices as well as indexes. I think the way to do this is to make a slice into a type of index.
Oct 13 2009
next sibling parent "Robert Jacques" <sandford jhu.edu> writes:
On Tue, 13 Oct 2009 11:56:36 -0400, Don <nospam nospam.com> wrote:

 Andrei Alexandrescu wrote:
 Right now we're in trouble with operators: opIndex and opIndexAssign  
 don't seem to be up to snuff because they don't catch operations like
  a[b] += c;
  with reasonable expressiveness and efficiency.
  Last night this idea occurred to me: we could simply use overloading  
 with the existing operator names. Consider:
  a += b
  gets rewritten as
  a.opAddAssign(b)
  Then how about this - rewrite this:
  a[b] += c
  as
  a.opAddAssign(b, c);
  There's no chance of ambiguity because the parameter counts are  
 different. Moreover, this scales to multiple indexes:
  a[b1, b2, ..., bn] = c
  gets rewritten as
  a.opAddAssign(b1, b2, ..., bn, c)
  What do you think? I may be missing some important cases or threats.
   Andrei

Well timed. I just wrote this operator overloading proposal, part 1. http://www.prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7 I concentrated on getting the use cases established. The indexing thing was something I didn't have a solution for. BTW we need to deal with slices as well as indexes. I think the way to do this is to make a slice into a type of index.

I've mentioned this problem before, in relation to multi-dimensional arrays: // Slice a row out of an Matrix row0 = myMatrix[0,0..$]; So basically, opIndex and opSlice need to merge to support this use case. I've always ended up doing this with using size_t[2] or size_t[3] (for slicing with strides) when I've coded Nd-arrays, though this is a bit clunky. However, a while ago someone mentioned that tuple, though cool/useful/etc wasn't being used as much (compared to other languages) because of a lack of syntactic sugar. Which gave me the idea of using the .. operator to be syntactic sugar for tuple, as it would solve two birds with one stone. (Maybe three, if you count MVR) Also needed is an extension of the opDollar to return different values based on the index: opDollar(size_t index); P.S. There's also at least one template bug blocking Nd-arrays and small vector types: (http://d.puremagic.com/issues/show_bug.cgi?id=2257). P.S.S. Another template issue is that templating both opX and opX_r generally results in an overload conflict.
Oct 13 2009
prev sibling next sibling parent Bill Baxter <wbaxter gmail.com> writes:
On Tue, Oct 13, 2009 at 8:56 AM, Don <nospam nospam.com> wrote:
 Well timed. I just wrote this operator overloading proposal, part 1.
 http://www.prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7
 I concentrated on getting the use cases established.

 The indexing thing was something I didn't have a solution for.

 BTW we need to deal with slices as well as indexes. I think the way to do
 this is to make a slice into a type of index.

I think it's a good start. In the list of properties, you should probably mention that 'a' is a scalar. But I wonder how the rules involving scalars could be enforced, given that it's possible to define new scalar types. One would have to tell the compiler somehow which types are scalars relative to the type being defined. The ++ operators don't make sense for many of the types you listed. Maybe that should be broken out. Actually three of the things really stand out as computer-isms that don't really belong with the other mathematical properties: x =y <==> x = x y x++ <==> ++x x =y returns x Also, you can add Clifford algebra, Grassmann algebra, and geometric algebra to the list of things where the mathematical properties hold. --bb
Oct 13 2009
prev sibling next sibling parent reply "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
Don wrote:
 Andrei Alexandrescu wrote:
 Right now we're in trouble with operators: opIndex and opIndexAssign 
 don't seem to be up to snuff because they don't catch operations like

 a[b] += c;

 with reasonable expressiveness and efficiency.

 Last night this idea occurred to me: we could simply use overloading 
 with the existing operator names. Consider:

 a += b

 gets rewritten as

 a.opAddAssign(b)

 Then how about this - rewrite this:

 a[b] += c

 as

 a.opAddAssign(b, c);

 There's no chance of ambiguity because the parameter counts are 
 different. Moreover, this scales to multiple indexes:

 a[b1, b2, ..., bn] = c

 gets rewritten as

 a.opAddAssign(b1, b2, ..., bn, c)

 What do you think? I may be missing some important cases or threats.


 Andrei

Well timed. I just wrote this operator overloading proposal, part 1. http://www.prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7 I concentrated on getting the use cases established. The indexing thing was something I didn't have a solution for. BTW we need to deal with slices as well as indexes. I think the way to do this is to make a slice into a type of index.

I like the idea of enforcing relationships between operators. In fact, I think we can take it even further, and require that operator overloading in general *must* follow mathematical rules, and anything else leads to undefined behaviour. For example, if n is an integer, a and b are scalars, and x and y are general types, the compiler should be free to rewrite n*x <--> x + x + ... + x <--> 2*x + 2*x + ... x^^n <--> x * x * ... * x <--> x^^2 * x^^2 * ... x/a + y/b <--> (b*x + a*y)/(a*b) and so on, based on what it finds to be the most efficient operations. (Note how I snuck my favourite suggestion for an exponentiation operator in there. I *really* want that.) -Lars
Oct 14 2009
parent reply Don <nospam nospam.com> writes:
Lars T. Kyllingstad wrote:
 Don wrote:
 Andrei Alexandrescu wrote:
 Right now we're in trouble with operators: opIndex and opIndexAssign 
 don't seem to be up to snuff because they don't catch operations like

 a[b] += c;

 with reasonable expressiveness and efficiency.

 Last night this idea occurred to me: we could simply use overloading 
 with the existing operator names. Consider:

 a += b

 gets rewritten as

 a.opAddAssign(b)

 Then how about this - rewrite this:

 a[b] += c

 as

 a.opAddAssign(b, c);

 There's no chance of ambiguity because the parameter counts are 
 different. Moreover, this scales to multiple indexes:

 a[b1, b2, ..., bn] = c

 gets rewritten as

 a.opAddAssign(b1, b2, ..., bn, c)

 What do you think? I may be missing some important cases or threats.


 Andrei

Well timed. I just wrote this operator overloading proposal, part 1. http://www.prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7 I concentrated on getting the use cases established. The indexing thing was something I didn't have a solution for. BTW we need to deal with slices as well as indexes. I think the way to do this is to make a slice into a type of index.

I like the idea of enforcing relationships between operators. In fact, I think we can take it even further, and require that operator overloading in general *must* follow mathematical rules, and anything else leads to undefined behaviour. For example, if n is an integer, a and b are scalars, and x and y are general types, the compiler should be free to rewrite n*x <--> x + x + ... + x <--> 2*x + 2*x + ... x^^n <--> x * x * ... * x <--> x^^2 * x^^2 * ... x/a + y/b <--> (b*x + a*y)/(a*b) and so on, based on what it finds to be the most efficient operations.

Unfortunately, the last one doesn't work for reals. a*b could overflow or underflow. x/ real.max + y / real.max is exactly 2.0 if x and y are both real.max But (real.max * x + real.max *y)/(real.max * real.max) is infinity/infinity = NaN. The others don't always work in general, either. I'm worried about decimal floats. Say n==10, then it's an exact operation; but addition isn't exact. It always works for n==2, since there's at most one roundoff in both cases. But I do feel that with floating-point, we've lost so many identities, that we must preserve every one which we have left.
 (Note how I snuck my favourite suggestion for an exponentiation operator 
 in there. I *really* want that.)

I want it too. Heck, I might even make a patch for it <g>.
Oct 14 2009
parent "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
Don wrote:
 Lars T. Kyllingstad wrote:
 Don wrote:
 Andrei Alexandrescu wrote:
 Right now we're in trouble with operators: opIndex and opIndexAssign 
 don't seem to be up to snuff because they don't catch operations like

 a[b] += c;

 with reasonable expressiveness and efficiency.

 Last night this idea occurred to me: we could simply use overloading 
 with the existing operator names. Consider:

 a += b

 gets rewritten as

 a.opAddAssign(b)

 Then how about this - rewrite this:

 a[b] += c

 as

 a.opAddAssign(b, c);

 There's no chance of ambiguity because the parameter counts are 
 different. Moreover, this scales to multiple indexes:

 a[b1, b2, ..., bn] = c

 gets rewritten as

 a.opAddAssign(b1, b2, ..., bn, c)

 What do you think? I may be missing some important cases or threats.


 Andrei

Well timed. I just wrote this operator overloading proposal, part 1. http://www.prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7 I concentrated on getting the use cases established. The indexing thing was something I didn't have a solution for. BTW we need to deal with slices as well as indexes. I think the way to do this is to make a slice into a type of index.

I like the idea of enforcing relationships between operators. In fact, I think we can take it even further, and require that operator overloading in general *must* follow mathematical rules, and anything else leads to undefined behaviour. For example, if n is an integer, a and b are scalars, and x and y are general types, the compiler should be free to rewrite n*x <--> x + x + ... + x <--> 2*x + 2*x + ... x^^n <--> x * x * ... * x <--> x^^2 * x^^2 * ... x/a + y/b <--> (b*x + a*y)/(a*b) and so on, based on what it finds to be the most efficient operations.

Unfortunately, the last one doesn't work for reals. a*b could overflow or underflow. x/ real.max + y / real.max is exactly 2.0 if x and y are both real.max But (real.max * x + real.max *y)/(real.max * real.max) is infinity/infinity = NaN.

Good point. I am thinking like a mathematician, not a programmer. :)
 The others don't always work in general, either. I'm worried about 
 decimal floats. Say n==10, then it's an exact operation; but addition 
 isn't exact. It always works for n==2, since there's at most one 
 roundoff in both cases.

But the case x*2 --> x+x would also likely be the most common in terms of optimisation, right?
 But I do feel that with floating-point, we've lost so many identities, 
 that we must preserve every one which we have left.
 
 (Note how I snuck my favourite suggestion for an exponentiation 
 operator in there. I *really* want that.)

I want it too. Heck, I might even make a patch for it <g>.

If you do, make sure to announce it loudly and clearly on the NG. Don't want to miss it. ;) -Lars
Oct 14 2009
prev sibling next sibling parent Bill Baxter <wbaxter gmail.com> writes:
On Wed, Oct 14, 2009 at 12:48 AM, Lars T. Kyllingstad
<public kyllingen.nospamnet> wrote:
 Don wrote:
 Andrei Alexandrescu wrote:
 Right now we're in trouble with operators: opIndex and opIndexAssign
 don't seem to be up to snuff because they don't catch operations like

 a[b] +=3D c;

 with reasonable expressiveness and efficiency.

 Last night this idea occurred to me: we could simply use overloading wi=



 the existing operator names. Consider:

 a +=3D b

 gets rewritten as

 a.opAddAssign(b)

 Then how about this - rewrite this:

 a[b] +=3D c

 as

 a.opAddAssign(b, c);

 There's no chance of ambiguity because the parameter counts are
 different. Moreover, this scales to multiple indexes:

 a[b1, b2, ..., bn] =3D c

 gets rewritten as

 a.opAddAssign(b1, b2, ..., bn, c)

 What do you think? I may be missing some important cases or threats.


 Andrei

Well timed. I just wrote this operator overloading proposal, part 1. http://www.prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7 I concentrated on getting the use cases established. The indexing thing was something I didn't have a solution for. BTW we need to deal with slices as well as indexes. I think the way to d=


 this is to make a slice into a type of index.

I like the idea of enforcing relationships between operators. In fact, I think we can take it even further, and require that operator overloading =

 general *must* follow mathematical rules, and anything else leads to
 undefined behaviour. For example, if n is an integer, a and b are scalars=

 and x and y are general types, the compiler should be free to rewrite

 =A0 =A0 =A0 =A0 n*x =A0<--> =A0x + x + ... + x =A0 =A0<--> =A02*x + 2*x +=

 =A0 =A0 =A0 =A0x^^n =A0<--> =A0x * x * ... * x =A0 =A0<--> =A0x^^2 * x^^2=

 =A0 x/a + y/b =A0<--> =A0(b*x + a*y)/(a*b)

 and so on, based on what it finds to be the most efficient operations. (N=

 how I snuck my favourite suggestion for an exponentiation operator in the=

 I *really* want that.)

You have to be careful when you go rewriting mathematical expressions on the computer, though. The numerical error for two mathematically identical expressions can be quite different when evaluated in finite precision arithmetic. I'd love an exponentiation operator, too. --bb
Oct 14 2009
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Don wrote:
 Well timed. I just wrote this operator overloading proposal, part 1.
 http://www.prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7
 I concentrated on getting the use cases established.

I'm not sure multiplication is generally commutative (e.g. in linear algebra it isn't). So why should a * x be interchangeable with x * a? Also, the much-discussed identity: x = y <--> x = x y is difficult to enforce statically in practice. I think some types would want to define both to achieve good efficiency. It would be hard for the compiler to render one unnecessary or to prove that the two are equivalent. Andrei
Oct 14 2009
parent Don <nospam nospam.com> writes:
Andrei Alexandrescu wrote:
 Don wrote:
 Well timed. I just wrote this operator overloading proposal, part 1.
 http://www.prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7
 I concentrated on getting the use cases established.

I'm not sure multiplication is generally commutative (e.g. in linear algebra it isn't). So why should a * x be interchangeable with x * a?

It only applies a is an int or real. Its purpose is to allow constant-folding in the compiler front-end (specifically, when a is a manifest constant).
 Also, the much-discussed identity:
 
 x  = y    <-->    x = x   y
 
 is difficult to enforce statically in practice. I think some types would 
 want to define both to achieve good efficiency. It would be hard for the 
 compiler to render one unnecessary or to prove that the two are equivalent.

Yes, it could not be enforced. But note that there would be no ambiguity as to which should be used in any given expression. I would propose that the opXXXAssign() variants should exist *only* for performance optimisation, and be completely divorced from the "+=" syntax (effectively, += would be discarded after the parsing step). My ancient Bugzilla proposal actually included opSubAssign() and opSubAssign_r() for x = x - y; and x = y - x; If the x = y <--> x = x y transformations became legal, this would allow unnecessary temporaries to be completely eliminated. The suggested transformation would be that x = x + y would be transformed into x.opAddAssign(y) whenever it exists, and x = y + x would become x.opAddAssign_r(y) The transformations would therefore be entirely predictable. It would make Numpy-style arithmetic impossible (where z=x; x+=y; modifies z, but z = x; x = x+y; does not modify z (under this proposal, the second would be transformed into the first)). Tightly defined semantics improve performance and reduce the potential for abuse. But, there are existing libraries/techniques which depend on C++'s cavalier, "anything goes" attitude to operator overloading. Are we able to sacrifice them?
Oct 14 2009
prev sibling next sibling parent "Robert Jacques" <sandford jhu.edu> writes:
On Wed, 14 Oct 2009 10:31:06 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 Don wrote:
 Well timed. I just wrote this operator overloading proposal, part 1.
 http://www.prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7
 I concentrated on getting the use cases established.

I'm not sure multiplication is generally commutative (e.g. in linear algebra it isn't). So why should a * x be interchangeable with x * a? Also, the much-discussed identity: x = y <--> x = x y is difficult to enforce statically in practice. I think some types would want to define both to achieve good efficiency. It would be hard for the compiler to render one unnecessary or to prove that the two are equivalent. Andrei

When a is a scaler, a * x <=> x * a generally holds. It's only when something isn't a scaler, i.e. x1 * x2 != x2 * x1, that community(?) doesn't hold.
Oct 14 2009
prev sibling next sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Wed, 14 Oct 2009 18:39:27 +0400, Robert Jacques <sandford jhu.edu>  
wrote:

 On Wed, 14 Oct 2009 10:31:06 -0400, Andrei Alexandrescu  
 <SeeWebsiteForEmail erdani.org> wrote:

 Don wrote:
 Well timed. I just wrote this operator overloading proposal, part 1.
 http://www.prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7
 I concentrated on getting the use cases established.

I'm not sure multiplication is generally commutative (e.g. in linear algebra it isn't). So why should a * x be interchangeable with x * a? Also, the much-discussed identity: x = y <--> x = x y is difficult to enforce statically in practice. I think some types would want to define both to achieve good efficiency. It would be hard for the compiler to render one unnecessary or to prove that the two are equivalent. Andrei

When a is a scaler, a * x <=> x * a generally holds. It's only when something isn't a scaler, i.e. x1 * x2 != x2 * x1, that community(?) doesn't hold.

It's commutativity (http://en.wikipedia.org/wiki/Commutativity)
Oct 14 2009
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 15 Oct 2009 02:58:51 -0400, Don <nospam nospam.com> wrote:

 Andrei Alexandrescu wrote:
 Also, the much-discussed identity:
  x  = y    <-->    x = x   y
  is difficult to enforce statically in practice. I think some types  
 would want to define both to achieve good efficiency. It would be hard  
 for the compiler to render one unnecessary or to prove that the two are  
 equivalent.

Yes, it could not be enforced. But note that there would be no ambiguity as to which should be used in any given expression. I would propose that the opXXXAssign() variants should exist *only* for performance optimisation, and be completely divorced from the "+=" syntax (effectively, += would be discarded after the parsing step). My ancient Bugzilla proposal actually included opSubAssign() and opSubAssign_r() for x = x - y; and x = y - x; If the x = y <--> x = x y transformations became legal, this would allow unnecessary temporaries to be completely eliminated. The suggested transformation would be that x = x + y would be transformed into x.opAddAssign(y) whenever it exists, and x = y + x would become x.opAddAssign_r(y) The transformations would therefore be entirely predictable.

Oh, I didn't realize that's what you meant. I thought that opXxxAssign was to be eliminated and x += y was to be transformed into x.opAssign(x.opXxx(y). I like this proposal better -- opXxxAssign can exist for optimization reasons, and enforcing the relationship between = and = by parsing one into the other. By parsing x += y into x = x + y, and allowing overloading of a chain of operations, you may even get more mileage out of something like x += y + z + w; Someone earlier suggested opXxx(a1, a2, ...) could be interpreted as an operator for dealing with chained operations. You could also maybe have an opChain or something that takes as arguments the operands and the operators to maybe perform some optimization (i.e. like reordering matrix operations). You should update your DIP to specify that opXxxAssign should be allowed for optimization purposes (BTW, classes could benefit from this, because then x += y *would* be the same as x = x + y). -Steve
Oct 15 2009
prev sibling next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 13 Oct 2009 11:16:01 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 Right now we're in trouble with operators: opIndex and opIndexAssign  
 don't seem to be up to snuff because they don't catch operations like

 a[b] += c;

 with reasonable expressiveness and efficiency.

 Last night this idea occurred to me: we could simply use overloading  
 with the existing operator names. Consider:

 a += b

 gets rewritten as

 a.opAddAssign(b)

 Then how about this - rewrite this:

 a[b] += c

 as

 a.opAddAssign(b, c);

 There's no chance of ambiguity because the parameter counts are  
 different. Moreover, this scales to multiple indexes:

 a[b1, b2, ..., bn] = c

 gets rewritten as

 a.opAddAssign(b1, b2, ..., bn, c)

I'm guessing you meant opAssign here, or meant to write +=?
 What do you think? I may be missing some important cases or threats.

It's simple, and gets rid of all opIndex operators except for opIndex itself. The question then becomes, what if you wanted to overload this? a[b][c] += d; You can do a[b] returns a ref. But then you now allow a[b] op x, thereby possibly exposing a private piece of info. This may or may not be important. I like the way your idea is going. -Steve
Oct 13 2009
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Steven Schveighoffer wrote:
 On Tue, 13 Oct 2009 11:16:01 -0400, Andrei Alexandrescu 
 <SeeWebsiteForEmail erdani.org> wrote:
 There's no chance of ambiguity because the parameter counts are 
 different. Moreover, this scales to multiple indexes:

 a[b1, b2, ..., bn] = c

 gets rewritten as

 a.opAddAssign(b1, b2, ..., bn, c)

I'm guessing you meant opAssign here, or meant to write +=?

Oh, sorry. I meant to write +=.
 What do you think? I may be missing some important cases or threats.

It's simple, and gets rid of all opIndex operators except for opIndex itself. The question then becomes, what if you wanted to overload this? a[b][c] += d; You can do a[b] returns a ref. But then you now allow a[b] op x, thereby possibly exposing a private piece of info. This may or may not be important. I like the way your idea is going.

Great. Indeed the proposed solution leaves a[b][c] += d problematic, and also prevents this potential development: a += b + c + d; to be rewritten as: a.opAddAssign(b, c, d); Andrei
Oct 13 2009
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Bill Baxter wrote:
 Huh? It didn't sound to me like it would get rid of anything, except
 for the use of the word "index" in many methods that have to do with
 index operations.  That just seems confusing to me.   I think the
 opIndexXxxAssign functions may need to be added, but adding them by
 overloading existing names doesn't seem a win to me.
 
 --bb

That's a good point. But something is inherently problematic about name explosion (In the proposed solution there is still an explosion in the count of functions that need to be written.) Now I realize there's also a need for opSliceXxxAssign, bleh. Unless we ascribe a distinct type to a .. b. Andrei
Oct 13 2009
prev sibling next sibling parent reply "Denis Koroskin" <2korden gmail.com> writes:
On Tue, 13 Oct 2009 19:16:01 +0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 Right now we're in trouble with operators: opIndex and opIndexAssign  
 don't seem to be up to snuff because they don't catch operations like

 a[b] += c;

 with reasonable expressiveness and efficiency.

 Last night this idea occurred to me: we could simply use overloading  
 with the existing operator names. Consider:

 a += b

 gets rewritten as

 a.opAddAssign(b)

 Then how about this - rewrite this:

 a[b] += c

 as

 a.opAddAssign(b, c);

 There's no chance of ambiguity because the parameter counts are  
 different. Moreover, this scales to multiple indexes:

 a[b1, b2, ..., bn] = c

 gets rewritten as

 a.opAddAssign(b1, b2, ..., bn, c)

 What do you think? I may be missing some important cases or threats.


 Andrei

How about this case: a[b1..b2] = c; ? I could be solved if b1..b2 would return some built-in range type, defined in object.d, though.
Oct 13 2009
next sibling parent "Robert Jacques" <sandford jhu.edu> writes:
On Tue, 13 Oct 2009 12:28:05 -0400, Denis Koroskin <2korden gmail.com>  
wrote:

 On Tue, 13 Oct 2009 19:16:01 +0400, Andrei Alexandrescu  
 <SeeWebsiteForEmail erdani.org> wrote:

 Right now we're in trouble with operators: opIndex and opIndexAssign  
 don't seem to be up to snuff because they don't catch operations like

 a[b] += c;

 with reasonable expressiveness and efficiency.

 Last night this idea occurred to me: we could simply use overloading  
 with the existing operator names. Consider:

 a += b

 gets rewritten as

 a.opAddAssign(b)

 Then how about this - rewrite this:

 a[b] += c

 as

 a.opAddAssign(b, c);

 There's no chance of ambiguity because the parameter counts are  
 different. Moreover, this scales to multiple indexes:

 a[b1, b2, ..., bn] = c

 gets rewritten as

 a.opAddAssign(b1, b2, ..., bn, c)

 What do you think? I may be missing some important cases or threats.


 Andrei

How about this case: a[b1..b2] = c; ? I could be solved if b1..b2 would return some built-in range type, defined in object.d, though.

That already has an operator: int opSliceAssign(int v, size_t x, size_t y); a[3..4] = v; // same as a.opSliceAssign(v,3,4);
Oct 13 2009
prev sibling next sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Tue, 13 Oct 2009 20:34:06 +0400, Robert Jacques <sandford jhu.edu>  
wrote:

 On Tue, 13 Oct 2009 12:28:05 -0400, Denis Koroskin <2korden gmail.com>  
 wrote:

 On Tue, 13 Oct 2009 19:16:01 +0400, Andrei Alexandrescu  
 <SeeWebsiteForEmail erdani.org> wrote:

 Right now we're in trouble with operators: opIndex and opIndexAssign  
 don't seem to be up to snuff because they don't catch operations like

 a[b] += c;

 with reasonable expressiveness and efficiency.

 Last night this idea occurred to me: we could simply use overloading  
 with the existing operator names. Consider:

 a += b

 gets rewritten as

 a.opAddAssign(b)

 Then how about this - rewrite this:

 a[b] += c

 as

 a.opAddAssign(b, c);

 There's no chance of ambiguity because the parameter counts are  
 different. Moreover, this scales to multiple indexes:

 a[b1, b2, ..., bn] = c

 gets rewritten as

 a.opAddAssign(b1, b2, ..., bn, c)

 What do you think? I may be missing some important cases or threats.


 Andrei

How about this case: a[b1..b2] = c; ? I could be solved if b1..b2 would return some built-in range type, defined in object.d, though.

That already has an operator: int opSliceAssign(int v, size_t x, size_t y); a[3..4] = v; // same as a.opSliceAssign(v,3,4);

I meant: a[b1..b2] += c; Another thing I dislike about this proposal is that "a[b] += c;" translates into "opAddAssign" and doesn't mention "index" while "a[b] = c;" does ("opIndexAssign").
Oct 13 2009
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 13 Oct 2009 12:44:21 -0400, Denis Koroskin <2korden gmail.com>  
wrote:

 Another thing I dislike about this proposal is that "a[b] += c;"  
 translates into "opAddAssign" and doesn't mention "index" while "a[b] =  
 c;" does ("opIndexAssign").

I think the optimization translates to opAssign as well: a[b] = c; => a.opAssign(b, c); On Tue, 13 Oct 2009 12:37:50 -0400, Bill Baxter <wbaxter gmail.com> wrote:
 Huh? It didn't sound to me like it would get rid of anything, except
 for the use of the word "index" in many methods that have to do with
 index operations.  That just seems confusing to me.   I think the
 opIndexXxxAssign functions may need to be added, but adding them by
 overloading existing names doesn't seem a win to me.

The point is to avoid having operator function names multiply out of control. Re-examining it, I agree with you -- It makes little sense to have an operator that involves an indexing lack the term Index. If only there was a way to make the indexing orthogonal to the other operation. For example something like: struct X { private int[] arr; opIndex(int idx) // declares a new "namespace" where idx is an implicitly passed argument { int opAssign(int x) { arr[idx] = x; return x; } } } I know this probably doesn't parse well, should opIndex be a keyword? or an attribute? -Steve
Oct 13 2009
prev sibling parent Bill Baxter <wbaxter gmail.com> writes:
On Tue, Oct 13, 2009 at 10:00 AM, Steven Schveighoffer
<schveiguy yahoo.com> wrote:
 On Tue, 13 Oct 2009 12:44:21 -0400, Denis Koroskin <2korden gmail.com>
 wrote:

 Another thing I dislike about this proposal is that "a[b] +=3D c;"
 translates into "opAddAssign" and doesn't mention "index" while "a[b] =


 does ("opIndexAssign").

I think the optimization translates to opAssign as well: a[b] =3D c; =3D> a.opAssign(b, c); On Tue, 13 Oct 2009 12:37:50 -0400, Bill Baxter <wbaxter gmail.com> wrote=

 Huh? It didn't sound to me like it would get rid of anything, except
 for the use of the word "index" in many methods that have to do with
 index operations. =A0That just seems confusing to me. =A0 I think the
 opIndexXxxAssign functions may need to be added, but adding them by
 overloading existing names doesn't seem a win to me.

The point is to avoid having operator function names multiply out of control. =A0Re-examining it, I agree with you -- It makes little sense to=

 an operator that involves an indexing lack the term Index.

 If only there was a way to make the indexing orthogonal to the other
 operation. =A0For example something like:

 struct X
 {
 =A0 private int[] arr;
 =A0 opIndex(int idx) // declares a new "namespace" where idx is an implic=

 passed argument
 =A0 {
 =A0 =A0 =A0int opAssign(int x)
 =A0 =A0 =A0{
 =A0 =A0 =A0 =A0 arr[idx] =3D x;
 =A0 =A0 =A0 =A0 return x;
 =A0 =A0 =A0}
 =A0 }
 }

 I know this probably doesn't parse well, should opIndex be a keyword? or =

 attribute?

I don't think the number of /names/ required is the problem. It's just the sheer number of functions themselves that's the issue. I think a lot of that could mostly be fixed by some smart macros. And until they exist, mixins can help. struct Vec { float x,y,z; mixin(implementOperators("+ - / * +=3D -=3D /=3D *=3D", q{ a.x op=3D b.x; a.y op=3D b.y; a.z op=3D b.z; }); } The code gives a list of operators to implement and one prototypical op=3D body. With a smart enough CTFE string function that's all you need to generate all the listed operators. Not sure how to work index operators into that. --bb
Oct 13 2009
prev sibling next sibling parent Bill Baxter <wbaxter gmail.com> writes:
On Tue, Oct 13, 2009 at 9:08 AM, Steven Schveighoffer
<schveiguy yahoo.com> wrote:
 On Tue, 13 Oct 2009 11:16:01 -0400, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 Right now we're in trouble with operators: opIndex and opIndexAssign don't
 seem to be up to snuff because they don't catch operations like

 a[b] += c;

 with reasonable expressiveness and efficiency.

 Last night this idea occurred to me: we could simply use overloading with
 the existing operator names. Consider:

 a += b

 gets rewritten as

 a.opAddAssign(b)

 Then how about this - rewrite this:

 a[b] += c

 as

 a.opAddAssign(b, c);

 There's no chance of ambiguity because the parameter counts are different.
 Moreover, this scales to multiple indexes:

 a[b1, b2, ..., bn] = c

 gets rewritten as

 a.opAddAssign(b1, b2, ..., bn, c)

I'm guessing you meant opAssign here, or meant to write +=?
 What do you think? I may be missing some important cases or threats.

It's simple, and gets rid of all opIndex operators except for opIndex itself.

Huh? It didn't sound to me like it would get rid of anything, except for the use of the word "index" in many methods that have to do with index operations. That just seems confusing to me. I think the opIndexXxxAssign functions may need to be added, but adding them by overloading existing names doesn't seem a win to me. --bb
Oct 13 2009
prev sibling next sibling parent "Robert Jacques" <sandford jhu.edu> writes:
On Tue, 13 Oct 2009 12:21:20 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 Steven Schveighoffer wrote:
 On Tue, 13 Oct 2009 11:16:01 -0400, Andrei Alexandrescu  
 <SeeWebsiteForEmail erdani.org> wrote:
 There's no chance of ambiguity because the parameter counts are  
 different. Moreover, this scales to multiple indexes:

 a[b1, b2, ..., bn] = c

 gets rewritten as

 a.opAddAssign(b1, b2, ..., bn, c)


Oh, sorry. I meant to write +=.
 What do you think? I may be missing some important cases or threats.

itself. The question then becomes, what if you wanted to overload this? a[b][c] += d; You can do a[b] returns a ref. But then you now allow a[b] op x, thereby possibly exposing a private piece of info. This may or may not be important. I like the way your idea is going.

Great. Indeed the proposed solution leaves a[b][c] += d problematic, and also prevents this potential development: a += b + c + d; to be rewritten as: a.opAddAssign(b, c, d); Andrei

Well, that last case I'd prefer handled by something more generic, like an opExpression(Expr, T...)(T params); (i.e. a way of doing expression template/ BLADE like stuff, without the syntactic or runtime overhead.
Oct 13 2009
prev sibling next sibling parent Michel Fortin <michel.fortin michelf.com> writes:
On 2009-10-13 11:16:01 -0400, Andrei Alexandrescu 
<SeeWebsiteForEmail erdani.org> said:

 Right now we're in trouble with operators: opIndex and opIndexAssign 
 don't seem to be up to snuff because they don't catch operations like
 
 a[b] += c;
 
 with reasonable expressiveness and efficiency.
 
 Last night this idea occurred to me: we could simply use overloading 
 with the existing operator names. Consider:
 
 a += b
 
 gets rewritten as
 
 a.opAddAssign(b)
 
 Then how about this - rewrite this:
 
 a[b] += c
 
 as
 
 a.opAddAssign(b, c);

I'd rewrite it as opIndexAddAssign(b, c); That way you can also rewrite: a[b..c] = d; as opSliceAddAssign(b, c, d);
 There's no chance of ambiguity because the parameter counts are 
 different. Moreover, this scales to multiple indexes:
 
 a[b1, b2, ..., bn] = c
 
 gets rewritten as
 
 a.opAddAssign(b1, b2, ..., bn, c)

That looks like a good idea, although I'd be a little tempted to put the variable-length part at the end so you can easily choose to use variadic arguments. Also noteworthy: none of this work if you want to mix index and slices: a[b, c..d] = f;
 What do you think? I may be missing some important cases or threats.

Wasn't the bigger problem with operator overloading the fact that you have to redefine it for every primitive operator? I seem to recall you arguing for a way to overload all the operators at the same time. Where's that going? -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Oct 13 2009
prev sibling next sibling parent reply JC <jcrapuchettes gmail.com> writes:
This idea along with a slice overload would save me a lot of pain and 
performance while working with matrices in my production code. Any ideas when 
this could be implemented in D1?
Jonathan

Andrei Alexandrescu wrote:
 Right now we're in trouble with operators: opIndex and opIndexAssign 
 don't seem to be up to snuff because they don't catch operations like
 
 a[b] += c;
 
 with reasonable expressiveness and efficiency.
 
 Last night this idea occurred to me: we could simply use overloading 
 with the existing operator names. Consider:
 
 a += b
 
 gets rewritten as
 
 a.opAddAssign(b)
 
 Then how about this - rewrite this:
 
 a[b] += c
 
 as
 
 a.opAddAssign(b, c);
 
 There's no chance of ambiguity because the parameter counts are 
 different. Moreover, this scales to multiple indexes:
 
 a[b1, b2, ..., bn] = c
 
 gets rewritten as
 
 a.opAddAssign(b1, b2, ..., bn, c)
 
 What do you think? I may be missing some important cases or threats.
 
 
 Andrei

Oct 13 2009
parent Bill Baxter <wbaxter gmail.com> writes:
On Tue, Oct 13, 2009 at 10:22 AM, JC <jcrapuchettes gmail.com> wrote:
 This idea along with a slice overload would save me a lot of pain and
 performance while working with matrices in my production code. Any ideas
 when this could be implemented in D1?
 Jonathan

It won't be implemented in D1. Stability -- it's the beauty and the beast of D1. --bb
Oct 13 2009
prev sibling next sibling parent Bill Baxter <wbaxter gmail.com> writes:
On Tue, Oct 13, 2009 at 10:08 AM, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:
 Bill Baxter wrote:
 Huh? It didn't sound to me like it would get rid of anything, except
 for the use of the word "index" in many methods that have to do with
 index operations. =A0That just seems confusing to me. =A0 I think the
 opIndexXxxAssign functions may need to be added, but adding them by
 overloading existing names doesn't seem a win to me.

 --bb

That's a good point. But something is inherently problematic about name explosion (In the proposed solution there is still an explosion in the co=

 of functions that need to be written.)

 Now I realize there's also a need for opSliceXxxAssign, bleh. Unless we
 ascribe a distinct type to a .. b.

Yeh, the name explosion is just a symptom of the real problem, which is function count explosion. That's what needs fixing, if anything. But I don't really think having a lot of functions is the issue, it's implementers having to *write* a lot of boring repetitive functions that is the problem. So if the drudgery can be automated somehow (in the cases where the pattern is regular), then that would solve the problem in my mind. Even if it was still a function explosion under the hood. --bb
Oct 13 2009
prev sibling next sibling parent "Robert Jacques" <sandford jhu.edu> writes:
On Tue, 13 Oct 2009 13:08:59 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 Bill Baxter wrote:
 Huh? It didn't sound to me like it would get rid of anything, except
 for the use of the word "index" in many methods that have to do with
 index operations.  That just seems confusing to me.   I think the
 opIndexXxxAssign functions may need to be added, but adding them by
 overloading existing names doesn't seem a win to me.
  --bb

That's a good point. But something is inherently problematic about name explosion (In the proposed solution there is still an explosion in the count of functions that need to be written.) Now I realize there's also a need for opSliceXxxAssign, bleh. Unless we ascribe a distinct type to a .. b. Andrei

A distinct type for a..b is needed to support the mixed slicing and index that occurs in Nd-array/Matrixes: i.e. auto row0 = myMatrix[0,0..$];
Oct 13 2009
prev sibling parent reply Chad J <chadjoan __spam.is.bad__gmail.com> writes:
Forgotten already?

http://prowiki.org/wiki4d/wiki.cgi?DocComments/Property#Semantic

This is the same problem as property lvalue-ness and it has the same
solution.  When property rewriting is done correctly, the opIndexAssign
problem can then be solved almost for free.

Just treat opIndex expressions as properties, and when they are the
subject of a side-effect then make sure the write property (AKA
opIndexAssign) gets called.
Oct 13 2009
parent Bill Baxter <wbaxter gmail.com> writes:
On Tue, Oct 13, 2009 at 10:39 AM, Chad J
<chadjoan __spam.is.bad__gmail.com> wrote:
 Forgotten already?

Apparently, yes!
 http://prowiki.org/wiki4d/wiki.cgi?DocComments/Property#Semantic

 This is the same problem as property lvalue-ness and it has the same
 solution. =A0When property rewriting is done correctly, the opIndexAssign
 problem can then be solved almost for free.

 Just treat opIndex expressions as properties, and when they are the
 subject of a side-effect then make sure the write property (AKA
 opIndexAssign) gets called.

Good call. --bb
Oct 13 2009