www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Operator overloading

reply aarti_pl <aarti interia.pl> writes:
Andrei Alexandrescu pisze:
 We're trying to make that work. D is due for an operator overhaul.

 Andrei

Is there any chance that we get possibility to overload "raw operators", like in C++? I think that they may coexist with currently defined operator overloads with simple semantic rules, which will not allow them to work together at the same time. I know that they may obfuscate code, but they are extremely useful when defining in D domain specific languages. For example I try to make SQL expressions work as D expressions: string query = "SELECT * FROM a WHERE id=5;"; --> Query query = Select(a).Where(Equals(id, 5)); * "id" is D object in second expression This translation is quite awful and unreadable. It would be so much better to get: Query query = Select(a).Where(id == 5); With implicit casts there will be perfect environment, to define DSL-s *in* D and *not* using unsafe strings. BR Marcin Kuszczak (aarti_pl)
Dec 23 2008
next sibling parent reply Denis Koroskin <2korden gmail.com> writes:
aarti_pl Wrote:

 Andrei Alexandrescu pisze:
  > We're trying to make that work. D is due for an operator overhaul.
  >
  > Andrei
 
 Is there any chance that we get possibility to overload "raw operators", 
 like in C++? I think that they may coexist with currently defined 
 operator overloads with simple semantic rules, which will not allow them 
 to work together at the same time.
 
 I know that they may obfuscate code, but they are extremely useful when 
 defining in D domain specific languages. For example I try to make SQL 
 expressions work as D expressions:
 
 string query = "SELECT * FROM a WHERE id=5;";
 -->
 Query query = Select(a).Where(Equals(id, 5));
 
 * "id" is D object in second expression
 
 This translation is quite awful and unreadable. It would be so much 
 better to get:
 
 Query query = Select(a).Where(id == 5);
 
 With implicit casts there will be perfect environment, to define DSL-s 
 *in* D and *not* using unsafe strings.
 
 BR
 Marcin Kuszczak
 (aarti_pl)

Slightly off-topic. How would you implement, say, LIKE condition? Similar to the following, perhaps: Query query = Select(a).Where(id.Like("%substring%")); You could implement "==" the same way: Query query = Select(a).Where(id.EqualsTo(5));
Dec 23 2008
next sibling parent aarti_pl <aarti interia.pl> writes:
Denis Koroskin pisze:
 Slightly off-topic.
 
 How would you implement, say, LIKE condition? Similar to the following,
perhaps:
 Query query = Select(a).Where(id.Like("%substring%"));

Well, probably: ...Like(id, "substring")... In this case I prefer this kind of syntax more than OO kind of syntax :-)
 You could implement "==" the same way:
 Query query = Select(a).Where(id.EqualsTo(5));

In fact LIKE is not as a big problem as other operators. You will rather not use nested LIKEs, but only the simplest form. The best solution would be to introduce into language concept of infix operator, so that any symbol could be such an operator. It would be most universal solution. But it would be probably too much to ask... Completely different situation than with LIKE is with comparison and logical operators. They form very complicated expressions eg.: Where(Or(And(Equals(surname, "Smith"), NotEquals(age, 25))), And(Equals(surname, "Neo"), NotEquals(age, 50))) With my syntax it would be: Where((surname == "Smith" && age != 25) || (surname == "Neo" && age != 50)) I don't know how to write it using your syntax, because I don't have good idea for AND/OR. But the most important argument is that syntax, where operators are parts of classes will cause unnecessary coupling of classes and concepts. In my case objects which are used to identify columns are universal. Using same identifiers you can also get values from resulting table. When I implement parts of expression in Column classes I will loose this decoupling: someone who just wants my resulting table object + column identifiers will have to see api for sql expressions, which he/she will not use at all. Best Regards (aarti_pl)
Dec 23 2008
prev sibling parent reply Christopher Wright <dhasenan gmail.com> writes:
Denis Koroskin wrote:
 Slightly off-topic.
 
 How would you implement, say, LIKE condition? Similar to the following,
perhaps:
 Query query = Select(a).Where(id.Like("%substring%"));
 
 You could implement "==" the same way:
 Query query = Select(a).Where(id.EqualsTo(5));

downs way: auto query1 = select(a) /where/ "id" /eq/ 5; auto query2 = select(a) /where/ "name" /like/ "%bob%";
Dec 23 2008
next sibling parent reply Sergey Gromov <snake.scaly gmail.com> writes:
Tue, 23 Dec 2008 19:14:16 -0500, Christopher Wright wrote:

 Denis Koroskin wrote:
 Slightly off-topic.
 
 How would you implement, say, LIKE condition? Similar to the following,
perhaps:
 Query query = Select(a).Where(id.Like("%substring%"));
 
 You could implement "==" the same way:
 Query query = Select(a).Where(id.EqualsTo(5));

downs way: auto query1 = select(a) /where/ "id" /eq/ 5; auto query2 = select(a) /where/ "name" /like/ "%bob%";

What a nice trick! I don't like operator abuse, but sometimes it's a lesser evil. And it's definitely the case with SQL queries.
Dec 24 2008
parent aarti_pl <aarti interia.pl> writes:
Sergey Gromov pisze:
 downs way:
 auto query1 = select(a) /where/ "id" /eq/ 5;
 auto query2 = select(a) /where/ "name" /like/ "%bob%";

What a nice trick! I don't like operator abuse, but sometimes it's a lesser evil. And it's definitely the case with SQL queries.

I agree. In D raw operator overloading should be probably as discouraging as cast operator :-) e.g. opRawEqual() opRawNotEquals() etc. BR Marcin Kuszczak (aarti_pl)
Dec 24 2008
prev sibling parent aarti_pl <aarti interia.pl> writes:
Christopher Wright pisze:
 Denis Koroskin wrote:
 Slightly off-topic.

 How would you implement, say, LIKE condition? Similar to the 
 following, perhaps:
 Query query = Select(a).Where(id.Like("%substring%"));

 You could implement "==" the same way:
 Query query = Select(a).Where(id.EqualsTo(5));

downs way: auto query1 = select(a) /where/ "id" /eq/ 5; auto query2 = select(a) /where/ "name" /like/ "%bob%";

Nice, but I am not so sure that as much intuitive? :-) BR Marcin Kuszczak (aarti_pl)
Dec 24 2008
prev sibling next sibling parent reply dennis luehring <dl.soluz gmx.net> writes:
 ...
 This translation is quite awful and unreadable. It would be so much 
 better to get:
 
 Query query = Select(a).Where(id == 5);

what speaks against an sql parsing mixin? would be more expressive, compiletime based and typesafe and even far more be able than what you can do with operator overloading
Dec 23 2008
next sibling parent reply Ary Borenszweig <ary esperanto.org.ar> writes:
dennis luehring escribió:
 ...
 This translation is quite awful and unreadable. It would be so much 
 better to get:

 Query query = Select(a).Where(id == 5);

what speaks against an sql parsing mixin? would be more expressive, compiletime based and typesafe and even far more be able than what you can do with operator overloading

Autocompletion and syntax coloring.
Dec 23 2008
next sibling parent reply aarti_pl <aarti interia.pl> writes:
Ary Borenszweig pisze:
 dennis luehring escribió:
 ...
 This translation is quite awful and unreadable. It would be so much 
 better to get:

 Query query = Select(a).Where(id == 5);

what speaks against an sql parsing mixin? would be more expressive, compiletime based and typesafe and even far more be able than what you can do with operator overloading

Autocompletion and syntax coloring.

Yes. And additionally: * you can not build safely your queries on runtime * when moving Query around in program you will not be able to make it typesafe, because it is just string * in my db access system it won't be possible to automatically create database, typesafely get results from resulting table and few other nice features... BR Marcin Kuszczak (aarti_pl)
Dec 23 2008
parent reply dennis luehring <dl.soluz gmx.net> writes:
aarti_pl schrieb:
 Ary Borenszweig pisze:
 dennis luehring escribió:
 ...
 This translation is quite awful and unreadable. It would be so much 
 better to get:

 Query query = Select(a).Where(id == 5);

what speaks against an sql parsing mixin? would be more expressive, compiletime based and typesafe and even far more be able than what you can do with operator overloading

Autocompletion and syntax coloring.


 Yes. And additionally:
 * you can not build safely your queries on runtime

that even not possible with operators - or?
 * when moving Query around in program you will not be able to make it 
 typesafe, because it is just string

not if you parse the string at compiletime an use the type information from outer scope for validation
 * in my db access system it won't be possible to automatically create 
 database, typesafely get results from resulting table and few other nice 
 features...

just a quick look into blades (library with vector math language extensions - based on compiletime strings) http://www.dsource.org/projects/mathextra/browser/trunk/blade/BladeDemo.d don referes to outer vars in his "strings" and you can typecheck them
Dec 23 2008
parent reply aarti_pl <aarti interia.pl> writes:
dennis luehring pisze:
 aarti_pl schrieb:
 Ary Borenszweig pisze:
 dennis luehring escribió:
 ...
 This translation is quite awful and unreadable. It would be so much 
 better to get:

 Query query = Select(a).Where(id == 5);

what speaks against an sql parsing mixin? would be more expressive, compiletime based and typesafe and even far more be able than what you can do with operator overloading

Autocompletion and syntax coloring.


 Yes. And additionally:
 * you can not build safely your queries on runtime

that even not possible with operators - or?

It is possible and it works :-) But with operator overloading it is so much more readable, than without it. Creating queries at runtime is quite common: age = 20; string query = "Select name, surname FROM persons WHERE age = " + to!(string)(age);
 * when moving Query around in program you will not be able to make it 
 typesafe, because it is just string

not if you parse the string at compiletime an use the type information from outer scope for validation

Compile time checking of SQLs will not be enough for any bigger program. How would you insert your data into database? INSERT needs values which are taken e.g. from GUI and completely unknown at compile-time.
 * in my db access system it won't be possible to automatically create 
 database, typesafely get results from resulting table and few other 
 nice features...

just a quick look into blades (library with vector math language extensions - based on compiletime strings) http://www.dsource.org/projects/mathextra/browser/trunk/blade/BladeDemo.d don referes to outer vars in his "strings" and you can typecheck them

It doesn't solve all problems. Just a one. BTW. In recent release Boost team added a whole brand new library for domain specific languages embedded languages called Proto. I think that it means that there is quite a big need for such features. E.g. for mocking frameworks which will also profit from extended DSL sublanguages support in programming languages. BR Marcin Kuszczak (aarti_pl)
Dec 23 2008
parent reply dennis luehring <dl.soluz gmx.net> writes:
 It is possible and it works :-)
 But with operator overloading it is so much more readable, than without it.
 
 Creating queries at runtime is quite common:
 
 age = 20;
 string query = "Select name, surname FROM persons WHERE age = " + 
 to!(string)(age);

that is not object + operator based - your flexibilty here comes from your string and the "runtime" interpretation of the sql semantic in your database ... if you need an additional "LIKE" based on an codition you need to rebuild your object/operator concatination in complete: if( gui.checkbox == true ) { SELECT a from b } else { SELECT a from b WHERE c LIKE '% ~ C ~ %' }
 Compile time checking of SQLs will not be enough for any bigger program.
 How would you insert your data into database? INSERT needs values which 
 are taken e.g. from GUI and completely unknown at compile-time.

why is a value that comes from an gui not compile-time? not the value is what you need - its the type (when we talk about type safetyness) how does vector-array get into Don's Blade strings? and why do i get compile-time errors when using the wrong type inside of the string? and you can't even speak of type-safetyness when using only strings in your database communication - how can be your results typesafe (i don't like string[string] results-set) because the safetiness is programatical
 BTW. In recent release Boost team added a whole brand new library for 
 domain specific languages embedded languages called Proto. I think that 
 it means that there is quite a big need for such features. E.g. for 
 mocking frameworks which will also profit from extended DSL sublanguages 
 support in programming languages.

you should read into the details of using mixins - they are much much more flexible than you think (not just boring strings) - and with the base feature-set of D something like mock wouldn't even exist and what the boost guys to is to get around c++ base feature-set to express new thing with c++ Ary does syntax coloring and smart auto-completen help while writing spirit based parsers? the complete meaning of ( * - \ ... is different to "normal" code btw: don syntaxtree is also an great example of using mixins http://www.dsource.org/projects/mathextra/browser/trunk/blade/SyntaxTree.d
Dec 23 2008
next sibling parent reply Ary Borenszweig <ary esperanto.org.ar> writes:
dennis luehring escribió:
 It is possible and it works :-)
 But with operator overloading it is so much more readable, than 
 without it.

 Creating queries at runtime is quite common:

 age = 20;
 string query = "Select name, surname FROM persons WHERE age = " + 
 to!(string)(age);

that is not object + operator based - your flexibilty here comes from your string and the "runtime" interpretation of the sql semantic in your database ... if you need an additional "LIKE" based on an codition you need to rebuild your object/operator concatination in complete: if( gui.checkbox == true ) { SELECT a from b } else { SELECT a from b WHERE c LIKE '% ~ C ~ %' }
 Compile time checking of SQLs will not be enough for any bigger program.
 How would you insert your data into database? INSERT needs values 
 which are taken e.g. from GUI and completely unknown at compile-time.

why is a value that comes from an gui not compile-time? not the value is what you need - its the type (when we talk about type safetyness) how does vector-array get into Don's Blade strings? and why do i get compile-time errors when using the wrong type inside of the string? and you can't even speak of type-safetyness when using only strings in your database communication - how can be your results typesafe (i don't like string[string] results-set) because the safetiness is programatical
 BTW. In recent release Boost team added a whole brand new library for 
 domain specific languages embedded languages called Proto. I think 
 that it means that there is quite a big need for such features. E.g. 
 for mocking frameworks which will also profit from extended DSL 
 sublanguages support in programming languages.

you should read into the details of using mixins - they are much much more flexible than you think (not just boring strings) - and with the base feature-set of D something like mock wouldn't even exist and what the boost guys to is to get around c++ base feature-set to express new thing with c++ Ary does syntax coloring and smart auto-completen help while writing spirit based parsers? the complete meaning of ( * - \ ... is different to "normal" code

I was thinking about LINQ, specifically. Ok, LINQ is too much (and too specific), I wouldn't like something like that in D, but something similar...
 
 btw: don syntaxtree is also an great example of using mixins
 http://www.dsource.org/projects/mathextra/browser/trunk/blade/SyntaxTree.d

Dec 24 2008
parent dennis luehring <dl.soluz gmx.net> writes:
 I was thinking about LINQ, specifically. Ok, LINQ is too much (and too 
 specific), I wouldn't like something like that in D, but something 
 similar...

my point is that operator overloading helps only i a very few cases - and even then (see syntaxhighliing in boost::spirit based parsers) it does not realy help but it think (again, except the compiler bugs) D is be able to do something like LINQ (as an example) with mixins (or ast-macros) without the need of extending the language itselfe - only the parsing on compiletime thing needs to be easier - but any solution(s) in this part of D will also help all other generic library developers
Dec 24 2008
prev sibling parent aarti_pl <aarti interia.pl> writes:
dennis luehring pisze:
 It is possible and it works :-)
 But with operator overloading it is so much more readable, than 
 without it.

 Creating queries at runtime is quite common:

 age = 20;
 string query = "Select name, surname FROM persons WHERE age = " + 
 to!(string)(age);

that is not object + operator based - your flexibilty here comes from your string and the "runtime" interpretation of the sql semantic in your database ... if you need an additional "LIKE" based on an codition you need to rebuild your object/operator concatination in complete: if( gui.checkbox == true ) { SELECT a from b } else { SELECT a from b WHERE c LIKE '% ~ C ~ %' }
 Compile time checking of SQLs will not be enough for any bigger program.
 How would you insert your data into database? INSERT needs values 
 which are taken e.g. from GUI and completely unknown at compile-time.

why is a value that comes from an gui not compile-time? not the value is what you need - its the type (when we talk about type safetyness) how does vector-array get into Don's Blade strings? and why do i get compile-time errors when using the wrong type inside of the string? and you can't even speak of type-safetyness when using only strings in your database communication - how can be your results typesafe (i don't like string[string] results-set) because the safetiness is programatical
 BTW. In recent release Boost team added a whole brand new library for 
 domain specific languages embedded languages called Proto. I think 
 that it means that there is quite a big need for such features. E.g. 
 for mocking frameworks which will also profit from extended DSL 
 sublanguages support in programming languages.

you should read into the details of using mixins - they are much much more flexible than you think (not just boring strings) - and with the base feature-set of D something like mock wouldn't even exist and what the boost guys to is to get around c++ base feature-set to express new thing with c++ Ary does syntax coloring and smart auto-completen help while writing spirit based parsers? the complete meaning of ( * - \ ... is different to "normal" code btw: don syntaxtree is also an great example of using mixins http://www.dsource.org/projects/mathextra/browser/trunk/blade/SyntaxTree.d

While I see your points here, there are still arguments against mixins. Please see my answer to Don's post. BR Marcin (aarti_pl)
Dec 24 2008
prev sibling next sibling parent reply dennis luehring <dl.soluz gmx.net> writes:
 Autocompletion and syntax coloring.

ok one reason - but is the sql syntax not crippled enough using operators - better if colored/autocompleted? what does coloring help here select x, y = (select u from b where ... ), from select( xyz from abc ) left join in the special case of inner selects, from selects, left/right/inner joins... the ultimate++ solution just works for very very simple statements the readability is flying away i personaly whould throw away the idea of getting functional style programming like sql into an imperative language using operator overloading or better -> i would not ask for more operator overloading features based on this need :-) what about other domain-specific languages like the blade stuff or an ebnf parser (dparser?) - should these be more operator overloading dependend for better coloring and auto-completion? i think operator overloding is not the answer here nothing agains autocompletion and syntax coloring - very much needed decents smart autocompletion shows how it should be
Dec 23 2008
next sibling parent reply Uno <unodgs tlen.pl> writes:
 the ultimate++ solution just works for very very simple statements

Select(PRODUCTS[ID], CUSTOMER[NAME]) .From(PRODUCTS) .InnerJoin(CUSTOMERS) .On(CUSTOMERS[PRODUCT_ID] == PRODUCTS[ID] && Like(PRODUCTS[NAME], "BOOK%")) .Where(PRODUCTS[ID] > 10); You can have another statement in where or even in select..
Dec 24 2008
parent reply dennis luehring <dl.soluz gmx.net> writes:
Uno schrieb:
 the ultimate++ solution just works for very very simple statements

Select(PRODUCTS[ID], CUSTOMER[NAME]) .From(PRODUCTS) .InnerJoin(CUSTOMERS) .On(CUSTOMERS[PRODUCT_ID] == PRODUCTS[ID] && Like(PRODUCTS[NAME], "BOOK%")) .Where(PRODUCTS[ID] > 10); You can have another statement in where or even in select..

and "from select" btw: i don't want to say: "its not possible" - i want to say: "it is even clear to handle in D" without going the no-other-way-in-c++ solution
Dec 24 2008
parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
dennis luehring Wrote:

 Uno schrieb:
 the ultimate++ solution just works for very very simple statements

Select(PRODUCTS[ID], CUSTOMER[NAME]) .From(PRODUCTS) .InnerJoin(CUSTOMERS) .On(CUSTOMERS[PRODUCT_ID] == PRODUCTS[ID] && Like(PRODUCTS[NAME], "BOOK%")) .Where(PRODUCTS[ID] > 10); You can have another statement in where or even in select..

and "from select" btw: i don't want to say: "its not possible" - i want to say: "it is even clear to handle in D" without going the no-other-way-in-c++ solution

I suppose you could do it that way, at least it would definately be readable rather than what i've seen in java (and ASP) before. query = "Select * from persons where firstname='"+fname+" and lastname='"+lname+" and age="+age+... ect.ect Recently i've started reading about lex and yacc (or flex and bison) and as a comment in there, it wouldn't be too hard to use substitution and have a pre-processor convert it into pure D code. SQLQuery(" Select * from persons where firstname=%fname% and lastname=%lname% and age=%age%") Just a thought.
Dec 24 2008
parent dennis luehring <dl.soluz gmx.net> writes:
Era Scarecrow schrieb:
  Recently i've started reading about lex and yacc (or flex and bison) and as a
comment in there, it wouldn't be too hard to use substitution and have a
pre-processor convert it into pure D code.

and the name of that pre-processor is "mixin" in D, you can parse and generate code in here :-)
Dec 25 2008
prev sibling parent reply BCS <ao pathlink.com> writes:
Reply to dennis,

 what about other domain-specific languages like the blade stuff
 or an ebnf parser (dparser?)

dparse http://www.dsource.org/projects/scrapple/browser/trunk/dparser/dparse.d
 - should these be more operator overloading dependend

Avoiding operator overloading was one of the objectives of dpares. My opinion of DSL's is that they should either use a syntax designed specifically for them or use a syntax that is designed to be much more general than for a general purpose language. But then I'm biased as this is getting into my day job.
Dec 24 2008
parent dennis luehring <dl.soluz gmx.net> writes:
BCS schrieb:

My opinion of DSL's is that they should either
use a syntax designed specifically for
them or use a syntax that is designed to be much more
general than for a general purpose language.

perfect answer and my opinion too
Dec 25 2008
prev sibling parent reply Ary Borenszweig <ary esperanto.org.ar> writes:
Ary Borenszweig escribió:
 dennis luehring escribió:
 ...
 This translation is quite awful and unreadable. It would be so much 
 better to get:

 Query query = Select(a).Where(id == 5);

what speaks against an sql parsing mixin? would be more expressive, compiletime based and typesafe and even far more be able than what you can do with operator overloading

Autocompletion and syntax coloring.

And also, you get a nice compile error message if the query is malformed. With a mixin you might end up with a stack of hard-to-interpret error messages. Finally, parsing strings is not easy, specially at compile time. And there's also the problem of lack of garbage collection at compile time...
Dec 23 2008
parent dennis luehring <dl.soluz gmx.net> writes:
Ary Borenszweig schrieb:
  > Finally, parsing strings is not easy, specially at compile time. And
 there's also the problem of lack of garbage collection at compile time...

ok thats 140% true but i think that using mixins to embedd domain-spec-lang into D is currently the best way doing it (except the compiler bugs around) the only other thing on my radar is the "ast-macro" feature walter,andrei,.... are talking about
Dec 23 2008
prev sibling next sibling parent aarti_pl <aarti interia.pl> writes:
dennis luehring pisze:
 ...
 This translation is quite awful and unreadable. It would be so much 
 better to get:

 Query query = Select(a).Where(id == 5);

what speaks against an sql parsing mixin? would be more expressive, compiletime based and typesafe and even far more be able than what you can do with operator overloading

Please see additional points, which I put into answer for Ary. BR Marcin Kuszczak (aarti_pl)
Dec 23 2008
prev sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Wed, 24 Dec 2008 00:01:20 +0300, Ary Borenszweig <ary esperanto.org.ar>
wrote:

 dennis luehring escribió:
 ...
 This translation is quite awful and unreadable. It would be so much  
 better to get:

 Query query = Select(a).Where(id == 5);

would be more expressive, compiletime based and typesafe and even far more be able than what you can do with operator overloading

Autocompletion and syntax coloring.

And some compile-time checks (so that you don't mistype your tables/columns): enum Artists { Id, Name, } enum Albums { Id, Name, ArtistId, } enum Songs { Id, Name, AlbumId, } auto request = Select(Songs.Id, Songs.Name). From!(Artists). InnerJoin!(Albums).OnEqual(Albums.ArtistId, Artists.Id). InnerJoin!(Songs).OnEqual(Songs.AlbumId, Albums.Id). Where(Artists.Name).EqualsTo("Beatles"). And(Albums.Name).EqualsTo("Yellow Submarine"). OrderBy(Songs.Name). Limit(0, 1); Just an example :)
Dec 23 2008
prev sibling next sibling parent reply Don <nospam nospam.com> writes:
aarti_pl wrote:
 Andrei Alexandrescu pisze:
  > We're trying to make that work. D is due for an operator overhaul.
  >
  > Andrei
 
 Is there any chance that we get possibility to overload "raw operators", 
 like in C++? I think that they may coexist with currently defined 
 operator overloads with simple semantic rules, which will not allow them 
 to work together at the same time.
 
 I know that they may obfuscate code, but they are extremely useful when 
 defining in D domain specific languages. For example I try to make SQL 
 expressions work as D expressions:

 
 string query = "SELECT * FROM a WHERE id=5;";
 -->
 Query query = Select(a).Where(Equals(id, 5));
 
 * "id" is D object in second expression
 
 This translation is quite awful and unreadable. It would be so much 
 better to get:
 
 Query query = Select(a).Where(id == 5);

I think that's still awful. I prefer: Query query = mixin(SQL("SELECT * FROM a WHERE id=5"));
 
 With implicit casts there will be perfect environment, to define DSL-s 
 *in* D and *not* using unsafe strings.
 
 BR
 Marcin Kuszczak
 (aarti_pl)

Dec 24 2008
parent reply aarti_pl <aarti interia.pl> writes:
Don pisze:
 aarti_pl wrote:
 Andrei Alexandrescu pisze:
  > We're trying to make that work. D is due for an operator overhaul.
  >
  > Andrei

 Is there any chance that we get possibility to overload "raw 
 operators", like in C++? I think that they may coexist with currently 
 defined operator overloads with simple semantic rules, which will not 
 allow them to work together at the same time.

 I know that they may obfuscate code, but they are extremely useful 
 when defining in D domain specific languages. For example I try to 
 make SQL expressions work as D expressions:

 string query = "SELECT * FROM a WHERE id=5;";
 -->
 Query query = Select(a).Where(Equals(id, 5));

 * "id" is D object in second expression

 This translation is quite awful and unreadable. It would be so much 
 better to get:

 Query query = Select(a).Where(id == 5);

I think that's still awful. I prefer: Query query = mixin(SQL("SELECT * FROM a WHERE id=5"));

Thanks for example - it talks to me more than 1000 words :-) Your proposal is indeed interesting. (And I think it is what Dennis Luehring was reffering to). But I have to say it has some drawbacks. Let me enumerate them: 1. Extending such a queries on runtime would be rather difficult IMHO, and will not look very nice. With my solution you can do something like this: --- Column cols = [person.name, person.surname, person.nick]; for(i=0; i<rows; i++) { InsertStatement insert = Insert(); foreach(Column col; cols) { insert ~= insert.insert(col, getText(i, col)); } db.execute(insert); } * example not tested, but something like this is already possible in my framework. Constructing queries dynamically is very useful for creating e.g. universal gui controls which could be just binded to database tables. Another useful example is sending SQL expression (not the whole statement) as an argument to function. To make it work nicely with mixins IMHO you will have to make this kind of SQL to look much uglier than in this simple case. Do you have any proposals how it could look like with mixins? 2. I see also problem with IDE-s which will have to be changed to understand DLSs in mixins. I think that it is doable and even can be done quite nice, but currently there is *no* support for such a feature. It means that you will not have: a. syntax highlighting b. no IDE proposing your names of variables and methods c. no refactoring Currently in Eclipse I can rename database column names in my program with only single mouse click (Java version of my framework, which is the main branch currently). When SQLs will be in form of string mixins they will not change during refactoring. You will have to write special refactoring tools for your library to make it done. 3. Sometimes having more expressiveness in language can avoid creating new DSLs. I think it is a Good Thing (tm). In fact I took that way and decided to integrate SQL DSL language into mother language. That way programmers don't have to learn new syntax, but just use existing syntax of mother language to solve specific problems. Using DSL in mother language resolves then into just writing special kind of API, which can be used in standard way to achieve goal. D and most other languages from C family are almost perfect candidates to make something like this with SQL. So while I can agree that original SQL looks much better than any kind of its imitation, I think that it is not necessary better for productivity. Additionally I think that string mixins need some more features to make it possible to solve point 2. IDE will have to know what kind of DSL is in mixin. Maybe something like this: mixin(sql.query, "SELECT ...."); mixin(sql.expression, "person.doctorid = doctor.id"); mixin(sql.statement, "INSERT ...."); ...where sql.query, sql.expression, sql.statement are kind of markers usefull for IDE. BR Marcin Kuszczak (aarti_pl)
Dec 24 2008
parent Don <nospam nospam.com> writes:
aarti_pl wrote:
 Don pisze:
 aarti_pl wrote:
 Andrei Alexandrescu pisze:
  > We're trying to make that work. D is due for an operator overhaul.
  >
  > Andrei

 Is there any chance that we get possibility to overload "raw 
 operators", like in C++? I think that they may coexist with currently 
 defined operator overloads with simple semantic rules, which will not 
 allow them to work together at the same time.

 I know that they may obfuscate code, but they are extremely useful 
 when defining in D domain specific languages. For example I try to 
 make SQL expressions work as D expressions:

 string query = "SELECT * FROM a WHERE id=5;";
 -->
 Query query = Select(a).Where(Equals(id, 5));

 * "id" is D object in second expression

 This translation is quite awful and unreadable. It would be so much 
 better to get:

 Query query = Select(a).Where(id == 5);

I think that's still awful. I prefer: Query query = mixin(SQL("SELECT * FROM a WHERE id=5"));

Thanks for example - it talks to me more than 1000 words :-) Your proposal is indeed interesting. (And I think it is what Dennis Luehring was reffering to). But I have to say it has some drawbacks. Let me enumerate them: 1. Extending such a queries on runtime would be rather difficult IMHO, and will not look very nice. With my solution you can do something like this: --- Column cols = [person.name, person.surname, person.nick]; for(i=0; i<rows; i++) { InsertStatement insert = Insert(); foreach(Column col; cols) { insert ~= insert.insert(col, getText(i, col)); } db.execute(insert); } * example not tested, but something like this is already possible in my framework. Constructing queries dynamically is very useful for creating e.g. universal gui controls which could be just binded to database tables. Another useful example is sending SQL expression (not the whole statement) as an argument to function. To make it work nicely with mixins IMHO you will have to make this kind of SQL to look much uglier than in this simple case. Do you have any proposals how it could look like with mixins?

There's nothing stopping you from using local variables in the mixin string. A simple possibility is to use $ for embedded variables. (You can avoid that, but it's much more difficult, and has some minor downsides, you can't have use a local variable called 'select', for example). I pretty much got this working for BLADE, but the CTFE memory bug makes it completely unusable at present.
 2. I see also problem with IDE-s which will have to be changed to 
 understand DLSs in mixins. I think that it is doable and even can be 
 done quite nice, but currently there is *no* support for such a feature. 
 It means that you will not have:
 
 a. syntax highlighting
 b. no IDE proposing your names of variables and methods
 c. no refactoring

That's true, and indeed very difficult to solve.
 3. Sometimes having more expressiveness in language can avoid creating 
 new DSLs. I think it is a Good Thing (tm). In fact I took that way and 
 decided to integrate SQL DSL language into mother language. That way 
 programmers don't have to learn new syntax, but just use existing syntax 
 of mother language to solve specific problems. Using DSL in mother 
 language resolves then into just writing special kind of API, which can 
 be used in standard way to achieve goal. D and most other languages from 
 C family are almost perfect candidates to make something like this with 
 SQL.
 
 So while I can agree that original SQL looks much better than any kind 
 of its imitation, I think that it is not necessary better for productivity.

That's a worthwhile criticism of DSLs in general.
Dec 26 2008
prev sibling parent reply The Anh Tran <trtheanh gmail.com> writes:
aarti_pl wrote:
 Andrei Alexandrescu pisze:
  > We're trying to make that work. D is due for an operator overhaul.
  >
  > Andrei
 
 Is there any chance that we get possibility to overload "raw operators", 
 like in C++? I think that they may coexist with currently defined 
 operator overloads with simple semantic rules, which will not allow them 
 to work together at the same time.
 ..........
 BR
 Marcin Kuszczak
 (aarti_pl)

Me also have a dream :D <Daydream mode> class Foo { auto op(++)(); // bar++ auto op(++)(int); // ++bar op(cast)(uint); // cast(uint)bar // opCast auto op(())(int, float); // Foo(123, 123.456) // opCall auto op(+)(Foo rhs); // bar1 + bar2 auto op(+=)(int); // bar += 1234; auto op(.)(); // bar.xyz // opDot Foo op([][][])(int, char, float); // bar[123]['x'][123.456] auto op([..])(); // i = bar2[] // opSlide auto op([..])(int, int); // bar[1..10] auto op([..]=)(float); // bar[] = 12.3 //opSlideAssign auto op([..]=)(int, int, float); // bar[1..3] = 123.4 } </Dream>
Dec 26 2008
next sibling parent reply "Bill Baxter" <wbaxter gmail.com> writes:
On Sat, Dec 27, 2008 at 9:42 AM, The Anh Tran <trtheanh gmail.com> wrote:
 aarti_pl wrote:
 Andrei Alexandrescu pisze:
  > We're trying to make that work. D is due for an operator overhaul.
  >
  > Andrei

 Is there any chance that we get possibility to overload "raw operators",
 like in C++? I think that they may coexist with currently defined operator
 overloads with simple semantic rules, which will not allow them to work
 together at the same time.
 ..........
 BR
 Marcin Kuszczak
 (aarti_pl)

Me also have a dream :D <Daydream mode> class Foo { auto op(++)(); // bar++ auto op(++)(int); // ++bar op(cast)(uint); // cast(uint)bar // opCast auto op(())(int, float); // Foo(123, 123.456) // opCall auto op(+)(Foo rhs); // bar1 + bar2 auto op(+=)(int); // bar += 1234; auto op(.)(); // bar.xyz // opDot Foo op([][][])(int, char, float); // bar[123]['x'][123.456] auto op([..])(); // i = bar2[] // opSlide auto op([..])(int, int); // bar[1..10] auto op([..]=)(float); // bar[] = 12.3 //opSlideAssign auto op([..]=)(int, int, float); // bar[1..3] = 123.4 } </Dream>

When I suggested this kind of thing long ago, Walter said that it encourages operator overload abuse, because it suggests that + is just a generic symbolic operator rather than something that specifically means "addition". That's why D uses "opAdd" instead. It's supposed to encourage only creating overloads that follow the original meaning of the operator closely. That way when you see a+b you can be reasonably sure that it means addition or something quite like it. It also goes hand-in-hand with design decisions like defining ++x to be x+=1, which means that in D it's impossible to make ++x mean something distinct from incrementing by 1. With C++ you can make ++x have whatever meaning you want. It can do something completely different from x+=1. The idea is that such freedom just makes code harder to read. Currently I don't think it makes much difference either way. The only real advantage I see to the alternate syntax is that it can be perhaps a little easier to remember. But I also I don't think Walter's idea about naming following usage does anything to stop someone like Downs from using opDiv to do something completely different from division. It probably never even occurred to Downs that Walter was trying to prevent him from abusing opDiv by naming it opDiv instead of operator(/). The people who are likely to abuse operators are precisely those who aren't likely to be daunted by mere naming. But anyway, just so you know, that's why D does things the way it does. So that means to see your dream come true you first have to convince Walter that it's a dream worth having. :-) --bb
Dec 26 2008
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Bill Baxter wrote:
 On Sat, Dec 27, 2008 at 9:42 AM, The Anh Tran <trtheanh gmail.com> wrote:
 aarti_pl wrote:
 Andrei Alexandrescu pisze:
  > We're trying to make that work. D is due for an operator overhaul.
  >
  > Andrei

 Is there any chance that we get possibility to overload "raw operators",
 like in C++? I think that they may coexist with currently defined operator
 overloads with simple semantic rules, which will not allow them to work
 together at the same time.
 ..........
 BR
 Marcin Kuszczak
 (aarti_pl)

<Daydream mode> class Foo { auto op(++)(); // bar++ auto op(++)(int); // ++bar op(cast)(uint); // cast(uint)bar // opCast auto op(())(int, float); // Foo(123, 123.456) // opCall auto op(+)(Foo rhs); // bar1 + bar2 auto op(+=)(int); // bar += 1234; auto op(.)(); // bar.xyz // opDot Foo op([][][])(int, char, float); // bar[123]['x'][123.456] auto op([..])(); // i = bar2[] // opSlide auto op([..])(int, int); // bar[1..10] auto op([..]=)(float); // bar[] = 12.3 //opSlideAssign auto op([..]=)(int, int, float); // bar[1..3] = 123.4 } </Dream>

When I suggested this kind of thing long ago, Walter said that it encourages operator overload abuse, because it suggests that + is just a generic symbolic operator rather than something that specifically means "addition". That's why D uses "opAdd" instead. It's supposed to encourage only creating overloads that follow the original meaning of the operator closely. That way when you see a+b you can be reasonably sure that it means addition or something quite like it.

I think that argument is rather weak and ought to be revisited. It's weak to start with as if writing "+" in a D program hardly evokes anything else but "plus". What the notation effectively achieved was put more burden on the programmer to memorize some names for the already-known symbols. I think the entire operator overloading business, which started from a legitimate desire to improve on C++'s, ended up worse off.
 It also goes hand-in-hand with design decisions like
 defining ++x to be x+=1, which means that in D it's impossible to make
 ++x mean something distinct from incrementing by 1.

This is the vomit in the fat lady's cleavage that shows just how bad the wine turned out to be. For iterators, increment is quite different from addition of an arbitrary number, so what D managed to do was effectively to cripple iterators. The standard library will use ranges with named functions so it avoids the issue, but if someone wants to define STL-style iterators they won't be able to.
 With C++ you can
 make ++x have whatever meaning you want.  It can do something
 completely different from x+=1.  The idea is that such freedom just
 makes code harder to read.

That sounds nice, but on the face of it I haven't heard of much code suffering from the problem.
 Currently I don't think it makes much difference either way.  The only
 real advantage I see to the alternate syntax is that it can be perhaps
 a little easier to remember.  But I also I don't think Walter's idea
 about naming following usage does anything to stop someone like Downs
 from using opDiv to do something completely different from division.
 It probably never even occurred to Downs that Walter was trying to
 prevent him from abusing opDiv by naming it opDiv instead of
 operator(/).  The people who are likely to abuse operators are
 precisely those who aren't likely to be daunted by mere naming.
 
 But anyway, just so you know, that's why D does things the way it
 does.  So that means to see your dream come true you first have to
 convince Walter that it's a dream worth having. :-)

The story is quite lame, so it may be worth scrutinizing it. Let's make D operators not suck. Wanna? Andrei
Dec 26 2008
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Jarrett Billingsley wrote:
 On Fri, Dec 26, 2008 at 11:20 PM, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:
 I think that argument is rather weak and ought to be revisited. It's weak to
 start with as if writing "+" in a D program hardly evokes anything else but
 "plus". What the notation effectively achieved was put more burden on the
 programmer to memorize some names for the already-known symbols. I think the
 entire operator overloading business, which started from a legitimate desire
 to improve on C++'s, ended up worse off.

It does, however, introduce a nice naming scheme for naming "meta-methods", that is, methods which are called indirectly to overload certain language structures. opApply is an example of such a method which doesn't truly have a corresponding operator, and I'm sure there could be other other methods for other language constructs, existing or theoretical. (though I'm sure you'll agree opApply's behavior could use a rehaul too)
 This is the vomit in the fat lady's cleavage that shows just how bad the
 wine turned out to be.

That has to be one of the most disgusting metaphors I've ever heard ;)
 For iterators, increment is quite different from
 addition of an arbitrary number, so what D managed to do was effectively to
 cripple iterators. The standard library will use ranges with named functions
 so it avoids the issue, but if someone wants to define STL-style iterators
 they won't be able to.

I suppose most people who _aren't_ coming from C++ (*cough* like me *cough*) won't be terribly unhappy about this situation.

I'm not sure how that computes. The particular notion has little to do with C++ and is rather fundamental, so not grokking it should motivate one to look into it (as opposed to being glad for not knowing). Andrei
Dec 26 2008
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Jarrett Billingsley wrote:
 On Sat, Dec 27, 2008 at 2:31 AM, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:
 For iterators, increment is quite different from
 addition of an arbitrary number, so what D managed to do was effectively
 to
 cripple iterators. The standard library will use ranges with named
 functions
 so it avoids the issue, but if someone wants to define STL-style
 iterators
 they won't be able to.

*cough*) won't be terribly unhappy about this situation.

C++ and is rather fundamental, so not grokking it should motivate one to look into it (as opposed to being glad for not knowing).

I got the impression from your argument that you thought that without an appropriate method of overloading the increment operator, it's not possible to duplicate the _syntax_ of STL-style iterators. My response is - who cares? I don't think of iterators as "pointers that can be incremented," and I doubt anyone who doesn't have a C++ background thinks of them that way either. It seems that the semantics of STL iterators can be implemented with methods just as well. If that's not what you were arguing - that is, if you're arguing that C++'s operator overloading is somehow more expressive and allows you to implement something that can't be implemented with method calls - then I will put my foot in my mouth ;)

Well I overstated my point, sorry. The nice thing about overloading ++ is that you can use the same algorithms with built-in types and user-defined types. You are, however, right that defining various functions such as increment and decrement for built-ins and also user-defined types would obviate the need for syntactically consistent operators. Andrei
Dec 27 2008
prev sibling parent reply Don <nospam nospam.com> writes:
Andrei Alexandrescu wrote:
 Bill Baxter wrote:
 On Sat, Dec 27, 2008 at 9:42 AM, The Anh Tran <trtheanh gmail.com> wrote:
 aarti_pl wrote:
 Andrei Alexandrescu pisze:
  > We're trying to make that work. D is due for an operator overhaul.
  >
  > Andrei

 Is there any chance that we get possibility to overload "raw 
 operators",
 like in C++? I think that they may coexist with currently defined 
 operator
 overloads with simple semantic rules, which will not allow them to work
 together at the same time.
 ..........
 BR
 Marcin Kuszczak
 (aarti_pl)

<Daydream mode> class Foo { auto op(++)(); // bar++ auto op(++)(int); // ++bar op(cast)(uint); // cast(uint)bar // opCast auto op(())(int, float); // Foo(123, 123.456) // opCall auto op(+)(Foo rhs); // bar1 + bar2 auto op(+=)(int); // bar += 1234; auto op(.)(); // bar.xyz // opDot Foo op([][][])(int, char, float); // bar[123]['x'][123.456] auto op([..])(); // i = bar2[] // opSlide auto op([..])(int, int); // bar[1..10] auto op([..]=)(float); // bar[] = 12.3 //opSlideAssign auto op([..]=)(int, int, float); // bar[1..3] = 123.4 } </Dream>

When I suggested this kind of thing long ago, Walter said that it encourages operator overload abuse, because it suggests that + is just a generic symbolic operator rather than something that specifically means "addition". That's why D uses "opAdd" instead. It's supposed to encourage only creating overloads that follow the original meaning of the operator closely. That way when you see a+b you can be reasonably sure that it means addition or something quite like it.

I think that argument is rather weak and ought to be revisited. It's weak to start with as if writing "+" in a D program hardly evokes anything else but "plus". What the notation effectively achieved was put more burden on the programmer to memorize some names for the already-known symbols. I think the entire operator overloading business, which started from a legitimate desire to improve on C++'s, ended up worse off.

I feel quite strongly that C++'s operator overloading was a failed experiment. The original intention (AFAIK) was to allow creation of mathematical entities which could use natural syntax. The classic example was complex numbers, and it works reasonably well for that, although it requires you to create an absurd number of repetitive functions. But for anything much more complicated, such as matrices, tensors, big integer arithmetic, etc -- it's an abject failure. It's clumsy, and creates masses of temporary objects, which kills performance so completely that it's unusable. But the whole point of operator overloading was to allow nice notation in a performace-oriented language! Expression templates are basically a hack to restore performance in most cases, but it comes at a massive cost in simplicity. And the performance even then is not always optimal. I think that Walter's idea, in tightening the semantics of overloaded operators, is the right approach. Unfortunately, it doesn't go far enough, so we get the worst of both worlds: the C++ freedom is curtailed, but there isn't enough power to replace it. Ultimately, I think that the problem is that ideally, '+' is not simply a call to a function called 'plus()'. What you'd like an operator to compile to, depends on the expression in which it is embedded. For maximum performance, an expression needs to be digested before it is converted into elementary functions. In my 'operator overloading without temporaries' proposal in Bugzilla, I showed that DEFINING a -= b as being identical to a = a - b, and then creating a symmetric operation for a = b - a allows optimal code generation in a great many cases. It's not a complete solution, though. In particular, irreducible temporaries need more thought. Ideally, in something like a += b * c + d, b*c would be created in a memory pool, and deleted at the end of the expression. (By contrast, a = b*c+d, would translate to a=b*c; a+=d; so no temporary is required). There are other, less serious problems which also need to be addressed. Defining ++a as a+=1 is probably a mistake. It raises lots of nasty issues. * If a is a complex number, a = a + 1 makes perfect sense. But it's not obvious that ++a is sensible. * What type is '1'? Is it an int, a uint, a long, ... You don't have that issue with increment. As I see it, there are two possible strategies: (1) Pursuing optimal performance, which requires semantic tightening, and reduced flexibility, or (2) Pursure simplicity and semantic flexibility, sacrificing performance. I think those two possibilities are mutually exclusive.
Dec 27 2008
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Don wrote:
 Andrei Alexandrescu wrote:
 Bill Baxter wrote:
 On Sat, Dec 27, 2008 at 9:42 AM, The Anh Tran <trtheanh gmail.com> 
 wrote:
 aarti_pl wrote:
 Andrei Alexandrescu pisze:
  > We're trying to make that work. D is due for an operator overhaul.
  >
  > Andrei

 Is there any chance that we get possibility to overload "raw 
 operators",
 like in C++? I think that they may coexist with currently defined 
 operator
 overloads with simple semantic rules, which will not allow them to 
 work
 together at the same time.
 ..........
 BR
 Marcin Kuszczak
 (aarti_pl)

<Daydream mode> class Foo { auto op(++)(); // bar++ auto op(++)(int); // ++bar op(cast)(uint); // cast(uint)bar // opCast auto op(())(int, float); // Foo(123, 123.456) // opCall auto op(+)(Foo rhs); // bar1 + bar2 auto op(+=)(int); // bar += 1234; auto op(.)(); // bar.xyz // opDot Foo op([][][])(int, char, float); // bar[123]['x'][123.456] auto op([..])(); // i = bar2[] // opSlide auto op([..])(int, int); // bar[1..10] auto op([..]=)(float); // bar[] = 12.3 //opSlideAssign auto op([..]=)(int, int, float); // bar[1..3] = 123.4 } </Dream>

When I suggested this kind of thing long ago, Walter said that it encourages operator overload abuse, because it suggests that + is just a generic symbolic operator rather than something that specifically means "addition". That's why D uses "opAdd" instead. It's supposed to encourage only creating overloads that follow the original meaning of the operator closely. That way when you see a+b you can be reasonably sure that it means addition or something quite like it.

I think that argument is rather weak and ought to be revisited. It's weak to start with as if writing "+" in a D program hardly evokes anything else but "plus". What the notation effectively achieved was put more burden on the programmer to memorize some names for the already-known symbols. I think the entire operator overloading business, which started from a legitimate desire to improve on C++'s, ended up worse off.

I feel quite strongly that C++'s operator overloading was a failed experiment. The original intention (AFAIK) was to allow creation of mathematical entities which could use natural syntax. The classic example was complex numbers, and it works reasonably well for that, although it requires you to create an absurd number of repetitive functions. But for anything much more complicated, such as matrices, tensors, big integer arithmetic, etc -- it's an abject failure. It's clumsy, and creates masses of temporary objects, which kills performance so completely that it's unusable. But the whole point of operator overloading was to allow nice notation in a performace-oriented language! Expression templates are basically a hack to restore performance in most cases, but it comes at a massive cost in simplicity. And the performance even then is not always optimal. I think that Walter's idea, in tightening the semantics of overloaded operators, is the right approach. Unfortunately, it doesn't go far enough, so we get the worst of both worlds: the C++ freedom is curtailed, but there isn't enough power to replace it.

Very well put.
 Ultimately, I think that the problem is that ideally, '+' is not simply 
 a call to a function called 'plus()'. What you'd like an operator to 
 compile to, depends on the expression in which it is embedded. For 
 maximum performance, an expression needs to be digested before it is 
 converted into elementary functions.
 
 In my 'operator overloading without temporaries' proposal in Bugzilla,
 I showed that DEFINING a -= b as being identical to a = a - b, and then 
  creating a symmetric operation for a = b - a allows optimal code 
 generation in a great many cases. It's not a complete solution, though.
 
 In particular, irreducible temporaries need more thought. Ideally, in 
 something like a += b * c + d, b*c would be created in a memory pool, 
 and deleted at the end of the expression.
 (By contrast, a = b*c+d, would translate to a=b*c; a+=d; so no temporary 
 is required).

That's an awesome proposal. I'd like to expand it to comprehend fusion as well. Consider: A = B + C - D; where the operands are matrices. The best hand-written implementation would loop once through the three matrices and assign to the destination element-wise A[i, j] = B[i, j] + C[i, j] - D[i, j]. However, with an approach that has only one operator application as its horizon, it is impossible to achieve that optimization. So I wonder what abstraction could be devised that makes it easy and natural to support such fusion. Expression templates achieve that by saving the right-hand expression tree as a type and then using it during the assignment. This requires a considerable effort and has some drawbacks.
 There are other, less serious problems which also need to be addressed.
 
 Defining ++a as a+=1 is probably a mistake. It raises lots of nasty issues.
 * If a is a complex number, a = a + 1 makes perfect sense. But it's not 
 obvious that ++a is sensible.
 * What type is '1'? Is it an int, a uint, a long, ... You don't have 
 that issue with increment.

Great points!
 As I see it, there are two possible strategies:
 (1) Pursuing optimal performance, which requires semantic tightening, 
 and reduced flexibility, or
 (2) Pursure simplicity and semantic flexibility, sacrificing performance.
 
 I think those two possibilities are mutually exclusive.

I tend to be more optimistic, but if asked to choose, I'd go for (1). One important lesson learned from C++'s operator overloading is that freedom was almost always badly used. Tellingly, whenever operator overloading is taught or talked about, the first caveat mentioned is that defining inconsistent batteries of operators is exceedingly easy. Andrei
Dec 27 2008
prev sibling parent reply aarti_pl <aarti interia.pl> writes:
Don pisze:
 As I see it, there are two possible strategies:
 (1) Pursuing optimal performance, which requires semantic tightening, 
 and reduced flexibility, or
 (2) Pursure simplicity and semantic flexibility, sacrificing performance.
 
 I think those two possibilities are mutually exclusive.

Well, I think you have just discovered two levels of programmers interests in operator overloading :-) * First level (group) is for programmers which want to use operators to create fast mathematical operations for user defined types. * Second level (group) is for programmers which want to use operators for creating better syntax, to make their users more happy. It seems that both groups have good reasons for their requests. As you said these two possibilities are mutually exclusive, so I think also solution for this challenge should be splitted, so that everyone could get what he want. One possible solution is to allow defining operator overloading on two levels also: 1. In first level, where operators are used for computation, there might be operators like today in D: opCmp, opEquals. They can be even stricter and/or better defined as needed. 2. Second level should be just raw operator access, which allows to use operator syntax in wider scope of cases. Compiler should enforce the rule that operators from two levels can not be intermixed in one class/struct. I think that such a solution could make these two groups have their goals achieved. BR Marcin Kuszczak (aarti_pl)
Dec 27 2008
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
aarti_pl wrote:
 Don pisze:
 As I see it, there are two possible strategies:
 (1) Pursuing optimal performance, which requires semantic tightening, 
 and reduced flexibility, or
 (2) Pursure simplicity and semantic flexibility, sacrificing performance.

 I think those two possibilities are mutually exclusive.

Well, I think you have just discovered two levels of programmers interests in operator overloading :-) * First level (group) is for programmers which want to use operators to create fast mathematical operations for user defined types. * Second level (group) is for programmers which want to use operators for creating better syntax, to make their users more happy. It seems that both groups have good reasons for their requests. As you said these two possibilities are mutually exclusive, so I think also solution for this challenge should be splitted, so that everyone could get what he want. One possible solution is to allow defining operator overloading on two levels also: 1. In first level, where operators are used for computation, there might be operators like today in D: opCmp, opEquals. They can be even stricter and/or better defined as needed. 2. Second level should be just raw operator access, which allows to use operator syntax in wider scope of cases. Compiler should enforce the rule that operators from two levels can not be intermixed in one class/struct. I think that such a solution could make these two groups have their goals achieved. BR Marcin Kuszczak (aarti_pl)

That can be achieved by only defining the low-level operators in the language and then developing a library layer on top of them. Andrei
Dec 27 2008
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
The Anh Tran wrote:
 aarti_pl wrote:
 Andrei Alexandrescu pisze:
  > We're trying to make that work. D is due for an operator overhaul.
  >
  > Andrei

 Is there any chance that we get possibility to overload "raw 
 operators", like in C++? I think that they may coexist with currently 
 defined operator overloads with simple semantic rules, which will not 
 allow them to work together at the same time.
 ..........
 BR
 Marcin Kuszczak
 (aarti_pl)

Me also have a dream :D <Daydream mode> class Foo { auto op(++)(); // bar++ auto op(++)(int); // ++bar

Hey, wasn't the implementation of the postincrement operator through an overload a rather untasty hack?
     op(cast)(uint); // cast(uint)bar // opCast
     auto op(())(int, float); // Foo(123, 123.456) // opCall
     
     auto op(+)(Foo rhs); // bar1 + bar2
     auto op(+=)(int); // bar += 1234;
     auto op(.)(); // bar.xyz // opDot
 
     Foo op([][][])(int, char, float); // bar[123]['x'][123.456]
 
     auto op([..])(); // i = bar2[] // opSlide
     auto op([..])(int, int); // bar[1..10]
 
     auto op([..]=)(float); // bar[] = 12.3 //opSlideAssign
     auto op([..]=)(int, int, float); // bar[1..3] = 123.4
 }
 </Dream>

Aside for a minor change in notation, there's no improvement. We're looking for much more broad improvements, such as offering the ability to overload several operators with only one function. Andrei
Dec 26 2008
parent reply The Anh Tran <trtheanh gmail.com> writes:
Andrei Alexandrescu wrote:
     auto op(++)(); // bar++
     auto op(++)(int); // ++bar

Hey, wasn't the implementation of the postincrement operator through an overload a rather untasty hack?

 Aside for a minor change in notation, there's no improvement. We're 
 looking for much more broad improvements, such as offering the ability 
 to overload several operators with only one function.
 
 Andrei

How about this: Unary op: ++f, ~f, ~f, +f, -f, *f auto operator(++, --, ~, !, +, -, *)() { // posfix is provided by compiler // and is only used for: foo++ foo-- static if (posfix) return op(this.value); else ... } Binary op: equality comparison f1 <= f2 bool operator(<, >, <=, >=, ==, !=)(Foo foo) { return op(this.value, foo.value); } For un-order object, he/she just list 'correct' operator(s) in op() list. Ex: bool operator(!=, ==)(Foo foo) {} Binary op: logic f1 || f2 && f3 bool operator(&&, ||, &, |, &)(Op[] oops, Foo[] foo) { auto result = foo[0].value; foreach (i, op ; oops) result = op( result, foo[i+1].value ); return result; } Binary op: bitwise auto operator(&, |, ^)(Op[], Foo[]) {} Binary op: add, sub, mul, div, shift, cat(~) f1 + f2 * f3 ~ f4 auto operator(+, -, *, /, %, <<, >>, ~, =)(Op[] oops, Foo[] foo() { auto result = foo[0].value; foreach (i, op ; oops) result = op( result, foo[i+1].value ); return result; } Op[], Foo[] should/may be re-arranged according to operator precedence??? Binary op: +=, -=, *=, <<=, ~= f1 *= f2 + f3 / f4 I 'think' the compiler can evaluate to: f1 = f1 * (f2 + f3 / f4);
Dec 27 2008
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
The Anh Tran wrote:
 Andrei Alexandrescu wrote:
     auto op(++)(); // bar++
     auto op(++)(int); // ++bar

Hey, wasn't the implementation of the postincrement operator through an overload a rather untasty hack?

 Aside for a minor change in notation, there's no improvement. We're 
 looking for much more broad improvements, such as offering the ability 
 to overload several operators with only one function.

 Andrei

How about this: Unary op: ++f, ~f, ~f, +f, -f, *f auto operator(++, --, ~, !, +, -, *)() { // posfix is provided by compiler // and is only used for: foo++ foo-- static if (posfix) return op(this.value); else ... } Binary op: equality comparison f1 <= f2 bool operator(<, >, <=, >=, ==, !=)(Foo foo) { return op(this.value, foo.value); } For un-order object, he/she just list 'correct' operator(s) in op() list. Ex: bool operator(!=, ==)(Foo foo) {}

Why invent new syntax when compile-time strings are already there? auto operator(string op)() if (op == "++" || op == "--") { return mixin(op ~ "this.value"); } etc. That, of course, is orthogonal to the semantic equivalences suggested by Don and does not solve fusion. Andrei
Dec 27 2008
parent reply John Reimer <terminal.node gmail.com> writes:
Hello Andrei,

 The Anh Tran wrote:
 
 Andrei Alexandrescu wrote:
 
 auto op(++)(); // bar++
 auto op(++)(int); // ++bar

an overload a rather untasty hack? Aside for a minor change in notation, there's no improvement. We're looking for much more broad improvements, such as offering the ability to overload several operators with only one function. Andrei

Unary op: ++f, ~f, ~f, +f, -f, *f auto operator(++, --, ~, !, +, -, *)() { // posfix is provided by compiler // and is only used for: foo++ foo-- static if (posfix) return op(this.value); else ... } Binary op: equality comparison f1 <= f2 bool operator(<, >, <=, >=, ==, !=)(Foo foo) { return op(this.value, foo.value); } For un-order object, he/she just list 'correct' operator(s) in op() list. Ex: bool operator(!=, ==)(Foo foo) {}

auto operator(string op)() if (op == "++" || op == "--") { return mixin(op ~ "this.value"); } etc. That, of course, is orthogonal to the semantic equivalences suggested by Don and does not solve fusion. Andrei

Once again, I'm no expert in this matter, but the compiler-time strings idea looks like a great solution for this. Was this brought up before? ...because that was kind of what I was thinking when you mentioned a new operator overloading syntax for D. -JJR
Dec 27 2008
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
John Reimer wrote:
 Hello Andrei,
 
 The Anh Tran wrote:

 Andrei Alexandrescu wrote:

 auto op(++)(); // bar++
 auto op(++)(int); // ++bar

an overload a rather untasty hack? Aside for a minor change in notation, there's no improvement. We're looking for much more broad improvements, such as offering the ability to overload several operators with only one function. Andrei

Unary op: ++f, ~f, ~f, +f, -f, *f auto operator(++, --, ~, !, +, -, *)() { // posfix is provided by compiler // and is only used for: foo++ foo-- static if (posfix) return op(this.value); else ... } Binary op: equality comparison f1 <= f2 bool operator(<, >, <=, >=, ==, !=)(Foo foo) { return op(this.value, foo.value); } For un-order object, he/she just list 'correct' operator(s) in op() list. Ex: bool operator(!=, ==)(Foo foo) {}

auto operator(string op)() if (op == "++" || op == "--") { return mixin(op ~ "this.value"); } etc. That, of course, is orthogonal to the semantic equivalences suggested by Don and does not solve fusion. Andrei

Once again, I'm no expert in this matter, but the compiler-time strings idea looks like a great solution for this. Was this brought up before? ...because that was kind of what I was thinking when you mentioned a new operator overloading syntax for D.

Only in private conversation between Walter and me. But anyway, let's not forget we need to address fusion as well. Andrei
Dec 27 2008
prev sibling next sibling parent "Jarrett Billingsley" <jarrett.billingsley gmail.com> writes:
On Fri, Dec 26, 2008 at 11:20 PM, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:
 I think that argument is rather weak and ought to be revisited. It's weak to
 start with as if writing "+" in a D program hardly evokes anything else but
 "plus". What the notation effectively achieved was put more burden on the
 programmer to memorize some names for the already-known symbols. I think the
 entire operator overloading business, which started from a legitimate desire
 to improve on C++'s, ended up worse off.

It does, however, introduce a nice naming scheme for naming "meta-methods", that is, methods which are called indirectly to overload certain language structures. opApply is an example of such a method which doesn't truly have a corresponding operator, and I'm sure there could be other other methods for other language constructs, existing or theoretical. (though I'm sure you'll agree opApply's behavior could use a rehaul too)
 This is the vomit in the fat lady's cleavage that shows just how bad the
 wine turned out to be.

That has to be one of the most disgusting metaphors I've ever heard ;)
 For iterators, increment is quite different from
 addition of an arbitrary number, so what D managed to do was effectively to
 cripple iterators. The standard library will use ranges with named functions
 so it avoids the issue, but if someone wants to define STL-style iterators
 they won't be able to.

I suppose most people who _aren't_ coming from C++ (*cough* like me *cough*) won't be terribly unhappy about this situation.
Dec 26 2008
prev sibling next sibling parent "Jarrett Billingsley" <jarrett.billingsley gmail.com> writes:
On Sat, Dec 27, 2008 at 2:31 AM, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:
 For iterators, increment is quite different from
 addition of an arbitrary number, so what D managed to do was effectively
 to
 cripple iterators. The standard library will use ranges with named
 functions
 so it avoids the issue, but if someone wants to define STL-style
 iterators
 they won't be able to.

I suppose most people who _aren't_ coming from C++ (*cough* like me *cough*) won't be terribly unhappy about this situation.

I'm not sure how that computes. The particular notion has little to do with C++ and is rather fundamental, so not grokking it should motivate one to look into it (as opposed to being glad for not knowing).

I got the impression from your argument that you thought that without an appropriate method of overloading the increment operator, it's not possible to duplicate the _syntax_ of STL-style iterators. My response is - who cares? I don't think of iterators as "pointers that can be incremented," and I doubt anyone who doesn't have a C++ background thinks of them that way either. It seems that the semantics of STL iterators can be implemented with methods just as well. If that's not what you were arguing - that is, if you're arguing that C++'s operator overloading is somehow more expressive and allows you to implement something that can't be implemented with method calls - then I will put my foot in my mouth ;)
Dec 27 2008
prev sibling next sibling parent "Bill Baxter" <wbaxter gmail.com> writes:
On Sat, Dec 27, 2008 at 11:19 PM, Jarrett Billingsley
<jarrett.billingsley gmail.com> wrote:
 On Sat, Dec 27, 2008 at 2:31 AM, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:
 For iterators, increment is quite different from
 addition of an arbitrary number, so what D managed to do was effectively
 to
 cripple iterators. The standard library will use ranges with named
 functions
 so it avoids the issue, but if someone wants to define STL-style
 iterators
 they won't be able to.

I suppose most people who _aren't_ coming from C++ (*cough* like me *cough*) won't be terribly unhappy about this situation.

I'm not sure how that computes. The particular notion has little to do with C++ and is rather fundamental, so not grokking it should motivate one to look into it (as opposed to being glad for not knowing).

I got the impression from your argument that you thought that without an appropriate method of overloading the increment operator, it's not possible to duplicate the _syntax_ of STL-style iterators. My response is - who cares? I don't think of iterators as "pointers that can be incremented," and I doubt anyone who doesn't have a C++ background thinks of them that way either. It seems that the semantics of STL iterators can be implemented with methods just as well.

There is a little something nice about being able to use a pointer and iterator interchangeably for templated container algorithms. You lose that if ++iter can't be used to iterate your iterator. But working around it is not really that hard. One little incr(T)(ref thing) template and you're pretty much done. But ++foo is a concise syntax that conveys the meaning of what's happening pretty well, so it's a shame not to be able to use it on iterators. (Actually I don't really recall why it can't be used for iterators in D other than it forces you to use more runtime errors for bad args to +=. Was there anything more to it than that?) --bb
Dec 27 2008
prev sibling parent "Jarrett Billingsley" <jarrett.billingsley gmail.com> writes:
On Sat, Dec 27, 2008 at 11:10 AM, Bill Baxter <wbaxter gmail.com> wrote:
 I got the impression from your argument that you thought that without
 an appropriate method of overloading the increment operator, it's not
 possible to duplicate the _syntax_ of STL-style iterators.  My
 response is - who cares?  I don't think of iterators as "pointers that
 can be incremented," and I doubt anyone who doesn't have a C++
 background thinks of them that way either.  It seems that the
 semantics of STL iterators can be implemented with methods just as
 well.

There is a little something nice about being able to use a pointer and iterator interchangeably for templated container algorithms. You lose that if ++iter can't be used to iterate your iterator.

Except that in D, you rarely use pointers, and ++ isn't defined on arrays. So you lose a lot of the unity that the syntax affords you in C++.
Dec 27 2008