www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Operator overloading

reply aarti_pl <aarti interia.pl> writes:
Andrei Alexandrescu pisze:

 We're trying to make that work. D is due for an operator overhaul.

 Andrei


Is there any chance that we get possibility to overload "raw operators", 
like in C++? I think that they may coexist with currently defined 
operator overloads with simple semantic rules, which will not allow them 
to work together at the same time.

I know that they may obfuscate code, but they are extremely useful when 
defining in D domain specific languages. For example I try to make SQL 
expressions work as D expressions:

string query = "SELECT * FROM a WHERE id=5;";
-->
Query query = Select(a).Where(Equals(id, 5));

* "id" is D object in second expression

This translation is quite awful and unreadable. It would be so much 
better to get:

Query query = Select(a).Where(id == 5);

With implicit casts there will be perfect environment, to define DSL-s 
*in* D and *not* using unsafe strings.

BR
Marcin Kuszczak
(aarti_pl)
Dec 23 2008
next sibling parent reply Denis Koroskin <2korden gmail.com> writes:
aarti_pl Wrote:


 Andrei Alexandrescu pisze:
  > We're trying to make that work. D is due for an operator overhaul.
  >
  > Andrei
 
 Is there any chance that we get possibility to overload "raw operators", 
 like in C++? I think that they may coexist with currently defined 
 operator overloads with simple semantic rules, which will not allow them 
 to work together at the same time.
 
 I know that they may obfuscate code, but they are extremely useful when 
 defining in D domain specific languages. For example I try to make SQL 
 expressions work as D expressions:
 
 string query = "SELECT * FROM a WHERE id=5;";
 -->
 Query query = Select(a).Where(Equals(id, 5));
 
 * "id" is D object in second expression
 
 This translation is quite awful and unreadable. It would be so much 
 better to get:
 
 Query query = Select(a).Where(id == 5);
 
 With implicit casts there will be perfect environment, to define DSL-s 
 *in* D and *not* using unsafe strings.
 
 BR
 Marcin Kuszczak
 (aarti_pl)


Slightly off-topic.

How would you implement, say, LIKE condition? Similar to the following, perhaps:
Query query = Select(a).Where(id.Like("%substring%"));

You could implement "==" the same way:
Query query = Select(a).Where(id.EqualsTo(5));
Dec 23 2008
next sibling parent aarti_pl <aarti interia.pl> writes:
Denis Koroskin pisze:

 Slightly off-topic.
 
 How would you implement, say, LIKE condition? Similar to the following,
perhaps:
 Query query = Select(a).Where(id.Like("%substring%"));


Well, probably:

...Like(id, "substring")...

In this case I prefer this kind of syntax more than OO kind of syntax :-)


 You could implement "==" the same way:
 Query query = Select(a).Where(id.EqualsTo(5));


In fact LIKE is not as a big problem as other operators. You will rather 
not use nested LIKEs, but only the simplest form.

The best solution would be to introduce into language concept of infix 
operator, so that any symbol could be such an operator. It would be most 
universal solution. But it would be probably too much to ask...

Completely different situation than with LIKE is with comparison and 
logical operators. They form very complicated expressions eg.:

Where(Or(And(Equals(surname, "Smith"), NotEquals(age, 25))), 
And(Equals(surname, "Neo"), NotEquals(age, 50)))

With my syntax it would be:
Where((surname == "Smith" && age != 25) || (surname == "Neo" && age != 50))

I don't know how to write it using your syntax, because I don't have 
good idea for AND/OR.

But the most important argument is that syntax, where operators are 
parts of classes will cause unnecessary coupling of classes and concepts.

In my case objects which are used to identify columns are universal. 
Using same identifiers you can also get values from resulting table.
When I implement parts of expression in Column classes I will loose this 
decoupling: someone who just wants my resulting table object + column 
identifiers will have to see api for sql expressions, which he/she will 
not use at all.

Best Regards
(aarti_pl)
Dec 23 2008
prev sibling parent reply Christopher Wright <dhasenan gmail.com> writes:
Denis Koroskin wrote:

 Slightly off-topic.
 
 How would you implement, say, LIKE condition? Similar to the following,
perhaps:
 Query query = Select(a).Where(id.Like("%substring%"));
 
 You could implement "==" the same way:
 Query query = Select(a).Where(id.EqualsTo(5));


downs way:
auto query1 = select(a) /where/ "id" /eq/ 5;
auto query2 = select(a) /where/ "name" /like/ "%bob%";
Dec 23 2008
next sibling parent reply Sergey Gromov <snake.scaly gmail.com> writes:
Tue, 23 Dec 2008 19:14:16 -0500, Christopher Wright wrote:


 Denis Koroskin wrote:

 Slightly off-topic.
 
 How would you implement, say, LIKE condition? Similar to the following,
perhaps:
 Query query = Select(a).Where(id.Like("%substring%"));
 
 You could implement "==" the same way:
 Query query = Select(a).Where(id.EqualsTo(5));


 
 downs way:
 auto query1 = select(a) /where/ "id" /eq/ 5;
 auto query2 = select(a) /where/ "name" /like/ "%bob%";


What a nice trick!

I don't like operator abuse, but sometimes it's a lesser evil.  And it's
definitely the case with SQL queries.
Dec 24 2008
parent aarti_pl <aarti interia.pl> writes:
Sergey Gromov pisze:

 downs way:
 auto query1 = select(a) /where/ "id" /eq/ 5;
 auto query2 = select(a) /where/ "name" /like/ "%bob%";


 
 What a nice trick!
 
 I don't like operator abuse, but sometimes it's a lesser evil.  And it's
 definitely the case with SQL queries.


I agree. In D raw operator overloading should be probably as 
discouraging as cast operator :-)

e.g.
opRawEqual()
opRawNotEquals()
etc.

BR
Marcin Kuszczak
(aarti_pl)
Dec 24 2008
prev sibling parent aarti_pl <aarti interia.pl> writes:
Christopher Wright pisze:

 Denis Koroskin wrote:

 Slightly off-topic.

 How would you implement, say, LIKE condition? Similar to the 
 following, perhaps:
 Query query = Select(a).Where(id.Like("%substring%"));

 You could implement "==" the same way:
 Query query = Select(a).Where(id.EqualsTo(5));


 
 downs way:
 auto query1 = select(a) /where/ "id" /eq/ 5;
 auto query2 = select(a) /where/ "name" /like/ "%bob%";


Nice, but I am not so sure that as much intuitive? :-)

BR
Marcin Kuszczak
(aarti_pl)
Dec 24 2008
prev sibling next sibling parent reply dennis luehring <dl.soluz gmx.net> writes:
 ...
 This translation is quite awful and unreadable. It would be so much 
 better to get:
 
 Query query = Select(a).Where(id == 5);


what speaks against an sql parsing mixin?

would be more expressive, compiletime based and typesafe
and even far more be able than what you can do with operator overloading
Dec 23 2008
next sibling parent reply Ary Borenszweig <ary esperanto.org.ar> writes:
dennis luehring escribió:

 ...
 This translation is quite awful and unreadable. It would be so much 
 better to get:

 Query query = Select(a).Where(id == 5);


 
 what speaks against an sql parsing mixin?
 
 would be more expressive, compiletime based and typesafe
 and even far more be able than what you can do with operator overloading


Autocompletion and syntax coloring.
Dec 23 2008
next sibling parent reply aarti_pl <aarti interia.pl> writes:
Ary Borenszweig pisze:

 dennis luehring escribió:

 ...
 This translation is quite awful and unreadable. It would be so much 
 better to get:

 Query query = Select(a).Where(id == 5);


 what speaks against an sql parsing mixin?

 would be more expressive, compiletime based and typesafe
 and even far more be able than what you can do with operator overloading


 
 Autocompletion and syntax coloring.


Yes. And additionally:
* you can not build safely your queries on runtime
* when moving Query around in program you will not be able to make it 
typesafe, because it is just string
* in my db access system it won't be possible to automatically create 
database, typesafely get results from resulting table and few other nice 
features...

BR
Marcin Kuszczak
(aarti_pl)
Dec 23 2008
parent reply dennis luehring <dl.soluz gmx.net> writes:
aarti_pl schrieb:

 Ary Borenszweig pisze:

 dennis luehring escribió:

 ...
 This translation is quite awful and unreadable. It would be so much 
 better to get:

 Query query = Select(a).Where(id == 5);


 what speaks against an sql parsing mixin?

 would be more expressive, compiletime based and typesafe
 and even far more be able than what you can do with operator overloading


 
 Autocompletion and syntax coloring.





 Yes. And additionally:
 * you can not build safely your queries on runtime


that even not possible with operators - or?


 * when moving Query around in program you will not be able to make it 
 typesafe, because it is just string


not if you parse the string at compiletime an use the
type information from outer scope for validation


 * in my db access system it won't be possible to automatically create 
 database, typesafely get results from resulting table and few other nice 
 features...


just a quick look into blades (library with vector math language 
extensions - based on compiletime strings)

http://www.dsource.org/projects/mathextra/browser/trunk/blade/BladeDemo.d

don referes to outer vars in his "strings" and you can typecheck them
Dec 23 2008
parent reply aarti_pl <aarti interia.pl> writes:
dennis luehring pisze:

 aarti_pl schrieb:

 Ary Borenszweig pisze:

 dennis luehring escribió:

 ...
 This translation is quite awful and unreadable. It would be so much 
 better to get:

 Query query = Select(a).Where(id == 5);


 what speaks against an sql parsing mixin?

 would be more expressive, compiletime based and typesafe
 and even far more be able than what you can do with operator 
 overloading


 Autocompletion and syntax coloring.




 

 Yes. And additionally:
 * you can not build safely your queries on runtime


 
 that even not possible with operators - or?


It is possible and it works :-)
But with operator overloading it is so much more readable, than without it.

Creating queries at runtime is quite common:

age = 20;
string query = "Select name, surname FROM persons WHERE age = " + 
to!(string)(age);


 * when moving Query around in program you will not be able to make it 
 typesafe, because it is just string


 
 not if you parse the string at compiletime an use the
 type information from outer scope for validation


Compile time checking of SQLs will not be enough for any bigger program.
How would you insert your data into database? INSERT needs values which 
are taken e.g. from GUI and completely unknown at compile-time.


 * in my db access system it won't be possible to automatically create 
 database, typesafely get results from resulting table and few other 
 nice features...


 
 just a quick look into blades (library with vector math language 
 extensions - based on compiletime strings)
 
 http://www.dsource.org/projects/mathextra/browser/trunk/blade/BladeDemo.d
 
 don referes to outer vars in his "strings" and you can typecheck them


It doesn't solve all problems. Just a one.

BTW. In recent release Boost team added a whole brand new library for 
domain specific languages embedded languages called Proto. I think that 
it means that there is quite a big need for such features. E.g. for 
mocking frameworks which will also profit from extended DSL sublanguages 
support in programming languages.

BR
Marcin Kuszczak
(aarti_pl)
Dec 23 2008
parent reply dennis luehring <dl.soluz gmx.net> writes:
 It is possible and it works :-)
 But with operator overloading it is so much more readable, than without it.
 
 Creating queries at runtime is quite common:
 
 age = 20;
 string query = "Select name, surname FROM persons WHERE age = " + 
 to!(string)(age);


that is not object + operator based - your flexibilty here comes from 
your string and the "runtime" interpretation of the sql semantic in your 
database ...

if you need an additional "LIKE" based on an codition you need to 
rebuild your object/operator concatination in complete:

if( gui.checkbox == true )
{
   SELECT a from b
}
else
{
   SELECT a from b WHERE c LIKE '% ~ C ~ %'
}


 Compile time checking of SQLs will not be enough for any bigger program.
 How would you insert your data into database? INSERT needs values which 
 are taken e.g. from GUI and completely unknown at compile-time.


why is a value that comes from an gui not compile-time?
not the value is what you need - its the type (when we talk about type 
safetyness)

how does vector-array get into Don's Blade strings? and why do i get
compile-time errors when using the wrong type inside of the string?

and you can't even speak of type-safetyness when using only strings in 
your database communication - how can be your results typesafe
(i don't like string[string] results-set) because the safetiness is 
programatical


 BTW. In recent release Boost team added a whole brand new library for 
 domain specific languages embedded languages called Proto. I think that 
 it means that there is quite a big need for such features. E.g. for 
 mocking frameworks which will also profit from extended DSL sublanguages 
 support in programming languages.


you should read into the details of using mixins - they are much much 
more flexible than you think (not just boring strings) - and with the 
base feature-set of D something like mock wouldn't even exist
and what the boost guys to is to get around c++ base feature-set to 
express new thing with c++

 Ary does syntax coloring and smart auto-completen help while writing
spirit based parsers? the complete meaning of ( * - \ ... is different 
to "normal" code

btw: don syntaxtree is also an great example of using mixins
http://www.dsource.org/projects/mathextra/browser/trunk/blade/SyntaxTree.d
Dec 23 2008
next sibling parent reply Ary Borenszweig <ary esperanto.org.ar> writes:
dennis luehring escribió:

 It is possible and it works :-)
 But with operator overloading it is so much more readable, than 
 without it.

 Creating queries at runtime is quite common:

 age = 20;
 string query = "Select name, surname FROM persons WHERE age = " + 
 to!(string)(age);


 
 that is not object + operator based - your flexibilty here comes from 
 your string and the "runtime" interpretation of the sql semantic in your 
 database ...
 
 if you need an additional "LIKE" based on an codition you need to 
 rebuild your object/operator concatination in complete:
 
 if( gui.checkbox == true )
 {
   SELECT a from b
 }
 else
 {
   SELECT a from b WHERE c LIKE '% ~ C ~ %'
 }
 

 Compile time checking of SQLs will not be enough for any bigger program.
 How would you insert your data into database? INSERT needs values 
 which are taken e.g. from GUI and completely unknown at compile-time.


 
 why is a value that comes from an gui not compile-time?
 not the value is what you need - its the type (when we talk about type 
 safetyness)
 
 how does vector-array get into Don's Blade strings? and why do i get
 compile-time errors when using the wrong type inside of the string?
 
 and you can't even speak of type-safetyness when using only strings in 
 your database communication - how can be your results typesafe
 (i don't like string[string] results-set) because the safetiness is 
 programatical
 

 BTW. In recent release Boost team added a whole brand new library for 
 domain specific languages embedded languages called Proto. I think 
 that it means that there is quite a big need for such features. E.g. 
 for mocking frameworks which will also profit from extended DSL 
 sublanguages support in programming languages.


 
 you should read into the details of using mixins - they are much much 
 more flexible than you think (not just boring strings) - and with the 
 base feature-set of D something like mock wouldn't even exist
 and what the boost guys to is to get around c++ base feature-set to 
 express new thing with c++
 
  Ary does syntax coloring and smart auto-completen help while writing
 spirit based parsers? the complete meaning of ( * - \ ... is different 
 to "normal" code


I was thinking about LINQ, specifically. Ok, LINQ is too much (and too 
specific), I wouldn't like something like that in D, but something 
similar...


 
 btw: don syntaxtree is also an great example of using mixins
 http://www.dsource.org/projects/mathextra/browser/trunk/blade/SyntaxTree.d
Dec 24 2008
parent dennis luehring <dl.soluz gmx.net> writes:
 I was thinking about LINQ, specifically. Ok, LINQ is too much (and too 
 specific), I wouldn't like something like that in D, but something 
 similar...


my point is that operator overloading helps only i a very few cases - 
and even then (see syntaxhighliing in boost::spirit based parsers) it 
does not realy help

but it think (again, except the compiler bugs) D is be able to do 
something like LINQ (as an example) with mixins (or ast-macros) without
the need of extending the language itselfe - only the parsing on 
compiletime thing needs to be easier - but any solution(s) in this part 
of D will also help all other generic library developers
Dec 24 2008
prev sibling parent aarti_pl <aarti interia.pl> writes:
dennis luehring pisze:

 It is possible and it works :-)
 But with operator overloading it is so much more readable, than 
 without it.

 Creating queries at runtime is quite common:

 age = 20;
 string query = "Select name, surname FROM persons WHERE age = " + 
 to!(string)(age);


 
 that is not object + operator based - your flexibilty here comes from 
 your string and the "runtime" interpretation of the sql semantic in your 
 database ...
 
 if you need an additional "LIKE" based on an codition you need to 
 rebuild your object/operator concatination in complete:
 
 if( gui.checkbox == true )
 {
   SELECT a from b
 }
 else
 {
   SELECT a from b WHERE c LIKE '% ~ C ~ %'
 }
 

 Compile time checking of SQLs will not be enough for any bigger program.
 How would you insert your data into database? INSERT needs values 
 which are taken e.g. from GUI and completely unknown at compile-time.


 
 why is a value that comes from an gui not compile-time?
 not the value is what you need - its the type (when we talk about type 
 safetyness)
 
 how does vector-array get into Don's Blade strings? and why do i get
 compile-time errors when using the wrong type inside of the string?
 
 and you can't even speak of type-safetyness when using only strings in 
 your database communication - how can be your results typesafe
 (i don't like string[string] results-set) because the safetiness is 
 programatical
 

 BTW. In recent release Boost team added a whole brand new library for 
 domain specific languages embedded languages called Proto. I think 
 that it means that there is quite a big need for such features. E.g. 
 for mocking frameworks which will also profit from extended DSL 
 sublanguages support in programming languages.


 
 you should read into the details of using mixins - they are much much 
 more flexible than you think (not just boring strings) - and with the 
 base feature-set of D something like mock wouldn't even exist
 and what the boost guys to is to get around c++ base feature-set to 
 express new thing with c++
 
  Ary does syntax coloring and smart auto-completen help while writing
 spirit based parsers? the complete meaning of ( * - \ ... is different 
 to "normal" code
 
 btw: don syntaxtree is also an great example of using mixins
 http://www.dsource.org/projects/mathextra/browser/trunk/blade/SyntaxTree.d


While I see your points here, there are still arguments against mixins. 
Please see my answer to Don's post.

BR
Marcin
(aarti_pl)
Dec 24 2008
prev sibling next sibling parent reply dennis luehring <dl.soluz gmx.net> writes:
 Autocompletion and syntax coloring.


ok one reason - but is the sql syntax not crippled enough using 
operators - better if colored/autocompleted?

what does coloring help here

select
   x,
   y = (select u from b where ... ),
from select( xyz from abc )
left join

in the special case of inner selects, from selects, left/right/inner 
joins...

the ultimate++ solution just works for very very simple statements
the readability is flying away

i personaly whould throw away the idea of getting functional style 
programming like sql into an imperative language using operator 
overloading or better -> i would not ask for more operator overloading 
features based on this need :-)

what about other domain-specific languages like the blade stuff
or an ebnf parser (dparser?) - should these be more operator overloading 
dependend for better coloring and auto-completion? i think operator 
overloding is not the answer here

nothing agains autocompletion and syntax coloring - very much needed
decents smart autocompletion shows how it should be
Dec 23 2008
next sibling parent reply Uno <unodgs tlen.pl> writes:
 the ultimate++ solution just works for very very simple statements


it works for every query. You can write for example:
Select(PRODUCTS[ID], CUSTOMER[NAME])
.From(PRODUCTS)
.InnerJoin(CUSTOMERS)
.On(CUSTOMERS[PRODUCT_ID] == PRODUCTS[ID] && Like(PRODUCTS[NAME], "BOOK%"))
.Where(PRODUCTS[ID] > 10);

You can have another statement in where or even in select..
Dec 24 2008
parent reply dennis luehring <dl.soluz gmx.net> writes:
Uno schrieb:

 the ultimate++ solution just works for very very simple statements


 it works for every query. You can write for example:
 Select(PRODUCTS[ID], CUSTOMER[NAME])
 .From(PRODUCTS)
 .InnerJoin(CUSTOMERS)
 .On(CUSTOMERS[PRODUCT_ID] == PRODUCTS[ID] && Like(PRODUCTS[NAME], "BOOK%"))
 .Where(PRODUCTS[ID] > 10);
 
 You can have another statement in where or even in select..


and "from select"

btw: i don't want to say: "its not possible" - i want
to say: "it is even clear to handle in D" without going the 
no-other-way-in-c++ solution
Dec 24 2008
parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
dennis luehring Wrote:


 Uno schrieb:

 the ultimate++ solution just works for very very simple statements


 it works for every query. You can write for example:
 Select(PRODUCTS[ID], CUSTOMER[NAME])
 .From(PRODUCTS)
 .InnerJoin(CUSTOMERS)
 .On(CUSTOMERS[PRODUCT_ID] == PRODUCTS[ID] && Like(PRODUCTS[NAME], "BOOK%"))
 .Where(PRODUCTS[ID] > 10);
 
 You can have another statement in where or even in select..


 
 and "from select"
 
 btw: i don't want to say: "its not possible" - i want
 to say: "it is even clear to handle in D" without going the 
 no-other-way-in-c++ solution
 


 I suppose you could do it that way, at least it would definately be readable
rather than what i've seen in java (and ASP) before.

 query = "Select * from persons where firstname='"+fname+" and
lastname='"+lname+" and age="+age+... ect.ect

 Recently i've started reading about lex and yacc (or flex and bison) and as a
comment in there, it wouldn't be too hard to use substitution and have a
pre-processor convert it into pure D code.

 SQLQuery("
Select * from persons
where firstname=%fname%
and lastname=%lname%
and age=%age%")

Just a thought.
Dec 24 2008
parent dennis luehring <dl.soluz gmx.net> writes:
Era Scarecrow schrieb:

  Recently i've started reading about lex and yacc (or flex and bison) and as a
comment in there, it wouldn't be too hard to use substitution and have a
pre-processor convert it into pure D code.


and the name of that pre-processor is "mixin" in D, you can parse and 
generate code in here :-)
Dec 25 2008
prev sibling parent reply BCS <ao pathlink.com> writes:
Reply to dennis,


 what about other domain-specific languages like the blade stuff
 or an ebnf parser (dparser?)


dparse  

http://www.dsource.org/projects/scrapple/browser/trunk/dparser/dparse.d


 - should these be more operator overloading dependend


Avoiding operator overloading was one of the objectives of dpares. My opinion 
of DSL's is that they should either use a syntax designed specifically for 
them or use a syntax that is designed to be much more general than for a 
general purpose language. But then I'm biased as this is getting into my 
day job.
Dec 24 2008
parent dennis luehring <dl.soluz gmx.net> writes:
BCS schrieb:


My opinion of DSL's is that they should either
use a syntax designed specifically for
them or use a syntax that is designed to be much more
general than for a general purpose language.


perfect answer and my opinion too
Dec 25 2008
prev sibling next sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Wed, 24 Dec 2008 00:01:20 +0300, Ary Borenszweig <ary esperanto.org.ar>
wrote:


 dennis luehring escribió:

 ...
 This translation is quite awful and unreadable. It would be so much  
 better to get:

 Query query = Select(a).Where(id == 5);


  what speaks against an sql parsing mixin?
  would be more expressive, compiletime based and typesafe
 and even far more be able than what you can do with operator overloading


 Autocompletion and syntax coloring.


And some compile-time checks (so that you don't mistype your tables/columns):

enum Artists
{
    Id,
    Name,
}

enum Albums
{
    Id,
    Name,
    ArtistId,
}

enum Songs
{
    Id,
    Name,
    AlbumId,
}

auto request = Select(Songs.Id, Songs.Name).
			From!(Artists).
				InnerJoin!(Albums).OnEqual(Albums.ArtistId, Artists.Id).
				InnerJoin!(Songs).OnEqual(Songs.AlbumId, Albums.Id).
			Where(Artists.Name).EqualsTo("Beatles").
			And(Albums.Name).EqualsTo("Yellow Submarine").
			OrderBy(Songs.Name).
			Limit(0, 1);

Just an example :)
Dec 23 2008
prev sibling parent reply Ary Borenszweig <ary esperanto.org.ar> writes:
Ary Borenszweig escribió:

 dennis luehring escribió:

 ...
 This translation is quite awful and unreadable. It would be so much 
 better to get:

 Query query = Select(a).Where(id == 5);


 what speaks against an sql parsing mixin?

 would be more expressive, compiletime based and typesafe
 and even far more be able than what you can do with operator overloading


 
 Autocompletion and syntax coloring.


And also, you get a nice compile error message if the query is 
malformed. With a mixin you might end up with a stack of 
hard-to-interpret error messages.

Finally, parsing strings is not easy, specially at compile time. And 
there's also the problem of lack of garbage collection at compile time...
Dec 23 2008
parent dennis luehring <dl.soluz gmx.net> writes:
Ary Borenszweig schrieb:
  > Finally, parsing strings is not easy, specially at compile time. And

 there's also the problem of lack of garbage collection at compile time...


ok thats 140% true

but i think that using mixins to embedd domain-spec-lang into D is 
currently the best way doing it (except the compiler bugs around)
the only other thing on my radar is the "ast-macro" feature 
walter,andrei,.... are talking about
Dec 23 2008
prev sibling parent aarti_pl <aarti interia.pl> writes:
dennis luehring pisze:

 ...
 This translation is quite awful and unreadable. It would be so much 
 better to get:

 Query query = Select(a).Where(id == 5);


 
 what speaks against an sql parsing mixin?
 
 would be more expressive, compiletime based and typesafe
 and even far more be able than what you can do with operator overloading
 


Please see additional points, which I put into answer for Ary.

BR
Marcin Kuszczak
(aarti_pl)
Dec 23 2008
prev sibling next sibling parent reply Don <nospam nospam.com> writes:
aarti_pl wrote:

 Andrei Alexandrescu pisze:
  > We're trying to make that work. D is due for an operator overhaul.
  >
  > Andrei
 
 Is there any chance that we get possibility to overload "raw operators", 
 like in C++? I think that they may coexist with currently defined 
 operator overloads with simple semantic rules, which will not allow them 
 to work together at the same time.
 
 I know that they may obfuscate code, but they are extremely useful when 
 defining in D domain specific languages. For example I try to make SQL 
 expressions work as D expressions:



 
 string query = "SELECT * FROM a WHERE id=5;";
 -->
 Query query = Select(a).Where(Equals(id, 5));
 
 * "id" is D object in second expression
 
 This translation is quite awful and unreadable. It would be so much 
 better to get:
 
 Query query = Select(a).Where(id == 5);


I think that's still awful. I prefer:
Query query = mixin(SQL("SELECT * FROM a WHERE id=5"));



 
 With implicit casts there will be perfect environment, to define DSL-s 
 *in* D and *not* using unsafe strings.
 
 BR
 Marcin Kuszczak
 (aarti_pl)
Dec 24 2008
parent reply aarti_pl <aarti interia.pl> writes:
Don pisze:

 aarti_pl wrote:

 Andrei Alexandrescu pisze:
  > We're trying to make that work. D is due for an operator overhaul.
  >
  > Andrei

 Is there any chance that we get possibility to overload "raw 
 operators", like in C++? I think that they may coexist with currently 
 defined operator overloads with simple semantic rules, which will not 
 allow them to work together at the same time.

 I know that they may obfuscate code, but they are extremely useful 
 when defining in D domain specific languages. For example I try to 
 make SQL expressions work as D expressions:


 

 string query = "SELECT * FROM a WHERE id=5;";
 -->
 Query query = Select(a).Where(Equals(id, 5));

 * "id" is D object in second expression

 This translation is quite awful and unreadable. It would be so much 
 better to get:

 Query query = Select(a).Where(id == 5);


 
 I think that's still awful. I prefer:
 Query query = mixin(SQL("SELECT * FROM a WHERE id=5"));


Thanks for example - it talks to me more than 1000 words :-)

Your proposal is indeed interesting. (And I think it is what Dennis 
Luehring was reffering to).

But I have to say it has some drawbacks. Let me enumerate them:

1. Extending such a queries on runtime would be rather difficult IMHO, 
and will not look very nice. With my solution you can do something like 
this:
---
Column cols = [person.name, person.surname, person.nick];
for(i=0; i<rows; i++) {
   InsertStatement insert = Insert();
   foreach(Column col; cols) {
     insert ~= insert.insert(col, getText(i, col));
   }
   db.execute(insert);
}

* example not tested, but something like this is already possible in my 
framework.

Constructing queries dynamically is very useful for creating e.g. 
universal gui controls which could be just binded to database tables.

Another useful example is sending SQL expression (not the whole 
statement) as an argument to function.

To make it work nicely with mixins IMHO you will have to make this kind 
of SQL to look much uglier than in this simple case.

Do you have any proposals how it could look like with mixins?

2. I see also problem with IDE-s which will have to be changed to 
understand DLSs in mixins. I think that it is doable and even can be 
done quite nice, but currently there is *no* support for such a feature. 
It means that you will not have:

a. syntax highlighting
b. no IDE proposing your names of variables and methods
c. no refactoring

Currently in Eclipse I can rename database column names in my program 
with only single mouse click (Java version of my framework, which is the 
main branch currently). When SQLs will be in form of string mixins they 
will not change during refactoring. You will have to write special 
refactoring tools for your library to make it done.

3. Sometimes having more expressiveness in language can avoid creating 
new DSLs. I think it is a Good Thing (tm). In fact I took that way and 
decided to integrate SQL DSL language into mother language. That way 
programmers don't have to learn new syntax, but just use existing syntax 
of mother language to solve specific problems. Using DSL in mother 
language resolves then into just writing special kind of API, which can 
be used in standard way to achieve goal. D and most other languages from 
C family are almost perfect candidates to make something like this with SQL.

So while I can agree that original SQL looks much better than any kind 
of its imitation, I think that it is not necessary better for productivity.

Additionally I think that string mixins need some more features to make 
it possible to solve point 2. IDE will have to know what kind of DSL is 
in mixin. Maybe something like this:

mixin(sql.query, "SELECT ....");
mixin(sql.expression, "person.doctorid = doctor.id");
mixin(sql.statement, "INSERT ....");

...where sql.query, sql.expression, sql.statement are kind of markers 
usefull for IDE.

BR
Marcin Kuszczak
(aarti_pl)
Dec 24 2008
parent Don <nospam nospam.com> writes:
aarti_pl wrote:

 Don pisze:

 aarti_pl wrote:

 Andrei Alexandrescu pisze:
  > We're trying to make that work. D is due for an operator overhaul.
  >
  > Andrei

 Is there any chance that we get possibility to overload "raw 
 operators", like in C++? I think that they may coexist with currently 
 defined operator overloads with simple semantic rules, which will not 
 allow them to work together at the same time.

 I know that they may obfuscate code, but they are extremely useful 
 when defining in D domain specific languages. For example I try to 
 make SQL expressions work as D expressions:



 string query = "SELECT * FROM a WHERE id=5;";
 -->
 Query query = Select(a).Where(Equals(id, 5));

 * "id" is D object in second expression

 This translation is quite awful and unreadable. It would be so much 
 better to get:

 Query query = Select(a).Where(id == 5);


 I think that's still awful. I prefer:
 Query query = mixin(SQL("SELECT * FROM a WHERE id=5"));


  >
 
 Thanks for example - it talks to me more than 1000 words :-)
 
 Your proposal is indeed interesting. (And I think it is what Dennis 
 Luehring was reffering to).
 
 But I have to say it has some drawbacks. Let me enumerate them:
 
 1. Extending such a queries on runtime would be rather difficult IMHO, 
 and will not look very nice. With my solution you can do something like 
 this:
 ---
 Column cols = [person.name, person.surname, person.nick];
 for(i=0; i<rows; i++) {
   InsertStatement insert = Insert();
   foreach(Column col; cols) {
     insert ~= insert.insert(col, getText(i, col));
   }
   db.execute(insert);
 }
 
 * example not tested, but something like this is already possible in my 
 framework.
 
 Constructing queries dynamically is very useful for creating e.g. 
 universal gui controls which could be just binded to database tables.
 
 Another useful example is sending SQL expression (not the whole 
 statement) as an argument to function.
 
 To make it work nicely with mixins IMHO you will have to make this kind 
 of SQL to look much uglier than in this simple case.
 
 Do you have any proposals how it could look like with mixins?


There's nothing stopping you from using local variables in the mixin 
string. A simple possibility is to use $ for embedded variables. (You 
can avoid that, but it's much more difficult, and has some minor 
downsides, you can't have use a local variable called 'select', for 
example).
I pretty much got this working for BLADE, but the CTFE memory bug makes 
it completely unusable at present.


 2. I see also problem with IDE-s which will have to be changed to 
 understand DLSs in mixins. I think that it is doable and even can be 
 done quite nice, but currently there is *no* support for such a feature. 
 It means that you will not have:
 
 a. syntax highlighting
 b. no IDE proposing your names of variables and methods
 c. no refactoring


That's true, and indeed very difficult to solve.


 3. Sometimes having more expressiveness in language can avoid creating 
 new DSLs. I think it is a Good Thing (tm). In fact I took that way and 
 decided to integrate SQL DSL language into mother language. That way 
 programmers don't have to learn new syntax, but just use existing syntax 
 of mother language to solve specific problems. Using DSL in mother 
 language resolves then into just writing special kind of API, which can 
 be used in standard way to achieve goal. D and most other languages from 
 C family are almost perfect candidates to make something like this with 
 SQL.
 
 So while I can agree that original SQL looks much better than any kind 
 of its imitation, I think that it is not necessary better for productivity.


That's a worthwhile criticism of DSLs in general.
Dec 26 2008
prev sibling next sibling parent reply The Anh Tran <trtheanh gmail.com> writes:
aarti_pl wrote:

 Andrei Alexandrescu pisze:
  > We're trying to make that work. D is due for an operator overhaul.
  >
  > Andrei
 
 Is there any chance that we get possibility to overload "raw operators", 
 like in C++? I think that they may coexist with currently defined 
 operator overloads with simple semantic rules, which will not allow them 
 to work together at the same time.
 ..........
 BR
 Marcin Kuszczak
 (aarti_pl)


Me also have a dream :D

<Daydream mode>
class Foo
{
	auto op(++)(); // bar++
	auto op(++)(int); // ++bar

	op(cast)(uint); // cast(uint)bar // opCast
	auto op(())(int, float); // Foo(123, 123.456) // opCall
	
	auto op(+)(Foo rhs); // bar1 + bar2
	auto op(+=)(int); // bar += 1234;
	auto op(.)(); // bar.xyz // opDot

	Foo op([][][])(int, char, float); // bar[123]['x'][123.456]

	auto op([..])(); // i = bar2[] // opSlide
	auto op([..])(int, int); // bar[1..10]

	auto op([..]=)(float); // bar[] = 12.3 //opSlideAssign
	auto op([..]=)(int, int, float); // bar[1..3] = 123.4
}
</Dream>
Dec 26 2008
next sibling parent reply "Bill Baxter" <wbaxter gmail.com> writes:
On Sat, Dec 27, 2008 at 9:42 AM, The Anh Tran <trtheanh gmail.com> wrote:

 aarti_pl wrote:

 Andrei Alexandrescu pisze:
  > We're trying to make that work. D is due for an operator overhaul.
  >
  > Andrei

 Is there any chance that we get possibility to overload "raw operators",
 like in C++? I think that they may coexist with currently defined operator
 overloads with simple semantic rules, which will not allow them to work
 together at the same time.
 ..........
 BR
 Marcin Kuszczak
 (aarti_pl)


 Me also have a dream :D

 <Daydream mode>
 class Foo
 {
        auto op(++)(); // bar++
        auto op(++)(int); // ++bar

        op(cast)(uint); // cast(uint)bar // opCast
        auto op(())(int, float); // Foo(123, 123.456) // opCall

        auto op(+)(Foo rhs); // bar1 + bar2
        auto op(+=)(int); // bar += 1234;
        auto op(.)(); // bar.xyz // opDot

        Foo op([][][])(int, char, float); // bar[123]['x'][123.456]

        auto op([..])(); // i = bar2[] // opSlide
        auto op([..])(int, int); // bar[1..10]

        auto op([..]=)(float); // bar[] = 12.3 //opSlideAssign
        auto op([..]=)(int, int, float); // bar[1..3] = 123.4
 }
 </Dream>


When I suggested this kind of thing long ago, Walter said that it
encourages operator overload abuse, because it suggests that  + is
just a generic symbolic operator rather than something that
specifically means "addition".  That's why D uses "opAdd" instead.
It's supposed to encourage only creating overloads that follow the
original meaning of the operator closely.  That way when you see a+b
you can be reasonably sure that it means addition or something quite
like it.  It also goes hand-in-hand with design decisions like
defining ++x to be x+=1, which means that in D it's impossible to make
++x mean something distinct from incrementing by 1.  With C++ you can
make ++x have whatever meaning you want.  It can do something
completely different from x+=1.  The idea is that such freedom just
makes code harder to read.

Currently I don't think it makes much difference either way.  The only
real advantage I see to the alternate syntax is that it can be perhaps
a little easier to remember.  But I also I don't think Walter's idea
about naming following usage does anything to stop someone like Downs
from using opDiv to do something completely different from division.
It probably never even occurred to Downs that Walter was trying to
prevent him from abusing opDiv by naming it opDiv instead of
operator(/).  The people who are likely to abuse operators are
precisely those who aren't likely to be daunted by mere naming.

But anyway, just so you know, that's why D does things the way it
does.  So that means to see your dream come true you first have to
convince Walter that it's a dream worth having. :-)

--bb
Dec 26 2008
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Bill Baxter wrote:

 On Sat, Dec 27, 2008 at 9:42 AM, The Anh Tran <trtheanh gmail.com> wrote:

 aarti_pl wrote:

 Andrei Alexandrescu pisze:
  > We're trying to make that work. D is due for an operator overhaul.
  >
  > Andrei

 Is there any chance that we get possibility to overload "raw operators",
 like in C++? I think that they may coexist with currently defined operator
 overloads with simple semantic rules, which will not allow them to work
 together at the same time.
 ..........
 BR
 Marcin Kuszczak
 (aarti_pl)


 Me also have a dream :D

 <Daydream mode>
 class Foo
 {
        auto op(++)(); // bar++
        auto op(++)(int); // ++bar

        op(cast)(uint); // cast(uint)bar // opCast
        auto op(())(int, float); // Foo(123, 123.456) // opCall

        auto op(+)(Foo rhs); // bar1 + bar2
        auto op(+=)(int); // bar += 1234;
        auto op(.)(); // bar.xyz // opDot

        Foo op([][][])(int, char, float); // bar[123]['x'][123.456]

        auto op([..])(); // i = bar2[] // opSlide
        auto op([..])(int, int); // bar[1..10]

        auto op([..]=)(float); // bar[] = 12.3 //opSlideAssign
        auto op([..]=)(int, int, float); // bar[1..3] = 123.4
 }
 </Dream>


 
 When I suggested this kind of thing long ago, Walter said that it
 encourages operator overload abuse, because it suggests that  + is
 just a generic symbolic operator rather than something that
 specifically means "addition".  That's why D uses "opAdd" instead.
 It's supposed to encourage only creating overloads that follow the
 original meaning of the operator closely.  That way when you see a+b
 you can be reasonably sure that it means addition or something quite
 like it.


I think that argument is rather weak and ought to be revisited. It's 
weak to start with as if writing "+" in a D program hardly evokes 
anything else but "plus". What the notation effectively achieved was put 
more burden on the programmer to memorize some names for the 
already-known symbols. I think the entire operator overloading business, 
which started from a legitimate desire to improve on C++'s, ended up 
worse off.


 It also goes hand-in-hand with design decisions like
 defining ++x to be x+=1, which means that in D it's impossible to make
 ++x mean something distinct from incrementing by 1.


This is the vomit in the fat lady's cleavage that shows just how bad the 
wine turned out to be. For iterators, increment is quite different from 
addition of an arbitrary number, so what D managed to do was effectively 
to cripple iterators. The standard library will use ranges with named 
functions so it avoids the issue, but if someone wants to define 
STL-style iterators they won't be able to.


 With C++ you can
 make ++x have whatever meaning you want.  It can do something
 completely different from x+=1.  The idea is that such freedom just
 makes code harder to read.


That sounds nice, but on the face of it I haven't heard of much code 
suffering from the problem.


 Currently I don't think it makes much difference either way.  The only
 real advantage I see to the alternate syntax is that it can be perhaps
 a little easier to remember.  But I also I don't think Walter's idea
 about naming following usage does anything to stop someone like Downs
 from using opDiv to do something completely different from division.
 It probably never even occurred to Downs that Walter was trying to
 prevent him from abusing opDiv by naming it opDiv instead of
 operator(/).  The people who are likely to abuse operators are
 precisely those who aren't likely to be daunted by mere naming.
 
 But anyway, just so you know, that's why D does things the way it
 does.  So that means to see your dream come true you first have to
 convince Walter that it's a dream worth having. :-)


The story is quite lame, so it may be worth scrutinizing it. Let's make 
D operators not suck. Wanna?


Andrei
Dec 26 2008
next sibling parent reply "Jarrett Billingsley" <jarrett.billingsley gmail.com> writes:
On Fri, Dec 26, 2008 at 11:20 PM, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:

 I think that argument is rather weak and ought to be revisited. It's weak to
 start with as if writing "+" in a D program hardly evokes anything else but
 "plus". What the notation effectively achieved was put more burden on the
 programmer to memorize some names for the already-known symbols. I think the
 entire operator overloading business, which started from a legitimate desire
 to improve on C++'s, ended up worse off.


It does, however, introduce a nice naming scheme for naming
"meta-methods", that is, methods which are called indirectly to
overload certain language structures.  opApply is an example of such a
method which doesn't truly have a corresponding operator, and I'm sure
there could be other other methods for other language constructs,
existing or theoretical.  (though I'm sure you'll agree opApply's
behavior could use a rehaul too)


 This is the vomit in the fat lady's cleavage that shows just how bad the
 wine turned out to be.


That has to be one of the most disgusting metaphors I've ever heard ;)


 For iterators, increment is quite different from
 addition of an arbitrary number, so what D managed to do was effectively to
 cripple iterators. The standard library will use ranges with named functions
 so it avoids the issue, but if someone wants to define STL-style iterators
 they won't be able to.


I suppose most people who _aren't_ coming from C++ (*cough* like me
*cough*) won't be terribly unhappy about this situation.
Dec 26 2008
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Jarrett Billingsley wrote:

 On Fri, Dec 26, 2008 at 11:20 PM, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 I think that argument is rather weak and ought to be revisited. It's weak to
 start with as if writing "+" in a D program hardly evokes anything else but
 "plus". What the notation effectively achieved was put more burden on the
 programmer to memorize some names for the already-known symbols. I think the
 entire operator overloading business, which started from a legitimate desire
 to improve on C++'s, ended up worse off.


 
 It does, however, introduce a nice naming scheme for naming
 "meta-methods", that is, methods which are called indirectly to
 overload certain language structures.  opApply is an example of such a
 method which doesn't truly have a corresponding operator, and I'm sure
 there could be other other methods for other language constructs,
 existing or theoretical.  (though I'm sure you'll agree opApply's
 behavior could use a rehaul too)
 

 This is the vomit in the fat lady's cleavage that shows just how bad the
 wine turned out to be.


 
 That has to be one of the most disgusting metaphors I've ever heard ;)
 

 For iterators, increment is quite different from
 addition of an arbitrary number, so what D managed to do was effectively to
 cripple iterators. The standard library will use ranges with named functions
 so it avoids the issue, but if someone wants to define STL-style iterators
 they won't be able to.


 
 I suppose most people who _aren't_ coming from C++ (*cough* like me
 *cough*) won't be terribly unhappy about this situation.


I'm not sure how that computes. The particular notion has little to do 
with C++ and is rather fundamental, so not grokking it should motivate 
one to look into it (as opposed to being glad for not knowing).

Andrei
Dec 26 2008
next sibling parent reply "Jarrett Billingsley" <jarrett.billingsley gmail.com> writes:
On Sat, Dec 27, 2008 at 2:31 AM, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:

 For iterators, increment is quite different from
 addition of an arbitrary number, so what D managed to do was effectively
 to
 cripple iterators. The standard library will use ranges with named
 functions
 so it avoids the issue, but if someone wants to define STL-style
 iterators
 they won't be able to.


 I suppose most people who _aren't_ coming from C++ (*cough* like me
 *cough*) won't be terribly unhappy about this situation.


 I'm not sure how that computes. The particular notion has little to do with
 C++ and is rather fundamental, so not grokking it should motivate one to
 look into it (as opposed to being glad for not knowing).


I got the impression from your argument that you thought that without
an appropriate method of overloading the increment operator, it's not
possible to duplicate the _syntax_ of STL-style iterators.  My
response is - who cares?  I don't think of iterators as "pointers that
can be incremented," and I doubt anyone who doesn't have a C++
background thinks of them that way either.  It seems that the
semantics of STL iterators can be implemented with methods just as
well.

If that's not what you were arguing - that is, if you're arguing that
C++'s operator overloading is somehow more expressive and allows you
to implement something that can't be implemented with method calls -
then I will put my foot in my mouth ;)
Dec 27 2008
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Jarrett Billingsley wrote:

 On Sat, Dec 27, 2008 at 2:31 AM, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 For iterators, increment is quite different from
 addition of an arbitrary number, so what D managed to do was effectively
 to
 cripple iterators. The standard library will use ranges with named
 functions
 so it avoids the issue, but if someone wants to define STL-style
 iterators
 they won't be able to.


 I suppose most people who _aren't_ coming from C++ (*cough* like me
 *cough*) won't be terribly unhappy about this situation.


 I'm not sure how that computes. The particular notion has little to do with
 C++ and is rather fundamental, so not grokking it should motivate one to
 look into it (as opposed to being glad for not knowing).


 
 I got the impression from your argument that you thought that without
 an appropriate method of overloading the increment operator, it's not
 possible to duplicate the _syntax_ of STL-style iterators.  My
 response is - who cares?  I don't think of iterators as "pointers that
 can be incremented," and I doubt anyone who doesn't have a C++
 background thinks of them that way either.  It seems that the
 semantics of STL iterators can be implemented with methods just as
 well.
 
 If that's not what you were arguing - that is, if you're arguing that
 C++'s operator overloading is somehow more expressive and allows you
 to implement something that can't be implemented with method calls -
 then I will put my foot in my mouth ;)


Well I overstated my point, sorry. The nice thing about overloading ++ 
is that you can use the same algorithms with built-in types and 
user-defined types. You are, however, right that defining various 
functions such as increment and decrement for built-ins and also 
user-defined types would obviate the need for syntactically consistent 
operators.

Andrei
Dec 27 2008
prev sibling next sibling parent "Bill Baxter" <wbaxter gmail.com> writes:
On Sat, Dec 27, 2008 at 11:19 PM, Jarrett Billingsley
<jarrett.billingsley gmail.com> wrote:

 On Sat, Dec 27, 2008 at 2:31 AM, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 For iterators, increment is quite different from
 addition of an arbitrary number, so what D managed to do was effectively
 to
 cripple iterators. The standard library will use ranges with named
 functions
 so it avoids the issue, but if someone wants to define STL-style
 iterators
 they won't be able to.


 I suppose most people who _aren't_ coming from C++ (*cough* like me
 *cough*) won't be terribly unhappy about this situation.


 I'm not sure how that computes. The particular notion has little to do with
 C++ and is rather fundamental, so not grokking it should motivate one to
 look into it (as opposed to being glad for not knowing).


 I got the impression from your argument that you thought that without
 an appropriate method of overloading the increment operator, it's not
 possible to duplicate the _syntax_ of STL-style iterators.  My
 response is - who cares?  I don't think of iterators as "pointers that
 can be incremented," and I doubt anyone who doesn't have a C++
 background thinks of them that way either.  It seems that the
 semantics of STL iterators can be implemented with methods just as
 well.


There is a little something nice about being able to use a pointer and
iterator interchangeably for templated container algorithms.  You lose
that if ++iter can't be used to iterate your iterator.  But working
around it is not really that hard.  One little incr(T)(ref thing)
template and you're pretty much done.  But ++foo is a concise syntax
that conveys the meaning of what's happening pretty well, so it's a
shame not to be able to use it on iterators.
(Actually I don't really recall why it can't be used for iterators in
D other than it forces you to use more runtime errors for bad args to
+=.  Was there anything more to it than that?)

--bb
Dec 27 2008
prev sibling parent "Jarrett Billingsley" <jarrett.billingsley gmail.com> writes:
On Sat, Dec 27, 2008 at 11:10 AM, Bill Baxter <wbaxter gmail.com> wrote:

 I got the impression from your argument that you thought that without
 an appropriate method of overloading the increment operator, it's not
 possible to duplicate the _syntax_ of STL-style iterators.  My
 response is - who cares?  I don't think of iterators as "pointers that
 can be incremented," and I doubt anyone who doesn't have a C++
 background thinks of them that way either.  It seems that the
 semantics of STL iterators can be implemented with methods just as
 well.


 There is a little something nice about being able to use a pointer and
 iterator interchangeably for templated container algorithms.  You lose
 that if ++iter can't be used to iterate your iterator.


Except that in D, you rarely use pointers, and ++ isn't defined on
arrays.  So you lose a lot of the unity that the syntax affords you in
C++.
Dec 27 2008
prev sibling parent reply Don <nospam nospam.com> writes:
Andrei Alexandrescu wrote:

 Bill Baxter wrote:

 On Sat, Dec 27, 2008 at 9:42 AM, The Anh Tran <trtheanh gmail.com> wrote:

 aarti_pl wrote:

 Andrei Alexandrescu pisze:
  > We're trying to make that work. D is due for an operator overhaul.
  >
  > Andrei

 Is there any chance that we get possibility to overload "raw 
 operators",
 like in C++? I think that they may coexist with currently defined 
 operator
 overloads with simple semantic rules, which will not allow them to work
 together at the same time.
 ..........
 BR
 Marcin Kuszczak
 (aarti_pl)


 Me also have a dream :D

 <Daydream mode>
 class Foo
 {
        auto op(++)(); // bar++
        auto op(++)(int); // ++bar

        op(cast)(uint); // cast(uint)bar // opCast
        auto op(())(int, float); // Foo(123, 123.456) // opCall

        auto op(+)(Foo rhs); // bar1 + bar2
        auto op(+=)(int); // bar += 1234;
        auto op(.)(); // bar.xyz // opDot

        Foo op([][][])(int, char, float); // bar[123]['x'][123.456]

        auto op([..])(); // i = bar2[] // opSlide
        auto op([..])(int, int); // bar[1..10]

        auto op([..]=)(float); // bar[] = 12.3 //opSlideAssign
        auto op([..]=)(int, int, float); // bar[1..3] = 123.4
 }
 </Dream>


 When I suggested this kind of thing long ago, Walter said that it
 encourages operator overload abuse, because it suggests that  + is
 just a generic symbolic operator rather than something that
 specifically means "addition".  That's why D uses "opAdd" instead.
 It's supposed to encourage only creating overloads that follow the
 original meaning of the operator closely.  That way when you see a+b
 you can be reasonably sure that it means addition or something quite
 like it.


 
 I think that argument is rather weak and ought to be revisited. It's 
 weak to start with as if writing "+" in a D program hardly evokes 
 anything else but "plus". What the notation effectively achieved was put 
 more burden on the programmer to memorize some names for the 
 already-known symbols. I think the entire operator overloading business, 
 which started from a legitimate desire to improve on C++'s, ended up 
 worse off.


I feel quite strongly that C++'s operator overloading was a failed 
experiment. The original intention (AFAIK) was to allow creation of 
mathematical entities which could use natural syntax. The classic 
example was complex numbers, and it works reasonably well for that, 
although it requires you to create an absurd number of repetitive functions.

But for anything much more complicated, such as matrices, tensors, big 
integer arithmetic, etc -- it's an abject failure. It's clumsy, and 
creates masses of temporary objects, which kills performance so 
completely that it's unusable. But the whole point of operator 
overloading was to allow nice notation in a performace-oriented 
language! Expression templates are basically a hack to restore 
performance in most cases, but it comes at a massive cost in simplicity.
And the performance even then is not always optimal.

I think that Walter's idea, in tightening the semantics of overloaded 
operators, is the right approach. Unfortunately, it doesn't go far 
enough, so we get the worst of both worlds: the C++ freedom is 
curtailed, but there isn't enough power to replace it.

Ultimately, I think that the problem is that ideally, '+' is not simply 
a call to a function called 'plus()'. What you'd like an operator to 
compile to, depends on the expression in which it is embedded. For 
maximum performance, an expression needs to be digested before it is 
converted into elementary functions.

In my 'operator overloading without temporaries' proposal in Bugzilla,
I showed that DEFINING a -= b as being identical to a = a - b, and then 
  creating a symmetric operation for a = b - a allows optimal code 
generation in a great many cases. It's not a complete solution, though.

In particular, irreducible temporaries need more thought. Ideally, in 
something like a += b * c + d, b*c would be created in a memory pool, 
and deleted at the end of the expression.
(By contrast, a = b*c+d, would translate to a=b*c; a+=d; so no temporary 
is required).

There are other, less serious problems which also need to be addressed.

Defining ++a as a+=1 is probably a mistake. It raises lots of nasty issues.
* If a is a complex number, a = a + 1 makes perfect sense. But it's not 
obvious that ++a is sensible.
* What type is '1'? Is it an int, a uint, a long, ... You don't have 
that issue with increment.

As I see it, there are two possible strategies:
(1) Pursuing optimal performance, which requires semantic tightening, 
and reduced flexibility, or
(2) Pursure simplicity and semantic flexibility, sacrificing performance.

I think those two possibilities are mutually exclusive.
Dec 27 2008
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Don wrote:

 Andrei Alexandrescu wrote:

 Bill Baxter wrote:

 On Sat, Dec 27, 2008 at 9:42 AM, The Anh Tran <trtheanh gmail.com> 
 wrote:

 aarti_pl wrote:

 Andrei Alexandrescu pisze:
  > We're trying to make that work. D is due for an operator overhaul.
  >
  > Andrei

 Is there any chance that we get possibility to overload "raw 
 operators",
 like in C++? I think that they may coexist with currently defined 
 operator
 overloads with simple semantic rules, which will not allow them to 
 work
 together at the same time.
 ..........
 BR
 Marcin Kuszczak
 (aarti_pl)


 Me also have a dream :D

 <Daydream mode>
 class Foo
 {
        auto op(++)(); // bar++
        auto op(++)(int); // ++bar

        op(cast)(uint); // cast(uint)bar // opCast
        auto op(())(int, float); // Foo(123, 123.456) // opCall

        auto op(+)(Foo rhs); // bar1 + bar2
        auto op(+=)(int); // bar += 1234;
        auto op(.)(); // bar.xyz // opDot

        Foo op([][][])(int, char, float); // bar[123]['x'][123.456]

        auto op([..])(); // i = bar2[] // opSlide
        auto op([..])(int, int); // bar[1..10]

        auto op([..]=)(float); // bar[] = 12.3 //opSlideAssign
        auto op([..]=)(int, int, float); // bar[1..3] = 123.4
 }
 </Dream>


 When I suggested this kind of thing long ago, Walter said that it
 encourages operator overload abuse, because it suggests that  + is
 just a generic symbolic operator rather than something that
 specifically means "addition".  That's why D uses "opAdd" instead.
 It's supposed to encourage only creating overloads that follow the
 original meaning of the operator closely.  That way when you see a+b
 you can be reasonably sure that it means addition or something quite
 like it.


 I think that argument is rather weak and ought to be revisited. It's 
 weak to start with as if writing "+" in a D program hardly evokes 
 anything else but "plus". What the notation effectively achieved was 
 put more burden on the programmer to memorize some names for the 
 already-known symbols. I think the entire operator overloading 
 business, which started from a legitimate desire to improve on C++'s, 
 ended up worse off.


 
 I feel quite strongly that C++'s operator overloading was a failed 
 experiment. The original intention (AFAIK) was to allow creation of 
 mathematical entities which could use natural syntax. The classic 
 example was complex numbers, and it works reasonably well for that, 
 although it requires you to create an absurd number of repetitive 
 functions.
 
 But for anything much more complicated, such as matrices, tensors, big 
 integer arithmetic, etc -- it's an abject failure. It's clumsy, and 
 creates masses of temporary objects, which kills performance so 
 completely that it's unusable. But the whole point of operator 
 overloading was to allow nice notation in a performace-oriented 
 language! Expression templates are basically a hack to restore 
 performance in most cases, but it comes at a massive cost in simplicity.
 And the performance even then is not always optimal.
 
 I think that Walter's idea, in tightening the semantics of overloaded 
 operators, is the right approach. Unfortunately, it doesn't go far 
 enough, so we get the worst of both worlds: the C++ freedom is 
 curtailed, but there isn't enough power to replace it.


Very well put.


 Ultimately, I think that the problem is that ideally, '+' is not simply 
 a call to a function called 'plus()'. What you'd like an operator to 
 compile to, depends on the expression in which it is embedded. For 
 maximum performance, an expression needs to be digested before it is 
 converted into elementary functions.
 
 In my 'operator overloading without temporaries' proposal in Bugzilla,
 I showed that DEFINING a -= b as being identical to a = a - b, and then 
  creating a symmetric operation for a = b - a allows optimal code 
 generation in a great many cases. It's not a complete solution, though.
 
 In particular, irreducible temporaries need more thought. Ideally, in 
 something like a += b * c + d, b*c would be created in a memory pool, 
 and deleted at the end of the expression.
 (By contrast, a = b*c+d, would translate to a=b*c; a+=d; so no temporary 
 is required).


That's an awesome proposal. I'd like to expand it to comprehend fusion 
as well. Consider:

A = B + C - D;

where the operands are matrices. The best hand-written implementation 
would loop once through the three matrices and assign to the destination 
element-wise A[i, j] = B[i, j] + C[i, j] - D[i, j]. However, with an 
approach that has only one operator application as its horizon, it is 
impossible to achieve that optimization. So I wonder what abstraction 
could be devised that makes it easy and natural to support such fusion. 
Expression templates achieve that by saving the right-hand expression 
tree as a type and then using it during the assignment. This requires a 
considerable effort and has some drawbacks.


 There are other, less serious problems which also need to be addressed.
 
 Defining ++a as a+=1 is probably a mistake. It raises lots of nasty issues.
 * If a is a complex number, a = a + 1 makes perfect sense. But it's not 
 obvious that ++a is sensible.
 * What type is '1'? Is it an int, a uint, a long, ... You don't have 
 that issue with increment.


Great points!


 As I see it, there are two possible strategies:
 (1) Pursuing optimal performance, which requires semantic tightening, 
 and reduced flexibility, or
 (2) Pursure simplicity and semantic flexibility, sacrificing performance.
 
 I think those two possibilities are mutually exclusive.


I tend to be more optimistic, but if asked to choose, I'd go for (1). 
One important lesson learned from C++'s operator overloading is that 
freedom was almost always badly used. Tellingly, whenever operator 
overloading is taught or talked about, the first caveat mentioned is 
that defining inconsistent batteries of operators is exceedingly easy.

Andrei
Dec 27 2008
prev sibling parent reply aarti_pl <aarti interia.pl> writes:
Don pisze:

 As I see it, there are two possible strategies:
 (1) Pursuing optimal performance, which requires semantic tightening, 
 and reduced flexibility, or
 (2) Pursure simplicity and semantic flexibility, sacrificing performance.
 
 I think those two possibilities are mutually exclusive.


Well, I think you have just discovered two levels of programmers 
interests in operator overloading :-)

* First level (group) is for programmers which want to use operators to 
create fast mathematical operations for user defined types.

* Second level (group) is for programmers which want to use operators 
for creating better syntax, to make their users more happy.

It seems that both groups have good reasons for their requests.

As you said these two possibilities are mutually exclusive, so I think 
also solution for this challenge should be splitted, so that everyone 
could get what he want.

One possible solution is to allow defining operator overloading on two 
levels also:

1. In first level, where operators are used for computation, there might 
be operators like today in D: opCmp, opEquals. They can be even stricter 
and/or better defined as needed.

2. Second level should be just raw operator access, which allows to use 
operator syntax in wider scope of cases.

Compiler should enforce the rule that operators from two levels can not 
be intermixed in one class/struct.

I think that such a solution could make these two groups have their 
goals achieved.

BR
Marcin Kuszczak
(aarti_pl)
Dec 27 2008
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
aarti_pl wrote:

 Don pisze:

 As I see it, there are two possible strategies:
 (1) Pursuing optimal performance, which requires semantic tightening, 
 and reduced flexibility, or
 (2) Pursure simplicity and semantic flexibility, sacrificing performance.

 I think those two possibilities are mutually exclusive.


 
 Well, I think you have just discovered two levels of programmers 
 interests in operator overloading :-)
 
 * First level (group) is for programmers which want to use operators to 
 create fast mathematical operations for user defined types.
 
 * Second level (group) is for programmers which want to use operators 
 for creating better syntax, to make their users more happy.
 
 It seems that both groups have good reasons for their requests.
 
 As you said these two possibilities are mutually exclusive, so I think 
 also solution for this challenge should be splitted, so that everyone 
 could get what he want.
 
 One possible solution is to allow defining operator overloading on two 
 levels also:
 
 1. In first level, where operators are used for computation, there might 
 be operators like today in D: opCmp, opEquals. They can be even stricter 
 and/or better defined as needed.
 
 2. Second level should be just raw operator access, which allows to use 
 operator syntax in wider scope of cases.
 
 Compiler should enforce the rule that operators from two levels can not 
 be intermixed in one class/struct.
 
 I think that such a solution could make these two groups have their 
 goals achieved.
 
 BR
 Marcin Kuszczak
 (aarti_pl)


That can be achieved by only defining the low-level operators in the 
language and then developing a library layer on top of them.

Andrei
Dec 27 2008
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
The Anh Tran wrote:

 aarti_pl wrote:

 Andrei Alexandrescu pisze:
  > We're trying to make that work. D is due for an operator overhaul.
  >
  > Andrei

 Is there any chance that we get possibility to overload "raw 
 operators", like in C++? I think that they may coexist with currently 
 defined operator overloads with simple semantic rules, which will not 
 allow them to work together at the same time.
 ..........
 BR
 Marcin Kuszczak
 (aarti_pl)


 
 Me also have a dream :D
 
 <Daydream mode>
 class Foo
 {
     auto op(++)(); // bar++
     auto op(++)(int); // ++bar


Hey, wasn't the implementation of the postincrement operator through an 
overload a rather untasty hack?


     op(cast)(uint); // cast(uint)bar // opCast
     auto op(())(int, float); // Foo(123, 123.456) // opCall
     
     auto op(+)(Foo rhs); // bar1 + bar2
     auto op(+=)(int); // bar += 1234;
     auto op(.)(); // bar.xyz // opDot
 
     Foo op([][][])(int, char, float); // bar[123]['x'][123.456]
 
     auto op([..])(); // i = bar2[] // opSlide
     auto op([..])(int, int); // bar[1..10]
 
     auto op([..]=)(float); // bar[] = 12.3 //opSlideAssign
     auto op([..]=)(int, int, float); // bar[1..3] = 123.4
 }
 </Dream>


Aside for a minor change in notation, there's no improvement. We're 
looking for much more broad improvements, such as offering the ability 
to overload several operators with only one function.

Andrei
Dec 26 2008
parent reply The Anh Tran <trtheanh gmail.com> writes:
Andrei Alexandrescu wrote:

     auto op(++)(); // bar++
     auto op(++)(int); // ++bar


 
 Hey, wasn't the implementation of the postincrement operator through an 
 overload a rather untasty hack?



 Aside for a minor change in notation, there's no improvement. We're 
 looking for much more broad improvements, such as offering the ability 
 to overload several operators with only one function.
 
 Andrei


How about this:

Unary op: ++f, ~f, ~f, +f, -f, *f
auto operator(++, --, ~, !, +, -, *)()
{
	// posfix is provided by compiler
	// and is only used for: foo++ foo--
	static if (posfix)
		return op(this.value);
	else ...
}

Binary op: equality comparison f1 <= f2
bool operator(<, >, <=, >=, ==, !=)(Foo foo)
{
	return op(this.value, foo.value);
}

For un-order object, he/she just list 'correct' operator(s) in op() 
list. Ex: bool operator(!=, ==)(Foo foo) {}


Binary op: logic f1 || f2 && f3
bool operator(&&, ||, &, |, &)(Op[] oops, Foo[] foo)
{
	auto result = foo[0].value;
	foreach (i, op ; oops)
		result = op( result, foo[i+1].value );
	return result;
}


Binary op: bitwise
auto operator(&, |, ^)(Op[], Foo[]) {}


Binary op: add, sub, mul, div, shift, cat(~) f1 + f2 * f3 ~ f4
auto operator(+, -, *, /, %, <<, >>, ~, =)(Op[] oops, Foo[] foo()
{	
	auto result = foo[0].value;
	foreach (i, op ; oops)
		result = op( result, foo[i+1].value );
	return result;
}
Op[], Foo[] should/may be re-arranged according to operator precedence???


Binary op: +=, -=, *=, <<=, ~= f1 *= f2 + f3 / f4
I 'think' the compiler can evaluate to: f1 = f1 * (f2 + f3 / f4);
Dec 27 2008
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
The Anh Tran wrote:

 Andrei Alexandrescu wrote:

     auto op(++)(); // bar++
     auto op(++)(int); // ++bar


 Hey, wasn't the implementation of the postincrement operator through 
 an overload a rather untasty hack?


 

 Aside for a minor change in notation, there's no improvement. We're 
 looking for much more broad improvements, such as offering the ability 
 to overload several operators with only one function.

 Andrei


 
 How about this:
 
 Unary op: ++f, ~f, ~f, +f, -f, *f
 auto operator(++, --, ~, !, +, -, *)()
 {
     // posfix is provided by compiler
     // and is only used for: foo++ foo--
     static if (posfix)
         return op(this.value);
     else ...
 }
 
 Binary op: equality comparison f1 <= f2
 bool operator(<, >, <=, >=, ==, !=)(Foo foo)
 {
     return op(this.value, foo.value);
 }
 
 For un-order object, he/she just list 'correct' operator(s) in op() 
 list. Ex: bool operator(!=, ==)(Foo foo) {}


Why invent new syntax when compile-time strings are already there?

auto operator(string op)() if (op == "++" || op == "--")
{
     return mixin(op ~ "this.value");
}

etc. That, of course, is orthogonal to the semantic equivalences 
suggested by Don and does not solve fusion.

Andrei
Dec 27 2008
parent reply John Reimer <terminal.node gmail.com> writes:
Hello Andrei,


 The Anh Tran wrote:
 

 Andrei Alexandrescu wrote:
 

 auto op(++)(); // bar++
 auto op(++)(int); // ++bar


 Hey, wasn't the implementation of the postincrement operator through
 an overload a rather untasty hack?
 
 Aside for a minor change in notation, there's no improvement. We're
 looking for much more broad improvements, such as offering the
 ability to overload several operators with only one function.
 
 Andrei
 


 How about this:
 
 Unary op: ++f, ~f, ~f, +f, -f, *f
 auto operator(++, --, ~, !, +, -, *)()
 {
 // posfix is provided by compiler
 // and is only used for: foo++ foo--
 static if (posfix)
 return op(this.value);
 else ...
 }
 Binary op: equality comparison f1 <= f2
 bool operator(<, >, <=, >=, ==, !=)(Foo foo)
 {
 return op(this.value, foo.value);
 }
 For un-order object, he/she just list 'correct' operator(s) in op()
 list. Ex: bool operator(!=, ==)(Foo foo) {}
 


 Why invent new syntax when compile-time strings are already there?
 
 auto operator(string op)() if (op == "++" || op == "--")
 {
 return mixin(op ~ "this.value");
 }
 etc. That, of course, is orthogonal to the semantic equivalences
 suggested by Don and does not solve fusion.
 
 Andrei
 



Once again, I'm no expert in this matter, but the compiler-time strings idea 
looks like a great solution for this.  Was this brought up before? ...because 
that was kind of what I was thinking when you mentioned a new operator
overloading 
syntax for D.

-JJR
Dec 27 2008
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
John Reimer wrote:

 Hello Andrei,
 

 The Anh Tran wrote:


 Andrei Alexandrescu wrote:


 auto op(++)(); // bar++
 auto op(++)(int); // ++bar


 Hey, wasn't the implementation of the postincrement operator through
 an overload a rather untasty hack?

 Aside for a minor change in notation, there's no improvement. We're
 looking for much more broad improvements, such as offering the
 ability to overload several operators with only one function.

 Andrei


 How about this:

 Unary op: ++f, ~f, ~f, +f, -f, *f
 auto operator(++, --, ~, !, +, -, *)()
 {
 // posfix is provided by compiler
 // and is only used for: foo++ foo--
 static if (posfix)
 return op(this.value);
 else ...
 }
 Binary op: equality comparison f1 <= f2
 bool operator(<, >, <=, >=, ==, !=)(Foo foo)
 {
 return op(this.value, foo.value);
 }
 For un-order object, he/she just list 'correct' operator(s) in op()
 list. Ex: bool operator(!=, ==)(Foo foo) {}


 Why invent new syntax when compile-time strings are already there?

 auto operator(string op)() if (op == "++" || op == "--")
 {
 return mixin(op ~ "this.value");
 }
 etc. That, of course, is orthogonal to the semantic equivalences
 suggested by Don and does not solve fusion.

 Andrei


 
 
 Once again, I'm no expert in this matter, but the compiler-time strings 
 idea looks like a great solution for this.  Was this brought up before? 
 ...because that was kind of what I was thinking when you mentioned a new 
 operator overloading syntax for D.


Only in private conversation between Walter and me. But anyway, let's 
not forget we need to address fusion as well.

Andrei
Dec 27 2008
prev sibling parent Tomasz Sowinski <tomeksowi i.am.on.gmail> writes:
Andrei Alexandrescu Wrote:


 [cut], such as offering the ability 
 to overload several operators with only one function.


Some time ago I tried to make operator overloading more generic. You can still
read it on Zoho:

http://export.writer.zoho.com/public/tomeksowi/Operator-Overloading
Dec 27 2008