www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Short list with things to finish for D2

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
We're entering the finale of D2 and I want to keep a short list of 
things that must be done and integrated in the release. It is clearly 
understood by all of us that there are many things that could and 
probably should be done.

1. Currently Walter and Don are diligently fixing the problems marked on 
the current manuscript.

2. User-defined operators must be revamped. Fortunately Don already put 
in an important piece of functionality (opDollar). What we're looking at 
is a two-pronged attack motivated by Don's proposal:

http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7

The two prongs are:

* Encode operators by compile-time strings. For example, instead of the 
plethora of opAdd, opMul, ..., we'd have this:

T opBinary(string op)(T rhs) { ... }

The string is "+", "*", etc. We need to design what happens with 
read-modify-write operators like "+=" (should they be dispatch to a 
different function? etc.) and also what happens with index-and-modify 
operators like "[]=", "[]+=" etc. Should we go with proxies? Absorb them 
in opBinary? Define another dedicated method? etc.

* Loop fusion that generalizes array-wise operations. This idea of 
Walter is, I think, very good because it generalizes and democratizes 
"magic". The idea is that, if you do

a = b + c;

and b + c does not make sense but b and c are ranges for which a.front = 
b.front + c.front does make sense, to automatically add the iteration 
paraphernalia.

3. It was mentioned in this group that if getopt() does not work in 
SafeD, then SafeD may as well pack and go home. I agree. We need to make 
it work. Three ideas discussed with Walter:

* Allow taking addresses of locals, but in that case switch allocation 
from stack to heap, just like with delegates. If we only do that in 
SafeD, behavior will be different than with regular D. In any case, it's 
an inefficient proposition, particularly for getopt() which actually 
does not need to escape the addresses - just fills them up.

* Allow  trusted (and maybe even  safe) functions to receive addresses 
of locals. Statically check that they never escape an address of a 
parameter. I think this is very interesting because it enlarges the 
common ground of D and SafeD.

* Figure out a way to reconcile "ref" with variadics. This is the actual 
reason why getopt chose to traffic in addresses, and fixing it is the 
logical choice and my personal favorite.

4. Allow private members inside a template using the eponymous trick:

template wyda(int x) {
    private enum geeba = x / 2;
    alias geeba wyda;
}

The names inside an eponymous template are only accessible to the 
current instantiation. For example, wyda!5 cannot access wyda!(4).geeba, 
only its own geeba. That we we elegantly avoid the issue "where is this 
symbol looked up?"

5. Chain exceptions instead of having a recurrent exception terminate 
the program. I'll dedicate a separate post to this.

6. There must be many things I forgot to mention, or that cause grief to 
many of us. Please add to/comment on this list.



Andrei
Nov 18 2009
next sibling parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
What about the business with signed/unsigned integer comparisons?
Nov 18 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Ellery Newcomer wrote:
 What about the business with signed/unsigned integer comparisons?
I think that's somewhere in the TDPL non-working snippets, but I added your message to my worklist to make sure. Andrei
Nov 18 2009
parent Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
Andrei Alexandrescu wrote:
 Ellery Newcomer wrote:
 What about the business with signed/unsigned integer comparisons?
I think that's somewhere in the TDPL non-working snippets, but I added your message to my worklist to make sure. Andrei
Cool, thanks
Nov 18 2009
prev sibling next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Andrei Alexandrescu:

 * Encode operators by compile-time strings. For example, instead of the 
 plethora of opAdd, opMul, ..., we'd have this:
 
 T opBinary(string op)(T rhs) { ... }
 
 The string is "+", "*", etc.
Can you show an example of defining an operator, like a minus, with that? In my set data structure I'd like to define "<=" among two sets as "is subset". Can that design allow me to overload just <= and >= ? (opCmp is not enough here). Bye, bearophile
Nov 18 2009
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
bearophile wrote:
 Andrei Alexandrescu:
 
 * Encode operators by compile-time strings. For example, instead of
 the plethora of opAdd, opMul, ..., we'd have this:
 
 T opBinary(string op)(T rhs) { ... }
 
 The string is "+", "*", etc.
Can you show an example of defining an operator, like a minus, with that?
struct BigInt { BigInt opBinary(string op)(BigInt rhs) if (op == "-") { ... } }
 In my set data structure I'd like to define "<=" among two sets as
 "is subset". Can that design allow me to overload just <= and >= ?
 (opCmp is not enough here).
It could if we decide to deprecate opCmp. I happen to like it; if you define a <= b for inclusion, people will think it's natural to also allow a < b for strict inclusion. But that's up for debate. I'm not sure what the best way is. Classes have opEquals and opCmp so the question is - do we want structs to be somewhat compatible with classes or not? My personal favorite choice would be to go full bore with compile-time strings. Andrei
Nov 18 2009
prev sibling next sibling parent reply Travis Boucher <boucher.travis gmail.com> writes:
bearophile wrote:
 Andrei Alexandrescu:
 
 * Encode operators by compile-time strings. For example, instead of the 
 plethora of opAdd, opMul, ..., we'd have this:

 T opBinary(string op)(T rhs) { ... }

 The string is "+", "*", etc.
Can you show an example of defining an operator, like a minus, with that?
T opBinary(string op)(T rhs) { static if (op == "-") return data - rhs.data; static if (op == "+") return data + rhs.data; // ... maybe this would work too ... mixin("return data " ~ op ~ "rhs.data;"); } I love this syntax over the tons of different operation functions. Makes it so much nicer, especially when supporting a bunch of different paramater types (vectors are a good example of this). T opBinary(string op)(T rhs) T opBinary(string op)(float[3] rhs) T opBinary(string op)(float rx, ry, rz) ....
 In my set data structure I'd like to define "<=" among two sets as "is
subset". Can that design allow me to overload just <= and >= ? (opCmp is not
enough here).
 
 Bye,
 bearophile
Nov 18 2009
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Travis Boucher wrote:
 bearophile wrote:
 Andrei Alexandrescu:

 * Encode operators by compile-time strings. For example, instead of 
 the plethora of opAdd, opMul, ..., we'd have this:

 T opBinary(string op)(T rhs) { ... }

 The string is "+", "*", etc.
Can you show an example of defining an operator, like a minus, with that?
T opBinary(string op)(T rhs) { static if (op == "-") return data - rhs.data; static if (op == "+") return data + rhs.data; // ... maybe this would work too ... mixin("return data " ~ op ~ "rhs.data;"); } I love this syntax over the tons of different operation functions. Makes it so much nicer, especially when supporting a bunch of different paramater types (vectors are a good example of this).
Indeed, the advantage of this is that you can use string mixins to implement many operations at once, instead of laboriously defining many functions. Andrei
Nov 18 2009
prev sibling next sibling parent reply grauzone <none example.net> writes:
bearophile wrote:
 Andrei Alexandrescu:
 
 * Encode operators by compile-time strings. For example, instead of the 
 plethora of opAdd, opMul, ..., we'd have this:

 T opBinary(string op)(T rhs) { ... }

 The string is "+", "*", etc.
I thought the problem with this was that the lexer/parser would have to know about semantics, which is against the goals of the language. Would the operator actually be inside quotes? Anyway, do we _really_ want to make it possible, that valid D code will look like ASCII art?
Nov 18 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
grauzone wrote:
 bearophile wrote:
 Andrei Alexandrescu:

 * Encode operators by compile-time strings. For example, instead of 
 the plethora of opAdd, opMul, ..., we'd have this:

 T opBinary(string op)(T rhs) { ... }

 The string is "+", "*", etc.
I thought the problem with this was that the lexer/parser would have to know about semantics, which is against the goals of the language. Would the operator actually be inside quotes?
It's simpler than that. a + b will be rewritten into a.opBinary!("+")(b) The rewrite is done long after lexing, so no low-level problems there.
 Anyway, do we _really_ want to make it possible, that valid D code will 
 look like ASCII art?
We want to improve the state of the art. If the attempt risks to end up doing the opposite, I'm all ears. Andrei
Nov 18 2009
parent reply grauzone <none example.net> writes:
Andrei Alexandrescu wrote:
 The rewrite is done long after lexing, so no low-level problems there.
Oh, I thought it would let you introduce new operators. But it's only about the existing ones. I find the idea to identify the operator using a string very sloppy and sillyl just like using string mixins for small delegates in std.algorithm etc.; but you'd probably say "it works and is useful" and "it's short" and "it solves the current problem", so... whatever.
Nov 18 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
grauzone wrote:
 Andrei Alexandrescu wrote:
 The rewrite is done long after lexing, so no low-level problems there.
Oh, I thought it would let you introduce new operators. But it's only about the existing ones. I find the idea to identify the operator using a string very sloppy and sillyl just like using string mixins for small delegates in std.algorithm etc.; but you'd probably say "it works and is useful" and "it's short" and "it solves the current problem", so... whatever.
We're trying to improve on the current situation, which forces the user to manually define a lot of small functions. If you have convincing reasons to argue that the current state of affairs is actually better, I'm all ears - both Walter and I could use less work, particularly if the outcome sucks (see e.g. T[new]). Also, if you have ideas on how things could be done in a way that you'd find not sloppy and not silly, that would be even better. Andrei
Nov 18 2009
next sibling parent reply grauzone <none example.net> writes:
Andrei Alexandrescu wrote:
 grauzone wrote:
 Andrei Alexandrescu wrote:
 The rewrite is done long after lexing, so no low-level problems there.
Oh, I thought it would let you introduce new operators. But it's only about the existing ones. I find the idea to identify the operator using a string very sloppy and sillyl just like using string mixins for small delegates in std.algorithm etc.; but you'd probably say "it works and is useful" and "it's short" and "it solves the current problem", so... whatever.
We're trying to improve on the current situation, which forces the user to manually define a lot of small functions. If you have convincing reasons to argue that the current state of affairs is actually better, I'm all ears - both Walter and I could use less work, particularly if the outcome sucks (see e.g. T[new]). Also, if you have ideas on how things could be done in a way that you'd find not sloppy and not silly, that would be even better.
If I had a better proposal, I'd post it. I'm just saying that's it's a bad hack, that _although_ solves the problem, will have negative side effects for other reasons. Does the current proposal make things simpler at all? All you're doing is to enable the programmer to "fix" the clumsy semantics by throwing lots of CTFE onto the problem. Why not generate the operator functions with CTFE in the first place...
Nov 18 2009
next sibling parent reply Travis Boucher <boucher.travis gmail.com> writes:
grauzone wrote:
 
 If I had a better proposal, I'd post it. I'm just saying that's it's a 
 bad hack, that _although_ solves the problem, will have negative side 
 effects for other reasons.
 
 Does the current proposal make things simpler at all? All you're doing 
 is to enable the programmer to "fix" the clumsy semantics by throwing 
 lots of CTFE onto the problem. Why not generate the operator functions 
 with CTFE in the first place...
From my point of view (trying different ways of implementing something as simple as a vector), this makes things much simpler without sacrificing functionality. Personally, I'd love to see an unknownMethod(string method)(...) thing implemented as well, but that might be asking for too much (and might sacrifice performance in some cases).
Nov 18 2009
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Travis Boucher wrote:
 grauzone wrote:
 If I had a better proposal, I'd post it. I'm just saying that's it's a 
 bad hack, that _although_ solves the problem, will have negative side 
 effects for other reasons.

 Does the current proposal make things simpler at all? All you're doing 
 is to enable the programmer to "fix" the clumsy semantics by throwing 
 lots of CTFE onto the problem. Why not generate the operator functions 
 with CTFE in the first place...
From my point of view (trying different ways of implementing something as simple as a vector), this makes things much simpler without sacrificing functionality. Personally, I'd love to see an unknownMethod(string method)(...) thing implemented as well, but that might be asking for too much (and might sacrifice performance in some cases).
Oh, that needs to be on the list! Thanks for reminding me, I added it to my list. Andrei
Nov 18 2009
prev sibling next sibling parent reply Leandro Lucarella <llucax gmail.com> writes:
grauzone, el 19 de noviembre a las 03:47 me escribiste:
 Does the current proposal make things simpler at all? All you're
 doing is to enable the programmer to "fix" the clumsy semantics by
 throwing lots of CTFE onto the problem. Why not generate the
 operator functions with CTFE in the first place...
I was about to say that, the solution is a hack. I could understand a hack if there were no other way to do it, but you can generate the code for the opXxx using CTFE/string mixins already: we already have a hackish solution. I don't think adding a new hack would be nice (specially when it will be a big change). Maybe a not-so-hackish solution can be found when AST macros get implemented. -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) ---------------------------------------------------------------------- Borrowing money from a friend is like having sex. It just completely changes the relationship. -- George Constanza
Nov 18 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Leandro Lucarella wrote:
 grauzone, el 19 de noviembre a las 03:47 me escribiste:
 Does the current proposal make things simpler at all? All you're
 doing is to enable the programmer to "fix" the clumsy semantics by
 throwing lots of CTFE onto the problem. Why not generate the
 operator functions with CTFE in the first place...
I was about to say that, the solution is a hack. I could understand a hack if there were no other way to do it, but you can generate the code for the opXxx using CTFE/string mixins already: we already have a hackish solution. I don't think adding a new hack would be nice (specially when it will be a big change). Maybe a not-so-hackish solution can be found when AST macros get implemented.
I am thinking that representing operators by their exact token representation is a principled approach because it allows for unambiguous mapping, testing with if and static if, and also allows saving source code by using only one string mixin. It would take more than just a statement that it's hackish to convince me it's hackish. I currently don't see the hackishness of the approach, and I consider it a vast improvement over the current state of affairs. I'd be grateful if you argued your point further and hopefully suggested an approach that is better. I want us to move fast with this. So it's just the right time to contribute. Andrei
Nov 18 2009
next sibling parent reply "Robert Jacques" <sandford jhu.edu> writes:
On Wed, 18 Nov 2009 23:33:54 -0500, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 Leandro Lucarella wrote:
 grauzone, el 19 de noviembre a las 03:47 me escribiste:
 Does the current proposal make things simpler at all? All you're
 doing is to enable the programmer to "fix" the clumsy semantics by
 throwing lots of CTFE onto the problem. Why not generate the
 operator functions with CTFE in the first place...
I was about to say that, the solution is a hack. I could understand a hack if there were no other way to do it, but you can generate the code for the opXxx using CTFE/string mixins already: we already have a hackish solution. I don't think adding a new hack would be nice (specially when it will be a big change). Maybe a not-so-hackish solution can be found when AST macros get implemented.
I am thinking that representing operators by their exact token representation is a principled approach because it allows for unambiguous mapping, testing with if and static if, and also allows saving source code by using only one string mixin. It would take more than just a statement that it's hackish to convince me it's hackish. I currently don't see the hackishness of the approach, and I consider it a vast improvement over the current state of affairs. I'd be grateful if you argued your point further and hopefully suggested an approach that is better. I want us to move fast with this. So it's just the right time to contribute. Andrei
For what it's worth, I used aliases of a template binary op function in order to do all the operator overloads of a small vec class. So I like your solution.
Nov 18 2009
parent Bill Baxter <wbaxter gmail.com> writes:
On Wed, Nov 18, 2009 at 8:55 PM, Robert Jacques <sandford jhu.edu> wrote:

 For what it's worth, I used aliases of a template binary op function in
 order to do all the operator overloads of a small vec class. So I like your
 solution.
Yeh, I've done something like that before too. So now instead of funnelling all kinds of opBlah functions into one template/mixin/whatever the compiler will just eliminate the middle man and take us right there to the template. I like it. --bb
Nov 19 2009
prev sibling next sibling parent reply Rainer Deyke <rainerd eldwood.com> writes:
Andrei Alexandrescu wrote:
 I am thinking that representing operators by their exact token
 representation is a principled approach because it allows for
 unambiguous mapping, testing with if and static if, and also allows
 saving source code by using only one string mixin. It would take more
 than just a statement that it's hackish to convince me it's hackish. I
 currently don't see the hackishness of the approach, and I consider it a
 vast improvement over the current state of affairs.
Isn't opBinary just a reduced-functionality version of opUnknownMethod (or whatever that is/was going to be called)? T opBinary(string op)(T rhs) { static if (op == "+") return data + rhs.data; else static if (op == "-") return data - rhs.data; ... else static assert(0, "Operator "~op~" not implemented"); } T opUnknownMethod(string op)(T rhs) { static if (op == "opAdd") return data + rhs.data; else static if (op == "opSub") return data - rhs.data; ... else static assert(0, "Method "~op~" not implemented"); } I'd much rather have opUnknownMethod than opBinary. If if I have opUnknownMethod, then opBinary becomes redundant. -- Rainer Deyke - rainerd eldwood.com
Nov 18 2009
next sibling parent reply Travis Boucher <boucher.travis gmail.com> writes:
Rainer Deyke wrote:
 Andrei Alexandrescu wrote:
 I am thinking that representing operators by their exact token
 representation is a principled approach because it allows for
 unambiguous mapping, testing with if and static if, and also allows
 saving source code by using only one string mixin. It would take more
 than just a statement that it's hackish to convince me it's hackish. I
 currently don't see the hackishness of the approach, and I consider it a
 vast improvement over the current state of affairs.
Isn't opBinary just a reduced-functionality version of opUnknownMethod (or whatever that is/was going to be called)? T opBinary(string op)(T rhs) { static if (op == "+") return data + rhs.data; else static if (op == "-") return data - rhs.data; ... else static assert(0, "Operator "~op~" not implemented"); } T opUnknownMethod(string op)(T rhs) { static if (op == "opAdd") return data + rhs.data; else static if (op == "opSub") return data - rhs.data; ... else static assert(0, "Method "~op~" not implemented"); } I'd much rather have opUnknownMethod than opBinary. If if I have opUnknownMethod, then opBinary becomes redundant.
Passing op as the symbol allows for mixin("this.data"~op~"that.data;"); What I was hoping for was a catch-all for unknown non-operation methods, which could allow for dispatching to functions that are not even known at runtime (ie. trigger a lookup in a shared object, or pass along to a scripting language engine).
Nov 18 2009
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Travis Boucher wrote:
 Rainer Deyke wrote:
 Andrei Alexandrescu wrote:
 I am thinking that representing operators by their exact token
 representation is a principled approach because it allows for
 unambiguous mapping, testing with if and static if, and also allows
 saving source code by using only one string mixin. It would take more
 than just a statement that it's hackish to convince me it's hackish. I
 currently don't see the hackishness of the approach, and I consider it a
 vast improvement over the current state of affairs.
Isn't opBinary just a reduced-functionality version of opUnknownMethod (or whatever that is/was going to be called)? T opBinary(string op)(T rhs) { static if (op == "+") return data + rhs.data; else static if (op == "-") return data - rhs.data; ... else static assert(0, "Operator "~op~" not implemented"); } T opUnknownMethod(string op)(T rhs) { static if (op == "opAdd") return data + rhs.data; else static if (op == "opSub") return data - rhs.data; ... else static assert(0, "Method "~op~" not implemented"); } I'd much rather have opUnknownMethod than opBinary. If if I have opUnknownMethod, then opBinary becomes redundant.
Passing op as the symbol allows for mixin("this.data"~op~"that.data;");
Also it does not require remembering the correspondence between symbols and their names.
 What I was hoping for was a catch-all for unknown non-operation methods, 
 which could allow for dispatching to functions that are not even known 
 at runtime (ie. trigger a lookup in a shared object, or pass along to a 
 scripting language engine).
 
I agree. This is something very interesting. Andrei
Nov 18 2009
prev sibling next sibling parent Bill Baxter <wbaxter gmail.com> writes:
On Wed, Nov 18, 2009 at 10:55 PM, Rainer Deyke <rainerd eldwood.com> wrote:
 Andrei Alexandrescu wrote:
 I am thinking that representing operators by their exact token
 representation is a principled approach because it allows for
 unambiguous mapping, testing with if and static if, and also allows
 saving source code by using only one string mixin. It would take more
 than just a statement that it's hackish to convince me it's hackish. I
 currently don't see the hackishness of the approach, and I consider it a
 vast improvement over the current state of affairs.
Isn't opBinary just a reduced-functionality version of opUnknownMethod (or whatever that is/was going to be called)? T opBinary(string op)(T rhs) { =A0 =A0static if (op =3D=3D "+") return data + rhs.data; =A0 =A0else static if (op =3D=3D "-") return data - rhs.data; =A0 =A0... =A0 =A0else static assert(0, "Operator "~op~" not implemented"); } T opUnknownMethod(string op)(T rhs) { =A0 =A0static if (op =3D=3D "opAdd") return data + rhs.data; =A0 =A0else static if (op =3D=3D "opSub") return data - rhs.data; =A0 =A0... =A0 =A0else static assert(0, "Method "~op~" not implemented"); } I'd much rather have opUnknownMethod than opBinary. =A0If if I have opUnknownMethod, then opBinary becomes redundant.
opBinary doesn't really become redundant because you still need the _r vari= ants. You could come up with some convention for the string that represents the _r flavor of opUnknownMethod!("-") but why not just make'em different methods? I don't think there's enough in common between a typical operator and a generic unknownMethod to make fusing them worth it. --bb
Nov 19 2009
prev sibling parent =?ISO-8859-1?Q?Pelle_M=E5nsson?= <pelle.mansson gmail.com> writes:
Rainer Deyke wrote:
 Andrei Alexandrescu wrote:
 I am thinking that representing operators by their exact token
 representation is a principled approach because it allows for
 unambiguous mapping, testing with if and static if, and also allows
 saving source code by using only one string mixin. It would take more
 than just a statement that it's hackish to convince me it's hackish. I
 currently don't see the hackishness of the approach, and I consider it a
 vast improvement over the current state of affairs.
Isn't opBinary just a reduced-functionality version of opUnknownMethod (or whatever that is/was going to be called)? T opBinary(string op)(T rhs) { static if (op == "+") return data + rhs.data; else static if (op == "-") return data - rhs.data; ... else static assert(0, "Operator "~op~" not implemented"); } T opUnknownMethod(string op)(T rhs) { static if (op == "opAdd") return data + rhs.data; else static if (op == "opSub") return data - rhs.data; ... else static assert(0, "Method "~op~" not implemented"); } I'd much rather have opUnknownMethod than opBinary. If if I have opUnknownMethod, then opBinary becomes redundant.
Shouldn't you use opUnknownMethod for, you know, unknown methods? Implementing binary operators with an unknown method method seems unclean.
Nov 20 2009
prev sibling next sibling parent reply Leandro Lucarella <llucax gmail.com> writes:
Andrei Alexandrescu, el 18 de noviembre a las 20:33 me escribiste:
 Leandro Lucarella wrote:
grauzone, el 19 de noviembre a las 03:47 me escribiste:
Does the current proposal make things simpler at all? All you're
doing is to enable the programmer to "fix" the clumsy semantics by
throwing lots of CTFE onto the problem. Why not generate the
operator functions with CTFE in the first place...
I was about to say that, the solution is a hack. I could understand a hack if there were no other way to do it, but you can generate the code for the opXxx using CTFE/string mixins already: we already have a hackish solution. I don't think adding a new hack would be nice (specially when it will be a big change). Maybe a not-so-hackish solution can be found when AST macros get implemented.
I am thinking that representing operators by their exact token representation is a principled approach because it allows for unambiguous mapping, testing with if and static if, and also allows saving source code by using only one string mixin. It would take more than just a statement that it's hackish to convince me it's hackish. I currently don't see the hackishness of the approach, and I consider it a vast improvement over the current state of affairs. I'd be grateful if you argued your point further and hopefully suggested an approach that is better. I want us to move fast with this. So it's just the right time to contribute.
What I found hackish about it is that the code is a string manipulation mess. You can already do a string manipulation mess to programatically implement all the operator overloading. About the ideal solution, I don't have it. I just have the impression that AST macros can help here, but it's been ages since they were postponed to D3 and I don't have any idea of what they would look like, so I can't propose a solution now. But what the hell, maybe it's not so bad to have a better solution now, even when not ideal. -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) ---------------------------------------------------------------------- Los sueños de los niños son especialmente importantes en su etapa de formación; si un niño no sueña es que será un pelotudo toda la vida. -- Ricardo Vaporeso
Nov 19 2009
parent reply Bill Baxter <wbaxter gmail.com> writes:
On Thu, Nov 19, 2009 at 12:47 PM, Leandro Lucarella <llucax gmail.com> wrote:
 Andrei Alexandrescu, el 18 de noviembre a las 20:33 me escribiste:
 Leandro Lucarella wrote:
grauzone, el 19 de noviembre a las 03:47 me escribiste:
Does the current proposal make things simpler at all? All you're
doing is to enable the programmer to "fix" the clumsy semantics by
throwing lots of CTFE onto the problem. Why not generate the
operator functions with CTFE in the first place...
I was about to say that, the solution is a hack. I could understand a hack if there were no other way to do it, but you can generate the code for the opXxx using CTFE/string mixins already: we already have a hackish solution. I don't think adding a new hack would be nice (specially when it will be a big change). Maybe a not-so-hackish solution can be found when AST macros get implemented.
I am thinking that representing operators by their exact token representation is a principled approach because it allows for unambiguous mapping, testing with if and static if, and also allows saving source code by using only one string mixin. It would take more than just a statement that it's hackish to convince me it's hackish. I currently don't see the hackishness of the approach, and I consider it a vast improvement over the current state of affairs. I'd be grateful if you argued your point further and hopefully suggested an approach that is better. I want us to move fast with this. So it's just the right time to contribute.
What I found hackish about it is that the code is a string manipulation mess. You can already do a string manipulation mess to programatically implement all the operator overloading.
This is true, but if you leave it entirely up to the programmer and string mixins, the mess is much more messy. You're going to end up with code like this: mixin(genBinaryOp("+", q{MyType}, q{MyType rhs}, q{ return this.impl + rhs.impl; })); instead of this: MyType opBinary(string op : "+")(MyType rhs) { return this.impl + rhs.impl; } I would have a hard time defending the former as the recommended D style. But the latter is not so bad. It looks like a regular template declaration, and code is code, not a string.
 About the ideal solution, I don't have it. I just have the impression that
 AST macros can help here, but it's been ages since they were postponed to
 D3 and I don't have any idea of what they would look like, so I can't
 propose a solution now.
Yeh, I think macros are the best solution. If we have macros then the declarations could look like genBinaryOp(+, MyType, MyType rhs, return this.impl + rhs.impl; ) And with definable syntax, it could maybe become MyType opBinary "+" (MyType rhs) { return this.impl + rhs.impl; } If we had such a macro then the macro could handle mapping symbols ("+") to names ("opAdd"). Some sort of Nemerle-ish definition of such a macro: macro opBinary(op, ret, args, code) syntax { $ret opBinary $op ($args) { $code } } { auto opFuncName = operatorNameForSymbol(op); <| $ret $opFuncName ($args) { $code } |> } I think definable syntax is probably a long way off for DMD. But the rest of it has a direct translation in terms of a CTFE string processing function. All the args are actually 'string' just not declared as such. <| ... |> is basically just a call to a big string manipulation function that replaces $var with whatever value the string var has. And there's an implicit 'return' of that string. --bb
Nov 19 2009
parent reply Leandro Lucarella <llucax gmail.com> writes:
Bill Baxter, el 19 de noviembre a las 14:16 me escribiste:
 What I found hackish about it is that the code is a string manipulation
 mess. You can already do a string manipulation mess to programatically
 implement all the operator overloading.
This is true, but if you leave it entirely up to the programmer and string mixins, the mess is much more messy. You're going to end up with code like this: mixin(genBinaryOp("+", q{MyType}, q{MyType rhs}, q{ return this.impl + rhs.impl; })); instead of this: MyType opBinary(string op : "+")(MyType rhs) { return this.impl + rhs.impl; } I would have a hard time defending the former as the recommended D style. But the latter is not so bad. It looks like a regular template declaration, and code is code, not a string.
But in this case you gained nothing comparing to opAdd(), where opBinary() is useful is where you use string mixins to avoid implementing the operators one by one. I know the opBinary() approach is less ugly than writing the code youself, but a library solution could be provided in the meantime (in the unexistent std.mixin module; which BTW, can't be named that way because "mixin" is a keyword ;). -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) ---------------------------------------------------------------------- When I was a child I caught a fleeting glimpse Out of the corner of my eye. I turned to look but it was gone I cannot put my finger on it now The child is grown, The dream is gone. I have become comfortably numb.
Nov 19 2009
parent Bill Baxter <wbaxter gmail.com> writes:
On Thu, Nov 19, 2009 at 5:17 PM, Leandro Lucarella <llucax gmail.com> wrote=
:
 Bill Baxter, el 19 de noviembre a las 14:16 me escribiste:
 What I found hackish about it is that the code is a string manipulatio=
n
 mess. You can already do a string manipulation mess to programatically
 implement all the operator overloading.
This is true, but if you leave it entirely up to the programmer and string mixins, the mess is much more messy. You're going to end up with code like this: mixin(genBinaryOp("+", q{MyType}, q{MyType rhs}, q{ =A0 =A0 return this.impl + rhs.impl; })); instead of this: MyType opBinary(string op : "+")(MyType rhs) =A0{ =A0 =A0 return this.impl + rhs.impl; } I would have a hard time defending the former as the recommended D style=
.
 But the latter is not so bad. =A0It looks like a regular template
 declaration, and code is code, not a string.
But in this case you gained nothing comparing to opAdd(), where opBinary(=
)
 is useful is where you use string mixins to avoid implementing the
 operators one by one.

 I know the opBinary() approach is less ugly than writing the code youself=
,
 but a library solution could be provided in the meantime (in the
 unexistent std.mixin module; which BTW, can't be named that way because
 "mixin" is a keyword ;).
My point was that the best you're going to be able to do with a library solution (even for multiple operators) would be to provide a "genBinaryOp" function, making your code look something like this: mixin(genBinaryOps("+ - * /", q{MyType}, q{MyType rhs}, q{ return this.impl $op rhs.impl; })); Or I guess with more effort you could probably have something like this: mixin(genBinaryOps("+ - * /", q{ MyType $opName(MyType rhs) { return this.impl $op rhs.impl; } })); But I still find the plain template idea cleaner for the user than a string generation function that does heck knows what. --bb
Nov 19 2009
prev sibling parent reply Bill Baxter <wbaxter gmail.com> writes:
On Wed, Nov 18, 2009 at 8:33 PM, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:

 I am thinking that representing operators by their exact token
 representation is a principled approach because it allows for unambiguous
 mapping,
There used to be an argument floating around that using 'opAdd" was better than "op+" because the former encourages people to only overload the method to do things that resemble addition. Whereas op+ says I'm just a symbol, do whatever you want with me. I never really bought into it... but it was what I was told years ago when I complained that opSub opMul etc were harder to remember than need be. :-) I guess D will have to change it's story now. --bb
Nov 19 2009
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Bill Baxter wrote:
 On Wed, Nov 18, 2009 at 8:33 PM, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:
 
 I am thinking that representing operators by their exact token
 representation is a principled approach because it allows for unambiguous
 mapping,
There used to be an argument floating around that using 'opAdd" was better than "op+" because the former encourages people to only overload the method to do things that resemble addition. Whereas op+ says I'm just a symbol, do whatever you want with me. I never really bought into it... but it was what I was told years ago when I complained that opSub opMul etc were harder to remember than need be. :-) I guess D will have to change it's story now. --bb
FWIW, I'm not buying into that either. May be one of those cases in which community's experience and insight has slowly obviated the party line. Andrei
Nov 19 2009
prev sibling parent reply lws <invalid email.com> writes:
On 2009-11-19 15:46:57 -0800, Bill Baxter <wbaxter gmail.com> said:

 On Wed, Nov 18, 2009 at 8:33 PM, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:
 
 I am thinking that representing operators by their exact token
 representation is a principled approach because it allows for unambiguous
 mapping,
There used to be an argument floating around that using 'opAdd" was better than "op+" because the former encourages people to only overload the method to do things that resemble addition. Whereas op+ says I'm just a symbol, do whatever you want with me. I never really bought into it... but it was what I was told years ago when I complained that opSub opMul etc were harder to remember than need be. :-) I guess D will have to change it's story now. --bb
The reason Walter provided me when I asked ~9 years ago was that "T operator+(T)" causes unnecessary headaches when parsing. opAdd does not produce these complications when parsing function declarations. T opBinary(string op)(T) also does not produce the same headaches. -SC
Dec 14 2009
parent reply Jesse Phillips <jessekphillips+D gmail.com> writes:
lws Wrote:

 On 2009-11-19 15:46:57 -0800, Bill Baxter <wbaxter gmail.com> said:
 
 On Wed, Nov 18, 2009 at 8:33 PM, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:
 
 I am thinking that representing operators by their exact token
 representation is a principled approach because it allows for unambiguous
 mapping,
There used to be an argument floating around that using 'opAdd" was better than "op+" because the former encourages people to only overload the method to do things that resemble addition. Whereas op+ says I'm just a symbol, do whatever you want with me. I never really bought into it... but it was what I was told years ago when I complained that opSub opMul etc were harder to remember than need be. :-) I guess D will have to change it's story now. --bb
The reason Walter provided me when I asked ~9 years ago was that "T operator+(T)" causes unnecessary headaches when parsing. opAdd does not produce these complications when parsing function declarations. T opBinary(string op)(T) also does not produce the same headaches. -SC
There is a page for the rational which I thought gave the OP's explination: http://digitalmars.com/d/2.0/rationale.html However it does have this nice little gem: "__ keywords should indicate a proprietary language extension, not a basic part of the language." Which makes you wonder why D has __traits, __gshared... or are those not basic parts of the language? :)
Dec 15 2009
next sibling parent reply KennyTM~ <kennytm gmail.com> writes:
On Dec 16, 09 07:21, Jesse Phillips wrote:
 lws Wrote:

 On 2009-11-19 15:46:57 -0800, Bill Baxter<wbaxter gmail.com>  said:

 On Wed, Nov 18, 2009 at 8:33 PM, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org>  wrote:

 I am thinking that representing operators by their exact token
 representation is a principled approach because it allows for unambiguous
 mapping,
There used to be an argument floating around that using 'opAdd" was better than "op+" because the former encourages people to only overload the method to do things that resemble addition. Whereas op+ says I'm just a symbol, do whatever you want with me. I never really bought into it... but it was what I was told years ago when I complained that opSub opMul etc were harder to remember than need be. :-) I guess D will have to change it's story now. --bb
The reason Walter provided me when I asked ~9 years ago was that "T operator+(T)" causes unnecessary headaches when parsing. opAdd does not produce these complications when parsing function declarations. T opBinary(string op)(T) also does not produce the same headaches. -SC
There is a page for the rational which I thought gave the OP's explination: http://digitalmars.com/d/2.0/rationale.html However it does have this nice little gem: "__ keywords should indicate a proprietary language extension, not a basic part of the language." Which makes you wonder why D has __traits, __gshared... or are those not basic parts of the language? :)
__gshared was deliberately named to discourage its use iirc.
Dec 16 2009
parent reply Jesse Phillips <jessekphillips+D gmail.com> writes:
KennyTM~ Wrote:

 On Dec 16, 09 07:21, Jesse Phillips wrote:
 http://digitalmars.com/d/2.0/rationale.html

 However it does have this nice little gem:

 "__ keywords should indicate a proprietary language extension, not a basic
part of the language."

 Which makes you wonder why D has __traits, __gshared... or are those not basic
parts of the language? :)
__gshared was deliberately named to discourage its use iirc.
Yes, which makes things even more inconsistent. __ indicates proprietary language extension __ we don't want you using this much (__gshared) __ this way we aren't adding a keyword to the standard namespace (__traits) At least those have been the reasons I've heard. This makes __ absolutely meaningless and good at making things ugly.
Dec 16 2009
parent "Joel C. Salomon" <joelcsalomon gmail.com> writes:
On 12/16/2009 2:36 PM, Jesse Phillips wrote:
 __ indicates proprietary language extension
 __ we don't want you using this much (__gshared)
 __ this way we aren't adding a keyword to the standard namespace (__traits)
 
 At least those have been the reasons I've heard. This makes __ absolutely
meaningless and good at making things ugly.
That’s actually the best reason: “__” makes things ugly, so the community is moved to find a cleaner way to express what these identifiers signify. —Joel Salomon
Dec 18 2009
prev sibling parent Robert Clipsham <robert octarineparrot.com> writes:
On 15/12/09 23:21, Jesse Phillips wrote:
 There is a page for the rational which I thought gave the OP's explination:

 http://digitalmars.com/d/2.0/rationale.html

 However it does have this nice little gem:

 "__ keywords should indicate a proprietary language extension, not a basic
part of the language."

 Which makes you wonder why D has __traits, __gshared... or are those not basic
parts of the language? :)
I'd drop this in bugzilla, I think this needs addressing before D2 is released :)
Dec 16 2009
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
grauzone wrote:
 Andrei Alexandrescu wrote:
 grauzone wrote:
 Andrei Alexandrescu wrote:
 The rewrite is done long after lexing, so no low-level problems there.
Oh, I thought it would let you introduce new operators. But it's only about the existing ones. I find the idea to identify the operator using a string very sloppy and sillyl just like using string mixins for small delegates in std.algorithm etc.; but you'd probably say "it works and is useful" and "it's short" and "it solves the current problem", so... whatever.
We're trying to improve on the current situation, which forces the user to manually define a lot of small functions. If you have convincing reasons to argue that the current state of affairs is actually better, I'm all ears - both Walter and I could use less work, particularly if the outcome sucks (see e.g. T[new]). Also, if you have ideas on how things could be done in a way that you'd find not sloppy and not silly, that would be even better.
If I had a better proposal, I'd post it. I'm just saying that's it's a bad hack, that _although_ solves the problem, will have negative side effects for other reasons.
What are those other reasons? I'd be grateful if you could spell a complete argument.
 Does the current proposal make things simpler at all? All you're doing 
 is to enable the programmer to "fix" the clumsy semantics by throwing 
 lots of CTFE onto the problem. Why not generate the operator functions 
 with CTFE in the first place...
I'm afraid there is a confusion, in which case let me dispel it. There's no CTFE involved. The way people would use the feature would be in one of the following ways: 1. Just use "if" or "static if" the old-school: T opBinary(string op)(T rhs) { static if (op == "+") return data + rhs.data; else static if (op == "-") return data - rhs.data; ... else static assert(0, "Operator "~op~" not implemented"); } This has not a lot of advantage beyond the fact that you don't need to remember the mappings from symbols to their names. 2. Use one mixin expression: T opBinary(string op)(T rhs) { return mixin("data "~op~" rhs.data"); } 3. Combine the two in various ways, also use restricted templates, forward to virtual functions, etc. etc. The improvements are the following: 1. There is no need to memorize or look up a long table of symbol-name correspondence (e.g. + is opAdd, etc.) 2. There are source code savings because most often types want to overload operators en masse, not just a couple of them. In that case there is a code explosion of many functions. Worse, if the code needs to do something clever (e.g. Variant or expression templates), the cleverness must be spread all over, or effort needs to be spent into mapping back names to their symbols, as Variant actually does. It's not a pretty sight, so I'd rather have the language facilitate good usage instead of it facilitating bad usage that must be reverted to good usage. 3. The syntax is extensible, for example Walter wanted for the longest time to make !<>= etc. overridable but couldn't bring himself to write abominable names like opNotGreaterThanOrWhatever. Now it's very simple to integrate all symbols properly and without aggravation. The cons I see are: 1. The user who writes even the old-school operators must have a basic understanding of compile-time parameterized functions. 2. Existing code gets broken. 3. The template functions can't be virtual (fixable with a forwarding thunk written once and for all). Any more thoughts, please let them known. Again, this is the ideal time to contribute. But "meh, it's a hack" is difficult to discuss. Andrei
Nov 18 2009
next sibling parent Chad J <chadjoan __spam.is.bad__gmail.com> writes:
Andrei Alexandrescu wrote:
 ...
 Any more thoughts, please let them known. ...
 
 
 Andrei
I like what I'm reading. Pretty clever! I imagine it'd be convenient to easily distinguish between pure binary operators and opAssign binary operators. I can easily envision myself marking operator overloads such as + and * as pure to gain a performance boost. Overloads like += and *= OTOH, often cannot be marked as pure. - Chad
Nov 19 2009
prev sibling parent reply grauzone <none example.net> writes:
Andrei Alexandrescu wrote:
 Any more thoughts, please let them known. Again, this is the ideal time 
 to contribute. But "meh, it's a hack" is difficult to discuss.
Well that's just like as if Bjarne Stroustrup would ask you: "What would you have done in my place? This looked like the right thing to do at this time!". And now we're working on a language that's supposed to replace C++. Partially I don't really know how opBinary is supposed to solve most operator overloading problems (listed in DIP7). It just looks like a stupid dispatch mechanism. It could be implemented by using CTFE and mixins without compiler changes: just let a CTFE function generate a dispatcher function for each opSomething to opBinary. Of course, if you think operators are something that's forwarded to something else, it'd be nicer if dmd would be doing this, because code gets shorter. So opSomething gets ditched in favor of opBinary. But actually, the functionality of opBinary can be provided as a template mixin or a CTFE function in Phobos. At least then the user has a choice what to use. (About the issue that you need to remember names for operator symbols: you know C++ has a fabulous idea how to get around this...) Now what about unary operators? Or very specific stuff like opApply? What's with opSomethingAssign (or "expr1[expr2] = expr3" in general)? opBinary doesn't seem to solve any of those. Also, all this just goes down to a generic "opOperator(char[] expression, T...)(T args)", where expression is actually an expression involved with the object. It feels like this leads to nothing. And opBinary is a just a quite arbitrary stop on that way to nothing. Just a hack to make code shorter for some use cases. One way of solving this issue about "extended operator overloading" would be to introduce proper AST macros. An AST macro could match on a leaf of an expression and replace it by custom code, and use this mechanism to deal with stuff like "expr1[expr2] = expr3" (and I don't see how opBinary would solve this... encode the expression as a string? fallback to naive code if opBinary fails to match?). At least that's what I thought AST macros would be capable to do. Anyway, AST macros got ditched in favor of const/immutable, so that's not an option. You also might feel about AST macros as a vague idea, that only solves "everything" because it's so vague and unspecified. Feel free to go on about this.
Nov 19 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
grauzone wrote:
 Andrei Alexandrescu wrote:
 Any more thoughts, please let them known. Again, this is the ideal 
 time to contribute. But "meh, it's a hack" is difficult to discuss.
Well that's just like as if Bjarne Stroustrup would ask you: "What would you have done in my place? This looked like the right thing to do at this time!". And now we're working on a language that's supposed to replace C++.
I'm not sure what you mean here.
 Partially I don't really know how opBinary is supposed to solve most 
 operator overloading problems (listed in DIP7). It just looks like a 
 stupid dispatch mechanism. It could be implemented by using CTFE and 
 mixins without compiler changes: just let a CTFE function generate a 
 dispatcher function for each opSomething to opBinary. Of course, if you 
 think operators are something that's forwarded to something else, it'd 
 be nicer if dmd would be doing this, because code gets shorter. So 
 opSomething gets ditched in favor of opBinary. But actually, the 
 functionality of opBinary can be provided as a template mixin or a CTFE 
 function in Phobos. At least then the user has a choice what to use.
Things could indeed be generated with a CTFE mixin, but first we'd need a hecatomb of names to be added: all floating-point comparison operators and all index-assign operators. With the proposed approach there is no more need to add all those names and have the users consult tables to know how they are named.
 (About the issue that you need to remember names for operator symbols: 
 you know C++ has a fabulous idea how to get around this...)
I think it's not as flexible a solution as passing a compile-time string because there is no way to actually use that token.
 Now what about unary operators?
opUnary.
 Or very specific stuff like opApply? 
opApply stays as it is.
 What's with opSomethingAssign (or "expr1[expr2]  = expr3" in general)? 
 opBinary doesn't seem to solve any of those.
opBinary does solve opIndex* morass because it only adds one function per category, not one function per operator. For example: struct T { // op can be "=", "+=", "-=" etc. E opAssign(string op)(E rhs) { ... } // op can be "=", "+=", "-=" etc. E opIndexAssign(string op)(size_t i, E rhs) { ... } } This was one motivation: instead of defining a lot of small functions that have each a specific name, define one function for each category of operations and encode the operator name as its own token. I don't understand exactly what the problem is with that.
 Also, all this just goes 
 down to a generic "opOperator(char[] expression, T...)(T args)", where 
 expression is actually an expression involved with the object. It feels 
 like this leads to nothing.
We need something to work with. "looks like a stupid mechanism" and "feels" are not things that foster further dialog.
 And opBinary is a just a quite arbitrary 
 stop on that way to nothing. Just a hack to make code shorter for some 
 use cases.
opBinary is a binary operator, hardly something someone would pull out of a hat. I'm not sure what you mean to say here.
 One way of solving this issue about "extended operator overloading" 
 would be to introduce proper AST macros. An AST macro could match on a 
 leaf of an expression and replace it by custom code, and use this 
 mechanism to deal with stuff like "expr1[expr2]  = expr3" (and I don't 
 see how opBinary would solve this... encode the expression as a string? 
 fallback to naive code if opBinary fails to match?). At least that's 
 what I thought AST macros would be capable to do.
See above on how the proposed approach addresses RMW operations on indexes.
 Anyway, AST macros got ditched in favor of const/immutable, so that's 
 not an option.
That I agree with. Andrei
Nov 19 2009
next sibling parent grauzone <none example.net> writes:
Andrei Alexandrescu wrote:
 opBinary does solve opIndex* morass because it only adds one function 
 per category, not one function per operator. For example:
 
 struct T {
     // op can be "=", "+=", "-=" etc.
     E opAssign(string op)(E rhs) { ... }
     // op can be "=", "+=", "-=" etc.
     E opIndexAssign(string op)(size_t i, E rhs) { ... }
 }
I see, that makes sense now. It shouldn't be "+=", though (rather "+"), because it makes chaining harder.
Nov 19 2009
prev sibling parent reply Bill Baxter <wbaxter gmail.com> writes:
On Thu, Nov 19, 2009 at 8:46 AM, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:
 grauzone wrote:
 What's with opSomethingAssign (or "expr1[expr2]  =3D expr3" in general)?
 opBinary doesn't seem to solve any of those.
opBinary does solve opIndex* morass because it only adds one function per category, not one function per operator. For example: struct T { =A0 =A0// op can be "=3D", "+=3D", "-=3D" etc. =A0 =A0E opAssign(string op)(E rhs) { ... } =A0 =A0// op can be "=3D", "+=3D", "-=3D" etc. =A0 =A0E opIndexAssign(string op)(size_t i, E rhs) { ... } }
Rewrite a.prop =3D x; =3D> a.opPropertyAssign!("prop", "=3D")(x); to that and we're really getting somewhere! --bb
Nov 19 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Bill Baxter wrote:
 On Thu, Nov 19, 2009 at 8:46 AM, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:
 grauzone wrote:
 What's with opSomethingAssign (or "expr1[expr2]  = expr3" in general)?
 opBinary doesn't seem to solve any of those.
opBinary does solve opIndex* morass because it only adds one function per category, not one function per operator. For example: struct T { // op can be "=", "+=", "-=" etc. E opAssign(string op)(E rhs) { ... } // op can be "=", "+=", "-=" etc. E opIndexAssign(string op)(size_t i, E rhs) { ... } }
Rewrite a.prop = x; => a.opPropertyAssign!("prop", "=")(x); to that and we're really getting somewhere! --bb
I swear I was thinking of that. Andrei
Nov 19 2009
parent reply =?ISO-8859-1?Q?Pelle_M=E5nsson?= <pelle.mansson gmail.com> writes:
Andrei Alexandrescu wrote:
 Bill Baxter wrote:
 On Thu, Nov 19, 2009 at 8:46 AM, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:
 grauzone wrote:
 What's with opSomethingAssign (or "expr1[expr2]  = expr3" in general)?
 opBinary doesn't seem to solve any of those.
opBinary does solve opIndex* morass because it only adds one function per category, not one function per operator. For example: struct T { // op can be "=", "+=", "-=" etc. E opAssign(string op)(E rhs) { ... } // op can be "=", "+=", "-=" etc. E opIndexAssign(string op)(size_t i, E rhs) { ... } }
Rewrite a.prop = x; => a.opPropertyAssign!("prop", "=")(x); to that and we're really getting somewhere! --bb
I swear I was thinking of that. Andrei
Is this doable without a performance drop?
Nov 20 2009
parent lws <invalid email.com> writes:
On 2009-11-20 02:18:03 -0800, Pelle Mnsson <pelle.mansson gmail.com> said:

 Andrei Alexandrescu wrote:
 Bill Baxter wrote:
 On Thu, Nov 19, 2009 at 8:46 AM, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:
 grauzone wrote:
 What's with opSomethingAssign (or "expr1[expr2]  = expr3" in general)?
 opBinary doesn't seem to solve any of those.
opBinary does solve opIndex* morass because it only adds one function per category, not one function per operator. For example: struct T { // op can be "=", "+=", "-=" etc. E opAssign(string op)(E rhs) { ... } // op can be "=", "+=", "-=" etc. E opIndexAssign(string op)(size_t i, E rhs) { ... } }
Rewrite a.prop = x; => a.opPropertyAssign!("prop", "=")(x); to that and we're really getting somewhere! --bb
I swear I was thinking of that. Andrei
Is this doable without a performance drop?
These are compile-time string mixin magic. It might make your code compile slower, but the metaprogramming engine in D is already 1000000000x faster than C++.
Dec 14 2009
prev sibling parent reply retard <re tard.com.invalid> writes:
Wed, 18 Nov 2009 18:35:18 -0800, Andrei Alexandrescu wrote:

 grauzone wrote:
 Andrei Alexandrescu wrote:
 The rewrite is done long after lexing, so no low-level problems there.
Oh, I thought it would let you introduce new operators. But it's only about the existing ones. I find the idea to identify the operator using a string very sloppy and sillyl just like using string mixins for small delegates in std.algorithm etc.; but you'd probably say "it works and is useful" and "it's short" and "it solves the current problem", so... whatever.
We're trying to improve on the current situation, which forces the user to manually define a lot of small functions. If you have convincing reasons to argue that the current state of affairs is actually better, I'm all ears - both Walter and I could use less work, particularly if the outcome sucks (see e.g. T[new]). Also, if you have ideas on how things could be done in a way that you'd find not sloppy and not silly, that would be even better.
Does the new system allow overriding only some binary operations and not all of them at once? I thought generic member functions were non-virtual?
Nov 19 2009
parent reply "Simen kjaeraas" <simen.kjaras gmail.com> writes:
retard <re tard.com.invalid> wrote:

 Does the new system allow overriding only some binary operations and not
 all of them at once?
struct S { S opBinary( string op )( S rhs ) if ( op == "+" ) { // Do stuff } }
 I thought generic member functions were non-virtual?
Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> wrote:
 3. The template functions can't be virtual (fixable with a forwarding  
 thunk written once and for all).
-- Simen
Nov 19 2009
parent retard <re tard.com.invalid> writes:
Thu, 19 Nov 2009 09:33:07 +0100, Simen kjaeraas wrote:

 retard <re tard.com.invalid> wrote:
 
 Does the new system allow overriding only some binary operations and
 not all of them at once?
struct S { S opBinary( string op )( S rhs ) if ( op == "+" ) { // Do stuff } }
 I thought generic member functions were non-virtual?
I meant this: class foo { int value; this(int a) { value = a; } foo opAdd(foo other) { return new foo(value + other.value); } } class bar : foo { int value2; this(int a, int b) { super(a); value2 = b; } override foo opAdd(foo other) { return new bar(value + other.value, value2+1); } }
 
 
 Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> wrote:
 3. The template functions can't be virtual (fixable with a forwarding
 thunk written once and for all).
Nov 19 2009
prev sibling next sibling parent KennyTM~ <kennytm gmail.com> writes:
On Nov 19, 09 07:48, bearophile wrote:
 Andrei Alexandrescu:

 * Encode operators by compile-time strings. For example, instead of the
 plethora of opAdd, opMul, ..., we'd have this:

 T opBinary(string op)(T rhs) { ... }

 The string is "+", "*", etc.
Can you show an example of defining an operator, like a minus, with that? In my set data structure I'd like to define "<=" among two sets as "is subset". Can that design allow me to overload just<= and>= ? (opCmp is not enough here). Bye, bearophile
float opCmp(X other) { if (this cannot compare with other) return float.nan; else ... }
Nov 18 2009
prev sibling parent reply Gzp <galap freemail.hu> writes:
bearophile wrote:
 Andrei Alexandrescu:
 
 * Encode operators by compile-time strings. For example, instead of the 
 plethora of opAdd, opMul, ..., we'd have this:

 T opBinary(string op)(T rhs) { ... }

 The string is "+", "*", etc.
Can you show an example of defining an operator, like a minus, with that? In my set data structure I'd like to define "<=" among two sets as "is subset". Can that design allow me to overload just <= and >= ? (opCmp is not enough here). Bye, bearophile
And what about the unary operators. Sorry I was not following the opBinary thread from the beginning, so it might have been discussed before. If opBinary has a syntax like this, unary operators require something like this at least just for completeness. (And maybe trinary operators ?: ) :) Can they still be overloaded ? Will they have a similar syntax ? If so what about the e++ and ++e operators? How they are distinct ? Or is the latter eliminated from the language ? Though, I like the idea, just a quick look at a code and you can see all the operators in a place. Bye, Gzp
Nov 18 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Gzp wrote:
 bearophile wrote:
 Andrei Alexandrescu:

 * Encode operators by compile-time strings. For example, instead of 
 the plethora of opAdd, opMul, ..., we'd have this:

 T opBinary(string op)(T rhs) { ... }

 The string is "+", "*", etc.
Can you show an example of defining an operator, like a minus, with that? In my set data structure I'd like to define "<=" among two sets as "is subset". Can that design allow me to overload just <= and >= ? (opCmp is not enough here). Bye, bearophile
And what about the unary operators. Sorry I was not following the opBinary thread from the beginning, so it might have been discussed before. If opBinary has a syntax like this, unary operators require something like this at least just for completeness.
Yes; opBinary was just given as an example. Unary operators look like this: T opUnary(string op)();
 (And maybe trinary operators ?: ) :)
 Can they still be overloaded ? Will they have a similar syntax ? If so
 what about the e++ and ++e operators? How they are distinct ? Or is the 
 latter eliminated from the language ?
 Though, I like the idea, just a quick look at a code and you can see all 
 the operators in a place.
 
 Bye, Gzp
I think user code should only define ++e. Then the compiler can use that to define e++. Andrei
Nov 19 2009
parent gzp <galap freemail.hu> writes:
 Yes; opBinary was just given as an example. Unary operators look like this:
 
 T opUnary(string op)();
 
 (And maybe trinary operators ?: ) :)
 Can they still be overloaded ? Will they have a similar syntax ? If so
 what about the e++ and ++e operators? How they are distinct ? Or is 
 the latter eliminated from the language ?
 Though, I like the idea, just a quick look at a code and you can see 
 all the operators in a place.

 Bye, Gzp
I think user code should only define ++e. Then the compiler can use that to define e++. Andrei
Thanks, that's great. Gzp
Nov 19 2009
prev sibling next sibling parent reply Jesse Phillips <jessekphillips+D gmail.com> writes:
Andrei Alexandrescu Wrote:

 6. There must be many things I forgot to mention, or that cause grief to 
 many of us. Please add to/comment on this list.
 
 Andrei
Well, there is this page on Wiki4D http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel#FutureDirections which has what the community is trying to track for changes. It also has some things that haven't got officially declined committed to.
Nov 18 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Jesse Phillips wrote:
 Andrei Alexandrescu Wrote:
 
 6. There must be many things I forgot to mention, or that cause
 grief to many of us. Please add to/comment on this list.
 
 Andrei
Well, there is this page on Wiki4D http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel#FutureDirections which has what the community is trying to track for changes. It also has some things that haven't got officially declined committed to.
Great, thanks. List looks very to the point and up to date. Andrei
Nov 18 2009
parent Jesse Phillips <jessekphillips gmail.com> writes:
On Wed, 18 Nov 2009 17:14:34 -0800, Andrei Alexandrescu wrote:

 Great, thanks. 
"List looks very to the point" Yeah no effort has been made to make it new comer friendly (not that there is a need too) "and up to date." Don has help out greatly for that.
Nov 18 2009
prev sibling next sibling parent reply dsimcha <dsimcha yahoo.com> writes:
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s
article
 6. There must be many things I forgot to mention, or that cause
grief to
 many of us. Please add to/comment on this list.
 Andrei
I assume we're mostly talking about spec stuff, not implementation stuff. Nonetheless, to the extent that the GC API is considered part of the language spec, I think we need to fix that. 1. My precise heap scanning patch has some ripple effects into the GC API and could affect what is done with the `new` operator. Therefore, we might not want to postpone this until after D2 is final. 2. We should probably tighten up the spec to make sure that a moving, fully precise, etc. GC can be implemented at some point in the future without modifying the spec after D2 is declared gold.
Nov 18 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s
 article
 6. There must be many things I forgot to mention, or that cause
grief to
 many of us. Please add to/comment on this list.
 Andrei
I assume we're mostly talking about spec stuff, not implementation stuff. Nonetheless, to the extent that the GC API is considered part of the language spec, I think we need to fix that. 1. My precise heap scanning patch has some ripple effects into the GC API and could affect what is done with the `new` operator. Therefore, we might not want to postpone this until after D2 is final. 2. We should probably tighten up the spec to make sure that a moving, fully precise, etc. GC can be implemented at some point in the future without modifying the spec after D2 is declared gold.
I think you can safely work on that asynchronously. TDPL won't include a "GC API Reference" - for that kind of stuff I think it's ok to refer to the online documentation. Thanks for working on this! Andrei
Nov 18 2009
parent dsimcha <dsimcha yahoo.com> writes:
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s
 article
 6. There must be many things I forgot to mention, or that cause
grief to
 many of us. Please add to/comment on this list.
 Andrei
I assume we're mostly talking about spec stuff, not implementation stuff. Nonetheless, to the extent that the GC API is considered part of the language spec, I think we need to fix that. 1. My precise heap scanning patch has some ripple effects into the GC API and could affect what is done with the `new` operator. Therefore, we might not want to postpone this until after D2 is final. 2. We should probably tighten up the spec to make sure that a moving, fully precise, etc. GC can be implemented at some point in the future without modifying the spec after D2 is declared gold.
I think you can safely work on that asynchronously. TDPL won't include a "GC API Reference" - for that kind of stuff I think it's ok to refer to the online documentation. Thanks for working on this! Andrei
Can you clarify the higher level development model, then? 1. Are we departing from the D1 model and allowing changes to Phobos after the language spec is declared final? If so, can those changes be breaking, at least at the binary level? 2. Are we punting on the GC API and allowing it to be implementation defined? I thought this API was supposed to be stable and allow for swapping GC implementations at link time. Then again, I think it's actually a bad idea to create such a stable API, since different GC implementations will require different configuration, meta-data, etc. and this is just a fact of life. Most user code will not interact directly with the GC, but will use `new`, builtin arrays, etc.
Nov 18 2009
prev sibling next sibling parent reply Kyle <none dontemail.com> writes:
Andrei Alexandrescu Wrote:

 6. There must be many things I forgot to mention, or that cause grief to 
 many of us. Please add to/comment on this list.
Uniform function call syntax.
Nov 18 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Kyle wrote:
 Andrei Alexandrescu Wrote:
 
 6. There must be many things I forgot to mention, or that cause grief to 
 many of us. Please add to/comment on this list.
Uniform function call syntax.
It's in the book. I'm adding this message as a reminder to add a test case. Thanks! Andrei
Nov 18 2009
next sibling parent Bill Baxter <wbaxter gmail.com> writes:
On Wed, Nov 18, 2009 at 5:12 PM, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:
 Kyle wrote:
 Andrei Alexandrescu Wrote:

 6. There must be many things I forgot to mention, or that cause grief to
 many of us. Please add to/comment on this list.
Uniform function call syntax.
It's in the book. I'm adding this message as a reminder to add a test case. Thanks!
It's in the book as working with all types? Or just built-in array types? If we're going to have that is there any reason not to have out-of-class operator overloads at last? If I can do this: void doSomething(MyClass x) { ... } MyClass x; x.doSomething(); Then why not this: void opUnary(string op)(MyClass x) { ... } MyClass x; !x A.k.a x.opUnary!("!")(); --bb
Nov 19 2009
prev sibling parent Lutger <lutger.blijdestijn gmail.com> writes:
Andrei Alexandrescu wrote:

 Kyle wrote:
 Andrei Alexandrescu Wrote:
 
 6. There must be many things I forgot to mention, or that cause grief to
 many of us. Please add to/comment on this list.
Uniform function call syntax.
It's in the book. I'm adding this message as a reminder to add a test case. Thanks! Andrei
I'm delighted, thanks!
Nov 20 2009
prev sibling next sibling parent reply Kyle <none dontemail.com> writes:
Andrei Alexandrescu Wrote:

 6. There must be many things I forgot to mention, or that cause grief to 
 many of us. Please add to/comment on this list.
Static function parameters
Nov 18 2009
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Kyle wrote:
 Andrei Alexandrescu Wrote:
 
 6. There must be many things I forgot to mention, or that cause grief to 
 many of us. Please add to/comment on this list.
Static function parameters
Walter tried to implement them and ran into a number of odd semantic issues. Andrei
Nov 18 2009
prev sibling next sibling parent reply dsimcha <dsimcha yahoo.com> writes:
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 3. It was mentioned in this group that if getopt() does not work in
 SafeD, then SafeD may as well pack and go home. I agree. We need to make
 it work. Three ideas discussed with Walter:
 * Allow taking addresses of locals, but in that case switch allocation
 from stack to heap, just like with delegates. If we only do that in
 SafeD, behavior will be different than with regular D. In any case, it's
 an inefficient proposition, particularly for getopt() which actually
 does not need to escape the addresses - just fills them up.
IMHO this is a terrible solution. SafeD should not cause major ripple effects for pieces of code that don't want to use it. I'm all for safe defaults even if they're less efficient or less flexible, but if D starts sacrificing performance or flexibility for safety **even when the programmer explicitly asks it not to**, then it will officially have become a bondage and discipline language. Furthermore, as you point out, having the semantics of something vary in subtle ways between SafeD and unsafe D is probably a recipe for confusion.
 * Allow  trusted (and maybe even  safe) functions to receive addresses
 of locals. Statically check that they never escape an address of a
 parameter. I think this is very interesting because it enlarges the
 common ground of D and SafeD.
This is a great idea if it can be implemented. Isn't escape analysis a pretty hard thing to get right, though, especially when you might not have the source code to the function being called?
 * Figure out a way to reconcile "ref" with variadics. This is the actual
 reason why getopt chose to traffic in addresses, and fixing it is the
 logical choice and my personal favorite.
This should be done eventually regardless of what happens with taking addresses of locals, though I'm not sure it still makes the short list if we solve the addresses of locals thing some other way.
Nov 18 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 3. It was mentioned in this group that if getopt() does not work in
 SafeD, then SafeD may as well pack and go home. I agree. We need to make
 it work. Three ideas discussed with Walter:
 * Allow taking addresses of locals, but in that case switch allocation
 from stack to heap, just like with delegates. If we only do that in
 SafeD, behavior will be different than with regular D. In any case, it's
 an inefficient proposition, particularly for getopt() which actually
 does not need to escape the addresses - just fills them up.
IMHO this is a terrible solution. SafeD should not cause major ripple effects for pieces of code that don't want to use it. I'm all for safe defaults even if they're less efficient or less flexible, but if D starts sacrificing performance or flexibility for safety **even when the programmer explicitly asks it not to**, then it will officially have become a bondage and discipline language. Furthermore, as you point out, having the semantics of something vary in subtle ways between SafeD and unsafe D is probably a recipe for confusion.
 * Allow  trusted (and maybe even  safe) functions to receive addresses
 of locals. Statically check that they never escape an address of a
 parameter. I think this is very interesting because it enlarges the
 common ground of D and SafeD.
This is a great idea if it can be implemented. Isn't escape analysis a pretty hard thing to get right, though, especially when you might not have the source code to the function being called?
Escape analysis is difficult when you don't have information about the functions you're passing the pointer to. For example: void fun(int* p) { if (condition) gun(p); } Now the problem is that fun's escape-or-not behavior depends on flow (i.e. condition) and on gun's escaping behavior. If we use safe and trusted to indicate unequivocally "no escape", then there is no analysis to be done - the hard part of the analysis has already been done manually by the user.
 * Figure out a way to reconcile "ref" with variadics. This is the actual
 reason why getopt chose to traffic in addresses, and fixing it is the
 logical choice and my personal favorite.
This should be done eventually regardless of what happens with taking addresses of locals, though I'm not sure it still makes the short list if we solve the addresses of locals thing some other way.
I agree. Andrei
Nov 18 2009
next sibling parent reply dsimcha <dsimcha yahoo.com> writes:
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 3. It was mentioned in this group that if getopt() does not work in
 SafeD, then SafeD may as well pack and go home. I agree. We need to make
 it work. Three ideas discussed with Walter:
 * Allow taking addresses of locals, but in that case switch allocation
 from stack to heap, just like with delegates. If we only do that in
 SafeD, behavior will be different than with regular D. In any case, it's
 an inefficient proposition, particularly for getopt() which actually
 does not need to escape the addresses - just fills them up.
IMHO this is a terrible solution. SafeD should not cause major ripple effects for pieces of code that don't want to use it. I'm all for safe defaults even if they're less efficient or less flexible, but if D starts sacrificing performance or flexibility for safety **even when the programmer explicitly asks it not to**, then it will officially have become a bondage and discipline language. Furthermore, as you point out, having the semantics of something vary in subtle ways between SafeD and unsafe D is probably a recipe for confusion.
 * Allow  trusted (and maybe even  safe) functions to receive addresses
 of locals. Statically check that they never escape an address of a
 parameter. I think this is very interesting because it enlarges the
 common ground of D and SafeD.
This is a great idea if it can be implemented. Isn't escape analysis a pretty hard thing to get right, though, especially when you might not have the source code to the function being called?
Escape analysis is difficult when you don't have information about the functions you're passing the pointer to. For example: void fun(int* p) { if (condition) gun(p); } Now the problem is that fun's escape-or-not behavior depends on flow (i.e. condition) and on gun's escaping behavior. If we use safe and trusted to indicate unequivocally "no escape", then there is no analysis to be done - the hard part of the analysis has already been done manually by the user.
But then the safe or trusted function wouldn't be able to escape pointers to heap or static data segment memory either, if I understand this proposal correctly.
Nov 18 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 3. It was mentioned in this group that if getopt() does not work in
 SafeD, then SafeD may as well pack and go home. I agree. We need to make
 it work. Three ideas discussed with Walter:
 * Allow taking addresses of locals, but in that case switch allocation
 from stack to heap, just like with delegates. If we only do that in
 SafeD, behavior will be different than with regular D. In any case, it's
 an inefficient proposition, particularly for getopt() which actually
 does not need to escape the addresses - just fills them up.
IMHO this is a terrible solution. SafeD should not cause major ripple effects for pieces of code that don't want to use it. I'm all for safe defaults even if they're less efficient or less flexible, but if D starts sacrificing performance or flexibility for safety **even when the programmer explicitly asks it not to**, then it will officially have become a bondage and discipline language. Furthermore, as you point out, having the semantics of something vary in subtle ways between SafeD and unsafe D is probably a recipe for confusion.
 * Allow  trusted (and maybe even  safe) functions to receive addresses
 of locals. Statically check that they never escape an address of a
 parameter. I think this is very interesting because it enlarges the
 common ground of D and SafeD.
This is a great idea if it can be implemented. Isn't escape analysis a pretty hard thing to get right, though, especially when you might not have the source code to the function being called?
Escape analysis is difficult when you don't have information about the functions you're passing the pointer to. For example: void fun(int* p) { if (condition) gun(p); } Now the problem is that fun's escape-or-not behavior depends on flow (i.e. condition) and on gun's escaping behavior. If we use safe and trusted to indicate unequivocally "no escape", then there is no analysis to be done - the hard part of the analysis has already been done manually by the user.
But then the safe or trusted function wouldn't be able to escape pointers to heap or static data segment memory either, if I understand this proposal correctly.
Yah. The question is to what extent is that necessary. Andrei
Nov 18 2009
parent reply dsimcha <dsimcha yahoo.com> writes:
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 3. It was mentioned in this group that if getopt() does not work in
 SafeD, then SafeD may as well pack and go home. I agree. We need to make
 it work. Three ideas discussed with Walter:
 * Allow taking addresses of locals, but in that case switch allocation
 from stack to heap, just like with delegates. If we only do that in
 SafeD, behavior will be different than with regular D. In any case, it's
 an inefficient proposition, particularly for getopt() which actually
 does not need to escape the addresses - just fills them up.
IMHO this is a terrible solution. SafeD should not cause major ripple
effects for
 pieces of code that don't want to use it.  I'm all for safe defaults even if
 they're less efficient or less flexible, but if D starts sacrificing
performance
 or flexibility for safety **even when the programmer explicitly asks it not
to**,
 then it will officially have become a bondage and discipline language.

 Furthermore, as you point out, having the semantics of something vary in subtle
 ways between SafeD and unsafe D is probably a recipe for confusion.


 * Allow  trusted (and maybe even  safe) functions to receive addresses
 of locals. Statically check that they never escape an address of a
 parameter. I think this is very interesting because it enlarges the
 common ground of D and SafeD.
This is a great idea if it can be implemented. Isn't escape analysis a pretty hard thing to get right, though, especially when you might not have the source code to the function being called?
Escape analysis is difficult when you don't have information about the functions you're passing the pointer to. For example: void fun(int* p) { if (condition) gun(p); } Now the problem is that fun's escape-or-not behavior depends on flow (i.e. condition) and on gun's escaping behavior. If we use safe and trusted to indicate unequivocally "no escape", then there is no analysis to be done - the hard part of the analysis has already been done manually by the user.
But then the safe or trusted function wouldn't be able to escape pointers to heap or static data segment memory either, if I understand this proposal
correctly.

 Yah. The question is to what extent is that necessary.
 Andrei
Too kludgey for me. I'd rather just see ref parameters get fixed and just don't allow taking the address of locals in safe functions. I'd say that, except in low-level systems programming that would probably not be safe for other reasons anyhow, there would be very few good if any good reasons to take the address of a local if reference tuples just worked.
Nov 18 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 3. It was mentioned in this group that if getopt() does not work in
 SafeD, then SafeD may as well pack and go home. I agree. We need to make
 it work. Three ideas discussed with Walter:
 * Allow taking addresses of locals, but in that case switch allocation
 from stack to heap, just like with delegates. If we only do that in
 SafeD, behavior will be different than with regular D. In any case, it's
 an inefficient proposition, particularly for getopt() which actually
 does not need to escape the addresses - just fills them up.
IMHO this is a terrible solution. SafeD should not cause major ripple
effects for
 pieces of code that don't want to use it.  I'm all for safe defaults even if
 they're less efficient or less flexible, but if D starts sacrificing
performance
 or flexibility for safety **even when the programmer explicitly asks it not
to**,
 then it will officially have become a bondage and discipline language.

 Furthermore, as you point out, having the semantics of something vary in subtle
 ways between SafeD and unsafe D is probably a recipe for confusion.


 * Allow  trusted (and maybe even  safe) functions to receive addresses
 of locals. Statically check that they never escape an address of a
 parameter. I think this is very interesting because it enlarges the
 common ground of D and SafeD.
This is a great idea if it can be implemented. Isn't escape analysis a pretty hard thing to get right, though, especially when you might not have the source code to the function being called?
Escape analysis is difficult when you don't have information about the functions you're passing the pointer to. For example: void fun(int* p) { if (condition) gun(p); } Now the problem is that fun's escape-or-not behavior depends on flow (i.e. condition) and on gun's escaping behavior. If we use safe and trusted to indicate unequivocally "no escape", then there is no analysis to be done - the hard part of the analysis has already been done manually by the user.
But then the safe or trusted function wouldn't be able to escape pointers to heap or static data segment memory either, if I understand this proposal
correctly.
 Yah. The question is to what extent is that necessary.
 Andrei
Too kludgey for me. I'd rather just see ref parameters get fixed and just don't allow taking the address of locals in safe functions. I'd say that, except in low-level systems programming that would probably not be safe for other reasons anyhow, there would be very few good if any good reasons to take the address of a local if reference tuples just worked.
Unfortunately it's more complicated than that. getopt takes pairs of strings and pointers. The strings don't necessarily have to be lvalues, so constraining getopt to only take references is not the right solution. Andrei
Nov 18 2009
parent =?ISO-8859-1?Q?Pelle_M=E5nsson?= <pelle.mansson gmail.com> writes:
Andrei Alexandrescu wrote:
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s 
 article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s 
 article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu 
 (SeeWebsiteForEmail erdani.org)'s article
 3. It was mentioned in this group that if getopt() does not work in
 SafeD, then SafeD may as well pack and go home. I agree. We need 
 to make
 it work. Three ideas discussed with Walter:
 * Allow taking addresses of locals, but in that case switch 
 allocation
 from stack to heap, just like with delegates. If we only do that in
 SafeD, behavior will be different than with regular D. In any 
 case, it's
 an inefficient proposition, particularly for getopt() which actually
 does not need to escape the addresses - just fills them up.
IMHO this is a terrible solution. SafeD should not cause major ripple
effects for
 pieces of code that don't want to use it.  I'm all for safe 
 defaults even if
 they're less efficient or less flexible, but if D starts 
 sacrificing performance
 or flexibility for safety **even when the programmer explicitly 
 asks it not
to**,
 then it will officially have become a bondage and discipline 
 language.

 Furthermore, as you point out, having the semantics of something 
 vary in subtle
 ways between SafeD and unsafe D is probably a recipe for confusion.


 * Allow  trusted (and maybe even  safe) functions to receive 
 addresses
 of locals. Statically check that they never escape an address of a
 parameter. I think this is very interesting because it enlarges the
 common ground of D and SafeD.
This is a great idea if it can be implemented. Isn't escape analysis a pretty hard thing to get right, though, especially when you might not have the source code to the function being called?
Escape analysis is difficult when you don't have information about the functions you're passing the pointer to. For example: void fun(int* p) { if (condition) gun(p); } Now the problem is that fun's escape-or-not behavior depends on flow (i.e. condition) and on gun's escaping behavior. If we use safe and trusted to indicate unequivocally "no escape", then there is no analysis to be done - the hard part of the analysis has already been done manually by the user.
But then the safe or trusted function wouldn't be able to escape pointers to heap or static data segment memory either, if I understand this proposal
correctly.
 Yah. The question is to what extent is that necessary.
 Andrei
Too kludgey for me. I'd rather just see ref parameters get fixed and just don't allow taking the address of locals in safe functions. I'd say that, except in low-level systems programming that would probably not be safe for other reasons anyhow, there would be very few good if any good reasons to take the address of a local if reference tuples just worked.
Unfortunately it's more complicated than that. getopt takes pairs of strings and pointers. The strings don't necessarily have to be lvalues, so constraining getopt to only take references is not the right solution. Andrei
How about allowing const references to rvalues?
Nov 20 2009
prev sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
Andrei Alexandrescu wrote:
 If we use  safe and  trusted to indicate unequivocally "no escape", then 
 there is no analysis to be done - the hard part of the analysis has 
 already been done manually by the user.
The problem then becomes: T[] foo(T[] t) { return t; } T[] bar() { T[3] a; return foo(a); }
Nov 19 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Walter Bright wrote:
 Andrei Alexandrescu wrote:
 If we use  safe and  trusted to indicate unequivocally "no escape", 
 then there is no analysis to be done - the hard part of the analysis 
 has already been done manually by the user.
The problem then becomes: T[] foo(T[] t) { return t; } T[] bar() { T[3] a; return foo(a); }
The "no escape" rule only applies to pointers, not arrays. Translating your example to pointers: T* foo(T* t) { return t; } T* bar() { T[] a; return foo(a.ptr); } Steve Schweighoffer has pointed out a while ago that the compiler cannot assume a scope of a returned value to be any larger than the scopes of its parameters. Your example and the example above are canonical and the most complicated analysis I know of in SafeD. It is admittedly a huge complication, but just something that you'll need to pay attention to when implementing. Andrei
Nov 19 2009
parent Walter Bright <newshound1 digitalmars.com> writes:
Andrei Alexandrescu wrote:
 Walter Bright wrote:
 Andrei Alexandrescu wrote:
 If we use  safe and  trusted to indicate unequivocally "no escape", 
 then there is no analysis to be done - the hard part of the analysis 
 has already been done manually by the user.
The problem then becomes: T[] foo(T[] t) { return t; } T[] bar() { T[3] a; return foo(a); }
The "no escape" rule only applies to pointers, not arrays. Translating your example to pointers: T* foo(T* t) { return t; } T* bar() { T[] a; return foo(a.ptr); } Steve Schweighoffer has pointed out a while ago that the compiler cannot assume a scope of a returned value to be any larger than the scopes of its parameters. Your example and the example above are canonical and the most complicated analysis I know of in SafeD. It is admittedly a huge complication, but just something that you'll need to pay attention to when implementing.
What if foo were a safe concat, like: T[] foo(T[] t, T[] s) { return t ~ s; } T[] bar(T[] s) { T[3] a; return foo(a, s); } That's perfectly safe, but how is the compiler not to give an error on that?
Nov 21 2009
prev sibling next sibling parent reply grauzone <none example.net> writes:
Andrei Alexandrescu wrote:
 3. It was mentioned in this group that if getopt() does not work in 
 SafeD, then SafeD may as well pack and go home. I agree. We need to make 
 it work. Three ideas discussed with Walter:
If that's such an issue, why don't you just change it and use a struct defined by the user? structs are natural name-value pairs that work at compile time, and they can always be returned from functions.
 4. Allow private members inside a template using the eponymous trick:
 
 template wyda(int x) {
    private enum geeba = x / 2;
    alias geeba wyda;
 }
Why is this important? Apart from this... Whatever happened to the concurrency stuff? Do you really want users to deal with the broken and incomplete implementation right now? What about the issues of constructing immutable data? (Eh, are we supposed to cast from mutable data, and hope everything goes right?) Right now, multithreaded programming in D2 is probably even more a pain than in C++. Also, you should fix the auto-flattening of tuples before it's too late. I think everyone agrees that auto-flattening is a bad idea, and that tuples should be nestable. Flattening can be done manually with an unary operator. (Introducing sane tuples (e.g. unify type and value tuples, sane and short syntax, and all that) can wait for later if it must. Introducing these can be downwards compatible, I hope.)
Nov 18 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
grauzone wrote:
 Andrei Alexandrescu wrote:
 3. It was mentioned in this group that if getopt() does not work in 
 SafeD, then SafeD may as well pack and go home. I agree. We need to 
 make it work. Three ideas discussed with Walter:
If that's such an issue, why don't you just change it and use a struct defined by the user? structs are natural name-value pairs that work at compile time, and they can always be returned from functions.
 4. Allow private members inside a template using the eponymous trick:

 template wyda(int x) {
    private enum geeba = x / 2;
    alias geeba wyda;
 }
Why is this important? Apart from this... Whatever happened to the concurrency stuff? Do you really want users to deal with the broken and incomplete implementation right now? What about the issues of constructing immutable data? (Eh, are we supposed to cast from mutable data, and hope everything goes right?) Right now, multithreaded programming in D2 is probably even more a pain than in C++. Also, you should fix the auto-flattening of tuples before it's too late. I think everyone agrees that auto-flattening is a bad idea, and that tuples should be nestable. Flattening can be done manually with an unary operator. (Introducing sane tuples (e.g. unify type and value tuples, sane and short syntax, and all that) can wait for later if it must. Introducing these can be downwards compatible, I hope.)
Non-flattening should be on the list but I am very afraid the solution would take a long time to design, implement, and debug. I must discuss this with Walter. Andrei
Nov 18 2009
next sibling parent reply Chad J <chadjoan __spam.is.bad__gmail.com> writes:
Andrei Alexandrescu wrote:
 grauzone wrote:
 Also, you should fix the auto-flattening of tuples before it's too
 late. I think everyone agrees that auto-flattening is a bad idea, and
 that tuples should be nestable. Flattening can be done manually with
 an unary operator.

 (Introducing sane tuples (e.g. unify type and value tuples, sane and
 short syntax, and all that) can wait for later if it must. Introducing
 these can be downwards compatible, I hope.)
Non-flattening should be on the list but I am very afraid the solution would take a long time to design, implement, and debug. I must discuss this with Walter. Andrei
Might I suggest a daring stop-gap: kill tuples altogether. Then we can implement them later correctly and it won't break backwards compatibility. Ideally it isn't an all-or-nothing proposition either. Maybe we can just kill the parts that are bad. Like disallow tuples-of-tuples, but allow tuples that are already flat. - Chad
Nov 19 2009
next sibling parent dsimcha <dsimcha yahoo.com> writes:
== Quote from Chad J (chadjoan __spam.is.bad__gmail.com)'s article
 Andrei Alexandrescu wrote:
 grauzone wrote:
 Also, you should fix the auto-flattening of tuples before it's too
 late. I think everyone agrees that auto-flattening is a bad idea, and
 that tuples should be nestable. Flattening can be done manually with
 an unary operator.

 (Introducing sane tuples (e.g. unify type and value tuples, sane and
 short syntax, and all that) can wait for later if it must. Introducing
 these can be downwards compatible, I hope.)
Non-flattening should be on the list but I am very afraid the solution would take a long time to design, implement, and debug. I must discuss this with Walter. Andrei
Might I suggest a daring stop-gap: kill tuples altogether. Then we can implement them later correctly and it won't break backwards compatibility. Ideally it isn't an all-or-nothing proposition either. Maybe we can just kill the parts that are bad. Like disallow tuples-of-tuples, but allow tuples that are already flat. - Chad
But that would destroy most of the metaprogramming capabilities of D.
Nov 19 2009
prev sibling parent Lutger <lutger.blijdestijn gmail.com> writes:
Chad J wrote:

 Andrei Alexandrescu wrote:
 grauzone wrote:
 Also, you should fix the auto-flattening of tuples before it's too
 late. I think everyone agrees that auto-flattening is a bad idea, and
 that tuples should be nestable. Flattening can be done manually with
 an unary operator.

 (Introducing sane tuples (e.g. unify type and value tuples, sane and
 short syntax, and all that) can wait for later if it must. Introducing
 these can be downwards compatible, I hope.)
Non-flattening should be on the list but I am very afraid the solution would take a long time to design, implement, and debug. I must discuss this with Walter. Andrei
Might I suggest a daring stop-gap: kill tuples altogether. Then we can implement them later correctly and it won't break backwards compatibility. Ideally it isn't an all-or-nothing proposition either. Maybe we can just kill the parts that are bad. Like disallow tuples-of-tuples, but allow tuples that are already flat. - Chad
By the time you would have restored phobos to about 50% of it's usefulness you could have implemented non-flattening tuples ten times over. I would switch back to D1 :)
Nov 21 2009
prev sibling parent reply grauzone <none example.net> writes:
Andrei Alexandrescu wrote:
 grauzone wrote:
 Andrei Alexandrescu wrote:
 3. It was mentioned in this group that if getopt() does not work in 
 SafeD, then SafeD may as well pack and go home. I agree. We need to 
 make it work. Three ideas discussed with Walter:
If that's such an issue, why don't you just change it and use a struct defined by the user? structs are natural name-value pairs that work at compile time, and they can always be returned from functions.
No reply to this one? I would have been curious.
Nov 19 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
grauzone wrote:
 Andrei Alexandrescu wrote:
 grauzone wrote:
 Andrei Alexandrescu wrote:
 3. It was mentioned in this group that if getopt() does not work in 
 SafeD, then SafeD may as well pack and go home. I agree. We need to 
 make it work. Three ideas discussed with Walter:
If that's such an issue, why don't you just change it and use a struct defined by the user? structs are natural name-value pairs that work at compile time, and they can always be returned from functions.
No reply to this one? I would have been curious.
You mean use a struct for the string-value pair? A struct cannot have a ref member, so it would still need to store a pointer. (But maybe I misunderstood the point.) Andrei
Nov 19 2009
parent reply grauzone <none example.net> writes:
Andrei Alexandrescu wrote:
 You mean use a struct for the string-value pair? A struct cannot have a 
 ref member, so it would still need to store a pointer. (But maybe I 
 misunderstood the point.)
Like this: void main(string[] commandline) { struct Args { string param1 = "can even have default arguments"; int param2; } Args args = getopt!(Args)(commandline); writefln("param1 = %s", args.param1); } No pointers. Instead of returning it, struct could be passed by ref, too.
Nov 19 2009
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
grauzone wrote:
 Andrei Alexandrescu wrote:
 You mean use a struct for the string-value pair? A struct cannot have 
 a ref member, so it would still need to store a pointer. (But maybe I 
 misunderstood the point.)
Like this: void main(string[] commandline) { struct Args { string param1 = "can even have default arguments"; int param2; } Args args = getopt!(Args)(commandline); writefln("param1 = %s", args.param1); } No pointers. Instead of returning it, struct could be passed by ref, too.
I think that's a good idea, but we still need a means to express the name of the parameters e.g. for mapping "--multi-word-option" to multiWordOption. Then I fear things will inevitably get thicker, and simplicity was the main attractiveness of getopt. (FWIW, getopt is somewhat dear to me; it was one of the first pieces of code I wrote in D. I'd dreamed for such a simple facility for years, and I was very glad that it was doable in D. If getopt starts becoming bigger and more nontrivial to use, I think some of the elegance of its initial design would be lost.) Andrei
Nov 19 2009
prev sibling parent Walter Bright <newshound1 digitalmars.com> writes:
grauzone wrote:
 Like this:
 
 void main(string[] commandline) {
     struct Args {
         string param1 = "can even have default arguments";
         int param2;
     }
     Args args = getopt!(Args)(commandline);
     writefln("param1 = %s", args.param1);
 }
 
 No pointers. Instead of returning it, struct could be passed by ref, too.
One nice thing about your proposal is the encapsulating of all the parameters into one struct, instead of them being random variables. The downside is it spreads out the call to getopt.
Nov 19 2009
prev sibling next sibling parent reply Justin Johansson <no spam.com> writes:
 6. There must be many things I forgot to mention, or that cause grief to 
 many of us. Please add to/comment on this list.
 
Sorry I'm not familiar with any prior discussion on this or even where it's at (basically I'm in D1 world), but what about constructors for structs Justin Johansson
Nov 18 2009
parent reply "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
Justin Johansson wrote:
 6. There must be many things I forgot to mention, or that cause grief 
 to many of us. Please add to/comment on this list.
Sorry I'm not familiar with any prior discussion on this or even where it's at (basically I'm in D1 world), but what about constructors for structs
That's been in D2 for a long time. :) http://www.digitalmars.com/d/2.0/struct.html#Struct-Constructor -Lars
Nov 19 2009
parent Justin Johansson <no spam.com> writes:
Lars T. Kyllingstad wrote:
 Justin Johansson wrote:
 6. There must be many things I forgot to mention, or that cause grief 
 to many of us. Please add to/comment on this list.
Sorry I'm not familiar with any prior discussion on this or even where it's at (basically I'm in D1 world), but what about constructors for structs
That's been in D2 for a long time. :) http://www.digitalmars.com/d/2.0/struct.html#Struct-Constructor -Lars
Thanks Lars. Brilliant. Guess I should have done my homework.
Nov 19 2009
prev sibling next sibling parent reply Don <nospam nospam.com> writes:
Andrei Alexandrescu wrote:
 We're entering the finale of D2 and I want to keep a short list of 
 things that must be done and integrated in the release. It is clearly 
 understood by all of us that there are many things that could and 
 probably should be done.
 
 1. Currently Walter and Don are diligently fixing the problems marked on 
 the current manuscript.
 
 2. User-defined operators must be revamped.
Should opIndex and opSlice be merged? This would be simpler, and would allow multi-dimensional slicing. Probably the simplest way to do this would be to use fixed length arrays of length 2 for slices. So, for example, if the indices are integers, then opIndex(int x) { } is the 1-D index, and opIndex(int[2] x) {} is a slice from x[0] to x[1], which exactly corresponding to the current opSlice(x[0]..x[1]). Since fixed-length arrays are now passed by value (yay!) this would be just as efficient as the current method. It'd be simple to implement. Downsides: (1) Arguably not terribly intuitive. (2) This would not let you have a slicing from A..A, while simultaneously allowing indexing by A[2]. But I don't think that makes sense anyway -- what would $ return? (Note that it doesn't stop you from having indexes of type A[2], and slice from A[2]..A[2], which might be important for containers of 2D vectors. It's only the case where indexing and slicing are different types, which is prohibited). This is the only solution I have which doesn't introducing more coupling between the language and standard library. Another possible solution is to define a special Slice struct in std.object; downside is that it creates yet another pseudo-keyword. A third solution would be to use tuples; currently I think it's impossible, but if tuple auto-flattening is removed there may be a chance. Even so, it might be a little clumsy to use. A fourth solution is to retain the status quo, and not provide the multi-dimensional slicing feature <g>. This is actually about the same amount of work as option (1), since opDollar would need to work for opSlice (I only implemented it for opIndex).
Nov 19 2009
next sibling parent reply "Denis Koroskin" <2korden gmail.com> writes:
On Thu, 19 Nov 2009 11:53:51 +0300, Don <nospam nospam.com> wrote:

 Andrei Alexandrescu wrote:
 We're entering the finale of D2 and I want to keep a short list of  
 things that must be done and integrated in the release. It is clearly  
 understood by all of us that there are many things that could and  
 probably should be done.
  1. Currently Walter and Don are diligently fixing the problems marked  
 on the current manuscript.
  2. User-defined operators must be revamped.
Should opIndex and opSlice be merged? This would be simpler, and would allow multi-dimensional slicing. Probably the simplest way to do this would be to use fixed length arrays of length 2 for slices. So, for example, if the indices are integers, then opIndex(int x) { } is the 1-D index, and opIndex(int[2] x) {} is a slice from x[0] to x[1], which exactly corresponding to the current opSlice(x[0]..x[1]). Since fixed-length arrays are now passed by value (yay!) this would be just as efficient as the current method. It'd be simple to implement. Downsides: (1) Arguably not terribly intuitive. (2) This would not let you have a slicing from A..A, while simultaneously allowing indexing by A[2]. But I don't think that makes sense anyway -- what would $ return? (Note that it doesn't stop you from having indexes of type A[2], and slice from A[2]..A[2], which might be important for containers of 2D vectors. It's only the case where indexing and slicing are different types, which is prohibited). This is the only solution I have which doesn't introducing more coupling between the language and standard library. Another possible solution is to define a special Slice struct in std.object; downside is that it creates yet another pseudo-keyword. A third solution would be to use tuples; currently I think it's impossible, but if tuple auto-flattening is removed there may be a chance. Even so, it might be a little clumsy to use. A fourth solution is to retain the status quo, and not provide the multi-dimensional slicing feature <g>. This is actually about the same amount of work as option (1), since opDollar would need to work for opSlice (I only implemented it for opIndex).
I'd like for a..b to be auto-magically rewritten into range(a, b). This way you can distinguish between indexing and slicing: struct Range { T lower, upper; // front, back, popFront, popBack, empty } Range!(T) range(T)(T lower, T upper) { return Range!(T)(lower, upper); } auto opIndex(Range!(int) range) { // corresponds to foo[a..b]; } auto opIndex(int index1, int index2) { // corresponds to foo[a, b]; } This will also get rid of a special case inside foreach: foreach (i; a..b) { } since a..b will be converted into a range, generic rules would apply to it.
Nov 19 2009
parent reply Don <nospam nospam.com> writes:
Denis Koroskin wrote:
 On Thu, 19 Nov 2009 11:53:51 +0300, Don <nospam nospam.com> wrote:
 
 Andrei Alexandrescu wrote:
 We're entering the finale of D2 and I want to keep a short list of 
 things that must be done and integrated in the release. It is clearly 
 understood by all of us that there are many things that could and 
 probably should be done.
  1. Currently Walter and Don are diligently fixing the problems 
 marked on the current manuscript.
  2. User-defined operators must be revamped.
Should opIndex and opSlice be merged? This would be simpler, and would allow multi-dimensional slicing. Probably the simplest way to do this would be to use fixed length arrays of length 2 for slices. So, for example, if the indices are integers, then opIndex(int x) { } is the 1-D index, and opIndex(int[2] x) {} is a slice from x[0] to x[1], which exactly corresponding to the current opSlice(x[0]..x[1]). Since fixed-length arrays are now passed by value (yay!) this would be just as efficient as the current method. It'd be simple to implement. Downsides: (1) Arguably not terribly intuitive. (2) This would not let you have a slicing from A..A, while simultaneously allowing indexing by A[2]. But I don't think that makes sense anyway -- what would $ return? (Note that it doesn't stop you from having indexes of type A[2], and slice from A[2]..A[2], which might be important for containers of 2D vectors. It's only the case where indexing and slicing are different types, which is prohibited). This is the only solution I have which doesn't introducing more coupling between the language and standard library. Another possible solution is to define a special Slice struct in std.object; downside is that it creates yet another pseudo-keyword. A third solution would be to use tuples; currently I think it's impossible, but if tuple auto-flattening is removed there may be a chance. Even so, it might be a little clumsy to use. A fourth solution is to retain the status quo, and not provide the multi-dimensional slicing feature <g>. This is actually about the same amount of work as option (1), since opDollar would need to work for opSlice (I only implemented it for opIndex).
I'd like for a..b to be auto-magically rewritten into range(a, b).
That's solution (2), but applied everywhere. One nice thing about that, is it would allow slices as function parameters. I've wanted that for: testall(&sin, &asin, -PI..PI); The downside is that you have to add this slice/range creature into std.object. (BTW, note that $ would still only be allowable inside opIndex expressions. I can't see how it could be made to work in general, it's too uncertain which dimension you are referring to, especially in a variadic function).
Nov 19 2009
parent bearophile <bearophileHUGS lycos.com> writes:
Don:
 The downside is that you have to add this slice/range creature into 
 std.object.
I think this is acceptable. Slice structs look important enough. Adding a stride then looks simpler: Range(start, stop, stride). Bye, bearophile
Nov 19 2009
prev sibling next sibling parent "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
Don wrote:
 Andrei Alexandrescu wrote:
 We're entering the finale of D2 and I want to keep a short list of 
 things that must be done and integrated in the release. It is clearly 
 understood by all of us that there are many things that could and 
 probably should be done.

 1. Currently Walter and Don are diligently fixing the problems marked 
 on the current manuscript.

 2. User-defined operators must be revamped.
Should opIndex and opSlice be merged? This would be simpler, and would allow multi-dimensional slicing.
Yes, please. :)
 Probably the simplest way to do this would be to use fixed length arrays 
  of length 2 for slices.
 So, for example, if the indices are integers, then
 opIndex(int x) { } is the 1-D index, and
 opIndex(int[2] x) {} is a slice from x[0] to x[1],
 which exactly corresponding to the current opSlice(x[0]..x[1]).
 
 Since fixed-length arrays are now passed by value (yay!) this would be 
 just as efficient as the current method. It'd be simple to implement.
 
 Downsides: (1) Arguably not terribly intuitive.
 [...]
I think this is the only way to do it, and I don't see why it's less intuitive than anything else. If it helps, you can always define alias size_t[2] slice_t; (or something similar) in object.d. // Submatrix slice real opIndex(slice_t rows, slice_t cols) { ... } Doesn't look bad at all. -Lars
Nov 19 2009
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 19 Nov 2009 03:53:51 -0500, Don <nospam nospam.com> wrote:

 Andrei Alexandrescu wrote:
 We're entering the finale of D2 and I want to keep a short list of  
 things that must be done and integrated in the release. It is clearly  
 understood by all of us that there are many things that could and  
 probably should be done.
  1. Currently Walter and Don are diligently fixing the problems marked  
 on the current manuscript.
  2. User-defined operators must be revamped.
Should opIndex and opSlice be merged? This would be simpler, and would allow multi-dimensional slicing. Probably the simplest way to do this would be to use fixed length arrays of length 2 for slices. So, for example, if the indices are integers, then opIndex(int x) { } is the 1-D index, and opIndex(int[2] x) {} is a slice from x[0] to x[1], which exactly corresponding to the current opSlice(x[0]..x[1]).
I hope you still mean to allow arguments other than int. Also, how does this work with Andrei's "opBinary" proposal? -Steve
Nov 19 2009
parent reply Don <nospam nospam.com> writes:
Steven Schveighoffer wrote:
 On Thu, 19 Nov 2009 03:53:51 -0500, Don <nospam nospam.com> wrote:
 
 Andrei Alexandrescu wrote:
 We're entering the finale of D2 and I want to keep a short list of 
 things that must be done and integrated in the release. It is clearly 
 understood by all of us that there are many things that could and 
 probably should be done.
  1. Currently Walter and Don are diligently fixing the problems 
 marked on the current manuscript.
  2. User-defined operators must be revamped.
Should opIndex and opSlice be merged? This would be simpler, and would allow multi-dimensional slicing. Probably the simplest way to do this would be to use fixed length arrays of length 2 for slices. So, for example, if the indices are integers, then opIndex(int x) { } is the 1-D index, and opIndex(int[2] x) {} is a slice from x[0] to x[1], which exactly corresponding to the current opSlice(x[0]..x[1]).
I hope you still mean to allow arguments other than int.
Read it. "So, _for example_, if the indices are integers,".
 Also, how does this work with Andrei's "opBinary" proposal?
I don't know -- I'm not proposing a solution to the indexing-and-slicing-expression issues. Still, combining indexing and slicing helps a little. But I don't yet know if the opBinary concept will work. I think the problem of verbosity in operator overloads (opAdd, opMul, ... all being nearly the same) is very unimportant and shouldn't be the focus of attention. Two things matter: (1) expressivity; and (2) performance. If you don't have these two, your verbosity will be shot to pieces anyway. This proposal, together with opDollar(), closes the last remaining element of (1).
Nov 19 2009
parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 19 Nov 2009 08:09:14 -0500, Don <nospam nospam.com> wrote:

 Steven Schveighoffer wrote:
 On Thu, 19 Nov 2009 03:53:51 -0500, Don <nospam nospam.com> wrote:

 Andrei Alexandrescu wrote:
 We're entering the finale of D2 and I want to keep a short list of  
 things that must be done and integrated in the release. It is clearly  
 understood by all of us that there are many things that could and  
 probably should be done.
  1. Currently Walter and Don are diligently fixing the problems  
 marked on the current manuscript.
  2. User-defined operators must be revamped.
Should opIndex and opSlice be merged? This would be simpler, and would allow multi-dimensional slicing. Probably the simplest way to do this would be to use fixed length arrays of length 2 for slices. So, for example, if the indices are integers, then opIndex(int x) { } is the 1-D index, and opIndex(int[2] x) {} is a slice from x[0] to x[1], which exactly corresponding to the current opSlice(x[0]..x[1]).
I hope you still mean to allow arguments other than int.
Read it. "So, _for example_, if the indices are integers,".
D'oh, sorry for the noise :(
 Also, how does this work with Andrei's "opBinary" proposal?
I don't know -- I'm not proposing a solution to the indexing-and-slicing-expression issues. Still, combining indexing and slicing helps a little. But I don't yet know if the opBinary concept will work. I think the problem of verbosity in operator overloads (opAdd, opMul, ... all being nearly the same) is very unimportant and shouldn't be the focus of attention. Two things matter: (1) expressivity; and (2) performance. If you don't have these two, your verbosity will be shot to pieces anyway. This proposal, together with opDollar(), closes the last remaining element of (1).
I agree. -Steve
Nov 19 2009
prev sibling next sibling parent reply "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
Andrei Alexandrescu wrote:
 We're entering the finale of D2 and I want to keep a short list of 
 things that must be done and integrated in the release. It is clearly 
 understood by all of us that there are many things that could and 
 probably should be done.
 
 1. Currently Walter and Don are diligently fixing the problems marked on 
 the current manuscript.
 
 2. User-defined operators must be revamped. Fortunately Don already put 
 in an important piece of functionality (opDollar). What we're looking at 
 is a two-pronged attack motivated by Don's proposal:
 
 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7
 
 The two prongs are:
 
 * Encode operators by compile-time strings. For example, instead of the 
 plethora of opAdd, opMul, ..., we'd have this:
 
 T opBinary(string op)(T rhs) { ... }
 
 The string is "+", "*", etc. We need to design what happens with 
 read-modify-write operators like "+=" (should they be dispatch to a 
 different function? etc.) and also what happens with index-and-modify 
 operators like "[]=", "[]+=" etc. Should we go with proxies? Absorb them 
 in opBinary? Define another dedicated method? etc.
 
 * Loop fusion that generalizes array-wise operations. This idea of 
 Walter is, I think, very good because it generalizes and democratizes 
 "magic". The idea is that, if you do
 
 a = b + c;
 
 and b + c does not make sense but b and c are ranges for which a.front = 
 b.front + c.front does make sense, to automatically add the iteration 
 paraphernalia.
 
 3. It was mentioned in this group that if getopt() does not work in 
 SafeD, then SafeD may as well pack and go home. I agree. We need to make 
 it work. Three ideas discussed with Walter:
 
 * Allow taking addresses of locals, but in that case switch allocation 
 from stack to heap, just like with delegates. If we only do that in 
 SafeD, behavior will be different than with regular D. In any case, it's 
 an inefficient proposition, particularly for getopt() which actually 
 does not need to escape the addresses - just fills them up.
 
 * Allow  trusted (and maybe even  safe) functions to receive addresses 
 of locals. Statically check that they never escape an address of a 
 parameter. I think this is very interesting because it enlarges the 
 common ground of D and SafeD.
 
 * Figure out a way to reconcile "ref" with variadics. This is the actual 
 reason why getopt chose to traffic in addresses, and fixing it is the 
 logical choice and my personal favorite.
 
 4. Allow private members inside a template using the eponymous trick:
 
 template wyda(int x) {
    private enum geeba = x / 2;
    alias geeba wyda;
 }
 
 The names inside an eponymous template are only accessible to the 
 current instantiation. For example, wyda!5 cannot access wyda!(4).geeba, 
 only its own geeba. That we we elegantly avoid the issue "where is this 
 symbol looked up?"
 
 5. Chain exceptions instead of having a recurrent exception terminate 
 the program. I'll dedicate a separate post to this.
 
 6. There must be many things I forgot to mention, or that cause grief to 
 many of us. Please add to/comment on this list.
7. opPow() 8. The magic "meta" namespace, aka. getting rid of is(typeof({...})) and __traits(...). -Lars
Nov 19 2009
parent Bill Baxter <wbaxter gmail.com> writes:
On Thu, Nov 19, 2009 at 1:23 AM, Lars T. Kyllingstad
<public kyllingen.nospamnet> wrote:

 7. opPow()
votes++
 8. The magic "meta" namespace, aka. getting rid of is(typeof({...})) and
 __traits(...).
votes++ --bb
Nov 19 2009
prev sibling next sibling parent Chad J <chadjoan __spam.is.bad__gmail.com> writes:
Andrei Alexandrescu wrote:
 * Encode operators by compile-time strings. For example, instead of the
 plethora of opAdd, opMul, ..., we'd have this:
 
 T opBinary(string op)(T rhs) { ... }
 
 The string is "+", "*", etc. We need to design what happens with
 read-modify-write operators like "+=" (should they be dispatch to a
 different function? etc.) and also what happens with index-and-modify
 operators like "[]=", "[]+=" etc. Should we go with proxies? Absorb them
 in opBinary? Define another dedicated method? etc.
 
Index-and-modify operators shouldn't exist. That's the wrong place for it. Indexing operators act like properties. It is up to the types that they get and set to define the opXxxAssign overloads. http://www.digitalmars.com/d/archives/digitalmars/D/A_possible_solution_for_the_opIndexXxxAssign_morass_98143.html#N98176 - Chad
Nov 19 2009
prev sibling next sibling parent bearophile <bearophileHUGS lycos.com> writes:
Andrei Alexandrescu:
 Static function parameters
Walter tried to implement them and ran into a number of odd semantic issues.
I'd like to know more about those semantic issues, I am curious. Bye, bearophile
Nov 19 2009
prev sibling next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 18 Nov 2009 18:14:08 -0500, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 We're entering the finale of D2 and I want to keep a short list of  
 things that must be done and integrated in the release. It is clearly  
 understood by all of us that there are many things that could and  
 probably should be done.

 1. Currently Walter and Don are diligently fixing the problems marked on  
 the current manuscript.

 2. User-defined operators must be revamped. Fortunately Don already put  
 in an important piece of functionality (opDollar). What we're looking at  
 is a two-pronged attack motivated by Don's proposal:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7

 The two prongs are:

 * Encode operators by compile-time strings. For example, instead of the  
 plethora of opAdd, opMul, ..., we'd have this:

 T opBinary(string op)(T rhs) { ... }

 The string is "+", "*", etc. We need to design what happens with  
 read-modify-write operators like "+=" (should they be dispatch to a  
 different function? etc.) and also what happens with index-and-modify  
 operators like "[]=", "[]+=" etc. Should we go with proxies? Absorb them  
 in opBinary? Define another dedicated method? etc.
I don't like this. The only useful thing I can see is if you wanted to write less code to do an operation on a wrapper aggregate, such as an array, where you could define all binary operations with a single mixin. Other than that, it munges together all binary operations into a single function, when all those operations are different it: 1) prevents code separation from things that are considered separately 2) makes operators non-virtual, which can be solved by a thunk, but that seems like a lot of boilerplate code that will just cause bloat 3) If you derive from a class that implements an operator, and you want to make that operator virtual, it will be impossible 4) auto-generated documentation is going to *really* suck 5) you can't define operators on interfaces, or if you do, it looks ridiculous (a thunk function that dispatches to the virtual methods). 6) implementing a new operator in a derived class is virtually impossible (no pun intended). I imagine that dcollections for example will be *very* hard to write with this change. Seems like you are trying to solve a very focused problem without looking at the new problems your solution will cause outside that domain. Can we do something like how opApply/ranges resolves? I.e. the compiler tries doing opAdd or opMul or whatever, and if that doesn't exist, try opBinary("+").
 3. It was mentioned in this group that if getopt() does not work in  
 SafeD, then SafeD may as well pack and go home. I agree. We need to make  
 it work. Three ideas discussed with Walter:

 * Allow taking addresses of locals, but in that case switch allocation  
 from stack to heap, just like with delegates. If we only do that in  
 SafeD, behavior will be different than with regular D. In any case, it's  
 an inefficient proposition, particularly for getopt() which actually  
 does not need to escape the addresses - just fills them up.
Perhaps, but getopt is probably not the poster child for optimizing performance -- you most likely call it once, changing that single application to use heap data isn't going to make a difference.
 * Allow  trusted (and maybe even  safe) functions to receive addresses  
 of locals. Statically check that they never escape an address of a  
 parameter. I think this is very interesting because it enlarges the  
 common ground of D and SafeD.
I think allowing calling trusted or safe functions with addresses to locals is no good for safe functions (i.e. a safe function calls a trusted function with an address to a local without heap-allocating). Remember the "returning a parameter array" problem...
 * Figure out a way to reconcile "ref" with variadics. This is the actual  
 reason why getopt chose to traffic in addresses, and fixing it is the  
 logical choice and my personal favorite.
This sounds like the best choice.
 6. There must be many things I forgot to mention, or that cause grief to  
 many of us. Please add to/comment on this list.
I know it's not part of the spec, but I'm not sure if you mention the array "data stomping" problem in the book. If not, the MRU cache needs to be implemented. -Steve
Nov 19 2009
next sibling parent dsimcha <dsimcha yahoo.com> writes:
== Quote from Steven Schveighoffer (schveiguy yahoo.com)'s article
 On Wed, 18 Nov 2009 18:14:08 -0500, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:
 We're entering the finale of D2 and I want to keep a short list of
 things that must be done and integrated in the release. It is clearly
 understood by all of us that there are many things that could and
 probably should be done.

 1. Currently Walter and Don are diligently fixing the problems marked on
 the current manuscript.

 2. User-defined operators must be revamped. Fortunately Don already put
 in an important piece of functionality (opDollar). What we're looking at
 is a two-pronged attack motivated by Don's proposal:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7

 The two prongs are:

 * Encode operators by compile-time strings. For example, instead of the
 plethora of opAdd, opMul, ..., we'd have this:

 T opBinary(string op)(T rhs) { ... }

 The string is "+", "*", etc. We need to design what happens with
 read-modify-write operators like "+=" (should they be dispatch to a
 different function? etc.) and also what happens with index-and-modify
 operators like "[]=", "[]+=" etc. Should we go with proxies? Absorb them
 in opBinary? Define another dedicated method? etc.
I don't like this. The only useful thing I can see is if you wanted to write less code to do an operation on a wrapper aggregate, such as an array, where you could define all binary operations with a single mixin. Other than that, it munges together all binary operations into a single function, when all those operations are different it: 1) prevents code separation from things that are considered separately 2) makes operators non-virtual, which can be solved by a thunk, but that seems like a lot of boilerplate code that will just cause bloat 3) If you derive from a class that implements an operator, and you want to make that operator virtual, it will be impossible 4) auto-generated documentation is going to *really* suck 5) you can't define operators on interfaces, or if you do, it looks ridiculous (a thunk function that dispatches to the virtual methods). 6) implementing a new operator in a derived class is virtually impossible (no pun intended). I imagine that dcollections for example will be *very* hard to write with this change. Seems like you are trying to solve a very focused problem without looking at the new problems your solution will cause outside that domain. Can we do something like how opApply/ranges resolves? I.e. the compiler tries doing opAdd or opMul or whatever, and if that doesn't exist, try opBinary("+").
This sounds like another candidate for inclusion in a std.mixins module. We could make a mixin that gives you back the old behavior for those cases were you need it: enum string oldOperatorOverloading = q{ T opBinary(string op)(T rhs) { static if(op == "+" && __traits(compiles, this.opAdd(T.init)) { return opAdd(rhs); } } // etc. }; Usage: class Foo { mixin(oldOperatorOverloading); Foo opAdd(Foo rhs) { /* do stuff. */ } }
Nov 19 2009
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Steven Schveighoffer wrote:
 On Wed, 18 Nov 2009 18:14:08 -0500, Andrei Alexandrescu 
 <SeeWebsiteForEmail erdani.org> wrote:
 
 We're entering the finale of D2 and I want to keep a short list of 
 things that must be done and integrated in the release. It is clearly 
 understood by all of us that there are many things that could and 
 probably should be done.

 1. Currently Walter and Don are diligently fixing the problems marked 
 on the current manuscript.

 2. User-defined operators must be revamped. Fortunately Don already 
 put in an important piece of functionality (opDollar). What we're 
 looking at is a two-pronged attack motivated by Don's proposal:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7

 The two prongs are:

 * Encode operators by compile-time strings. For example, instead of 
 the plethora of opAdd, opMul, ..., we'd have this:

 T opBinary(string op)(T rhs) { ... }

 The string is "+", "*", etc. We need to design what happens with 
 read-modify-write operators like "+=" (should they be dispatch to a 
 different function? etc.) and also what happens with index-and-modify 
 operators like "[]=", "[]+=" etc. Should we go with proxies? Absorb 
 them in opBinary? Define another dedicated method? etc.
I don't like this. The only useful thing I can see is if you wanted to write less code to do an operation on a wrapper aggregate, such as an array, where you could define all binary operations with a single mixin. Other than that, it munges together all binary operations into a single function, when all those operations are different it: 1) prevents code separation from things that are considered separately
(I'll retort inline for each point.) That's quite exactly the opposite of what my experience with C++ and D operator overloading suggests: most of the time (a) I need to overload operators in large groups, (b) I need to do virtually the same actions for each operator in a group. Note that with opBinary you have unprecedented flexibility on how you want to group operators. Consider: struct A { A opBinary(string op)(A rhs) if (op == "+" || op == "-" || op == "*" || op == "/" || op == "^^") { ... } A opBinary(string op)(A rhs) if (op == "~") { ... } ... } So anyway I contend that your argument is not correct. The "if" clause allows you to separate code for things that are considered separately. So essentially you can do things with one function per operator if you so wanted. Correct?
 2) makes operators non-virtual, which can be solved by a thunk, but that 
 seems like a lot of boilerplate code that will just cause bloat
Bloat of source or bloat of binary code? I don't know about the latter, but the former is actually nothing to worry about - it's easier to define an interface or a mixin to convert from the proposed approach to the old approach, than vice versa.
 3) If you derive from a class that implements an operator, and you want 
 to make that operator virtual, it will be impossible
It means that base class didn't mean for that function to make the operator overridable. If they wanted to make it configurable, they would have forwarded the operator to a virtual function.
 4) auto-generated documentation is going to *really* suck
Agreed.
 5) you can't define operators on interfaces, or if you do, it looks 
 ridiculous (a thunk function that dispatches to the virtual methods).
interface Ridiculous { // Final functions in interfaces are allowed per TDPL Ridiculous opBinary(string op)(Ridiculous rhs) { return opAdd(rhs); } // Implement this Ridiculous opAdd(Ridiculous); } You can group things as you wish and combine virtual calls with string comparisons if that helps: interface Ridiculous { // Final functions in interfaces are allowed per TDPL Ridiculous opArith(string op)(Ridiculous rhs) if (op == "+" || op == "-" || op == "*" || op == "/" || op == "^^") { return opArith(op, rhs); } // Implement this Ridiculous opArith(string, Ridiculous); }
 6) implementing a new operator in a derived class is virtually 
 impossible (no pun intended).
class Base { Base opBinary(string op)(Base rhs) if (op == "+") { ... } } class Derived : Base { Derived opBinary(string op)(Derived rhs) if (op == "-") { ... } } When you do so, you retain the advantage of grouping operators together (I think it's most likely that Base defines operators of one kind e.g. arithmetic and Derived defines operators of a different kind e.g. logic or catenation). Add thunking as you need and you're good to go.
 I imagine that dcollections for example will be *very* hard to write 
 with this change.
I hope my arguments above convinced you to the contrary.
 Seems like you are trying to solve a very focused problem without 
 looking at the new problems your solution will cause outside that domain.
You are correct in that I'm trying to smooth things primarily for structs. But I'll say that the templated approach is no slouch and can accommodate classes with virtual functions very capably, even though it is a bit more work than before. One question is whether it's more often to overload operators for structs vs. classes. I imagine dcollections defines catenation and slicing, but not the bulk of operators. But the vast majority of operator overloading application is with value types as far as I can tell.
 Can we do something like how opApply/ranges resolves? I.e. the compiler 
 tries doing opAdd or opMul or whatever, and if that doesn't exist, try 
 opBinary("+").
I wouldn't want to have too many layers that do essentially the same thing.
 3. It was mentioned in this group that if getopt() does not work in 
 SafeD, then SafeD may as well pack and go home. I agree. We need to 
 make it work. Three ideas discussed with Walter:

 * Allow taking addresses of locals, but in that case switch allocation 
 from stack to heap, just like with delegates. If we only do that in 
 SafeD, behavior will be different than with regular D. In any case, 
 it's an inefficient proposition, particularly for getopt() which 
 actually does not need to escape the addresses - just fills them up.
Perhaps, but getopt is probably not the poster child for optimizing performance -- you most likely call it once, changing that single application to use heap data isn't going to make a difference.
I agree. My fear is that getopt is only an example of a class of functions.
 * Allow  trusted (and maybe even  safe) functions to receive addresses 
 of locals. Statically check that they never escape an address of a 
 parameter. I think this is very interesting because it enlarges the 
 common ground of D and SafeD.
I think allowing calling trusted or safe functions with addresses to locals is no good for safe functions (i.e. a safe function calls a trusted function with an address to a local without heap-allocating). Remember the "returning a parameter array" problem...
I've been thinking more of examples where you pass a pointer to a trusted or safe function and that function escapes the pointer. I couldn't find an example. So maybe allowing that is a good solution. How would returning a parameter array break things?
 * Figure out a way to reconcile "ref" with variadics. This is the 
 actual reason why getopt chose to traffic in addresses, and fixing it 
 is the logical choice and my personal favorite.
This sounds like the best choice.
Well it's not that simple. As I explained in a different post, getopt takes (string, pointer, string, pointer, string, pointer, ...). Now we need to make it take references instead of pointers, but the strings should stay values. We can't express a checkered constraint like that. Incidentally there's a theory for allowing that, it's called "regular types" inspired from regular grammars. With a regular type you can define getopt signature as one or more pairs of string and ref. (Unfortunately C++ defined regular types differently which makes things difficult to search.) Anyhow, I don't think such an approach would help D - it's too complicated.
 6. There must be many things I forgot to mention, or that cause grief 
 to many of us. Please add to/comment on this list.
I know it's not part of the spec, but I'm not sure if you mention the array "data stomping" problem in the book. If not, the MRU cache needs to be implemented.
Yes, it will be because the book has a few failing unittests. In fact, I was hoping I could talk you or David into doing it :o). Andrei
Nov 19 2009
parent reply dsimcha <dsimcha yahoo.com> writes:
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 Yes, it will be because the book has a few failing unittests. In fact, I
 was hoping I could talk you or David into doing it :o).
 Andrei
Unfortunately, I've come to hate the MRU idea because it would fail miserably for large arrays. I've explained this before, but not particularly thoroughly, so I'll try to explain it more thoroughly here. Let's say you have an array that takes up more than half of the total memory you are using. You try to append to it and: 1. The GC runs. The MRU cache is therefore cleared. 2. Your append succeeds, but the array is reallocated. 3. You try to append again. Now, because you have a huge piece of garbage that you just created by reallocating on the last append, the GC needs to run again. The MRU cache is cleared again. 4. Goto 2. Basically, for really huge arrays, we will have the nasty surprise that arrays are reallocated almost every time. Unless something can be done about this, my vote is as follows: 1. a ~= b -> syntactic sugar for a = a ~ b. It's inefficient, but at least it's predictably inefficient and people won't use it if they care at all about performance. 2. .length always reallocates when increasing the length of the array. 3. Get a library type with identical syntax to slices that truly owns its contents and supports efficient appending, resizing, etc. I'd be willing to write this (and even write it soon) if noone else wants to, especially since I think we might be able to use it to define unique ownership for arrays to allow essentially a safe assumeUnique for arrays and kill two birds w/ one stone.
Nov 19 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 Yes, it will be because the book has a few failing unittests. In fact, I
 was hoping I could talk you or David into doing it :o).
 Andrei
Unfortunately, I've come to hate the MRU idea because it would fail miserably for large arrays. I've explained this before, but not particularly thoroughly, so I'll try to explain it more thoroughly here. Let's say you have an array that takes up more than half of the total memory you are using. You try to append to it and: 1. The GC runs. The MRU cache is therefore cleared. 2. Your append succeeds, but the array is reallocated. 3. You try to append again. Now, because you have a huge piece of garbage that you just created by reallocating on the last append, the GC needs to run again. The MRU cache is cleared again. 4. Goto 2.
This is not a matter of principles, but one of implementation. When you GC, you can adjust the cache instead of clearing it.
 Basically, for really huge arrays, we will have the nasty surprise that arrays
are
 reallocated almost every time.  Unless something can be done about this, my
vote
 is as follows:
 
 1.  a ~= b -> syntactic sugar for a = a ~ b.  It's inefficient, but at least
it's
 predictably inefficient and people won't use it if they care at all about
performance.
 
 2.  .length always reallocates when increasing the length of the array.
 
 3.  Get a library type with identical syntax to slices that truly owns its
 contents and supports efficient appending, resizing, etc.  I'd be willing to
write
 this (and even write it soon) if noone else wants to, especially since I think
we
 might be able to use it to define unique ownership for arrays to allow
essentially
 a safe assumeUnique for arrays and kill two birds w/ one stone.
UniqueArray is something we need regardless - for message passing. Andrei
Nov 19 2009
parent reply dsimcha <dsimcha yahoo.com> writes:
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 Yes, it will be because the book has a few failing unittests. In fact, I
 was hoping I could talk you or David into doing it :o).
 Andrei
Unfortunately, I've come to hate the MRU idea because it would fail miserably for large arrays. I've explained this before, but not particularly thoroughly, so I'll try to explain it more thoroughly here. Let's say you have an array that takes up more than half of the total memory you are using. You try to append to it and: 1. The GC runs. The MRU cache is therefore cleared. 2. Your append succeeds, but the array is reallocated. 3. You try to append again. Now, because you have a huge piece of garbage that you just created by reallocating on the last append, the GC needs to run again. The MRU cache is cleared again. 4. Goto 2.
This is not a matter of principles, but one of implementation. When you GC, you can adjust the cache instead of clearing it.
Technically true, but what is a matter of principles is whether the implementation of arrays should be very tightly coupled to the implementation of the GC. Fixing this issue would have massive ripple effects throughout the already spaghetti code-like GC, and might affect GC performance. For every single object the GC freed, it would have to look through the MRU cache and remove it from there if present, too. The point is that this **can** be done, but we probably don't **want** to introduce this kind of coupling, especially if we want our GC model to be sane enough that people might actually come along and write us a better GC one day.
Nov 19 2009
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 Yes, it will be because the book has a few failing unittests. In fact, I
 was hoping I could talk you or David into doing it :o).
 Andrei
Unfortunately, I've come to hate the MRU idea because it would fail miserably for large arrays. I've explained this before, but not particularly thoroughly, so I'll try to explain it more thoroughly here. Let's say you have an array that takes up more than half of the total memory you are using. You try to append to it and: 1. The GC runs. The MRU cache is therefore cleared. 2. Your append succeeds, but the array is reallocated. 3. You try to append again. Now, because you have a huge piece of garbage that you just created by reallocating on the last append, the GC needs to run again. The MRU cache is cleared again. 4. Goto 2.
This is not a matter of principles, but one of implementation. When you GC, you can adjust the cache instead of clearing it.
Technically true, but what is a matter of principles is whether the implementation of arrays should be very tightly coupled to the implementation of the GC. Fixing this issue would have massive ripple effects throughout the already spaghetti code-like GC, and might affect GC performance. For every single object the GC freed, it would have to look through the MRU cache and remove it from there if present, too. The point is that this **can** be done, but we probably don't **want** to introduce this kind of coupling, especially if we want our GC model to be sane enough that people might actually come along and write us a better GC one day.
I agree. But probably there might be a solution out there that solves this issue in a better way. We should keep on looking. Maybe something that refines the MRU, or something that obviates the MRU altogether. Andrei
Nov 19 2009
prev sibling next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 19 Nov 2009 12:01:25 -0500, dsimcha <dsimcha yahoo.com> wrote:

 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s  
 article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s  
article
 Yes, it will be because the book has a few failing unittests. In  
fact, I
 was hoping I could talk you or David into doing it :o).
 Andrei
Unfortunately, I've come to hate the MRU idea because it would fail
miserably for
 large arrays.  I've explained this before, but not particularly  
thoroughly, so
 I'll try to explain it more thoroughly here.  Let's say you have an  
array that
 takes up more than half of the total memory you are using.  You try  
to append to
 it and:

 1.  The GC runs.  The MRU cache is therefore cleared.

 2.  Your append succeeds, but the array is reallocated.

 3.  You try to append again.  Now, because you have a huge piece of  
garbage that
 you just created by reallocating on the last append, the GC needs to  
run again.
 The MRU cache is cleared again.

 4.  Goto 2.
This is not a matter of principles, but one of implementation. When you GC, you can adjust the cache instead of clearing it.
Technically true, but what is a matter of principles is whether the implementation of arrays should be very tightly coupled to the implementation of the GC. Fixing this issue would have massive ripple effects throughout the already spaghetti code-like GC, and might affect GC performance. For every single object the GC freed, it would have to look through the MRU cache and remove it from there if present, too.
You perform the lookup via MRU cache (after mark, before sweep). I see it as a single function call at the right place in the GC.
 The point is that this **can** be done, but we probably don't **want** to
 introduce this kind of coupling, especially if we want our GC model to  
 be sane
 enough that people might actually come along and write us a better GC  
 one day.
What about implementing it as a hook "do this between mark and sweep"? Then it becomes decoupled from the GC. -Steve
Nov 19 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Steven Schveighoffer wrote:
 On Thu, 19 Nov 2009 12:01:25 -0500, dsimcha <dsimcha yahoo.com> wrote:
 
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s 
 article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s 
article
 Yes, it will be because the book has a few failing unittests. In 
fact, I
 was hoping I could talk you or David into doing it :o).
 Andrei
Unfortunately, I've come to hate the MRU idea because it would fail
miserably for
 large arrays.  I've explained this before, but not particularly 
thoroughly, so
 I'll try to explain it more thoroughly here.  Let's say you have an 
array that
 takes up more than half of the total memory you are using.  You try 
to append to
 it and:

 1.  The GC runs.  The MRU cache is therefore cleared.

 2.  Your append succeeds, but the array is reallocated.

 3.  You try to append again.  Now, because you have a huge piece of 
garbage that
 you just created by reallocating on the last append, the GC needs 
to run again.
 The MRU cache is cleared again.

 4.  Goto 2.
This is not a matter of principles, but one of implementation. When you GC, you can adjust the cache instead of clearing it.
Technically true, but what is a matter of principles is whether the implementation of arrays should be very tightly coupled to the implementation of the GC. Fixing this issue would have massive ripple effects throughout the already spaghetti code-like GC, and might affect GC performance. For every single object the GC freed, it would have to look through the MRU cache and remove it from there if present, too.
You perform the lookup via MRU cache (after mark, before sweep). I see it as a single function call at the right place in the GC.
 The point is that this **can** be done, but we probably don't **want** to
 introduce this kind of coupling, especially if we want our GC model to 
 be sane
 enough that people might actually come along and write us a better GC 
 one day.
What about implementing it as a hook "do this between mark and sweep"? Then it becomes decoupled from the GC. -Steve
I think these are great ideas, but you'd need to transport certain information to the cache so it can adjust its pointers. Anyhow, I believe this is worth exploring because it can help with a great many other things such as weak pointers and similar checks and adjustments (there was a paper on GC assertions that I don't have time to dig right now. Aw what the heck, found it: http://www.eecs.tufts.edu/~eaftan/gcassertions-mspc-2008.pdf Andrei
Nov 19 2009
parent reply dsimcha <dsimcha yahoo.com> writes:
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 Steven Schveighoffer wrote:
 On Thu, 19 Nov 2009 12:01:25 -0500, dsimcha <dsimcha yahoo.com> wrote:

 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s
 article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s
article
 Yes, it will be because the book has a few failing unittests. In
fact, I
 was hoping I could talk you or David into doing it :o).
 Andrei
Unfortunately, I've come to hate the MRU idea because it would fail
miserably for
 large arrays.  I've explained this before, but not particularly
thoroughly, so
 I'll try to explain it more thoroughly here.  Let's say you have an
array that
 takes up more than half of the total memory you are using.  You try
to append to
 it and:

 1.  The GC runs.  The MRU cache is therefore cleared.

 2.  Your append succeeds, but the array is reallocated.

 3.  You try to append again.  Now, because you have a huge piece of
garbage that
 you just created by reallocating on the last append, the GC needs
to run again.
 The MRU cache is cleared again.

 4.  Goto 2.
This is not a matter of principles, but one of implementation. When you GC, you can adjust the cache instead of clearing it.
Technically true, but what is a matter of principles is whether the implementation of arrays should be very tightly coupled to the implementation of the GC. Fixing this issue would have massive ripple effects throughout the already spaghetti code-like GC, and might affect GC performance. For every single object the GC freed, it would have to look through the MRU cache and remove it from there if present, too.
You perform the lookup via MRU cache (after mark, before sweep). I see it as a single function call at the right place in the GC.
 The point is that this **can** be done, but we probably don't **want** to
 introduce this kind of coupling, especially if we want our GC model to
 be sane
 enough that people might actually come along and write us a better GC
 one day.
What about implementing it as a hook "do this between mark and sweep"? Then it becomes decoupled from the GC. -Steve
I think these are great ideas, but you'd need to transport certain information to the cache so it can adjust its pointers. Anyhow, I believe this is worth exploring because it can help with a great many other things such as weak pointers and similar checks and adjustments (there was a paper on GC assertions that I don't have time to dig right now. Aw what the heck, found it: http://www.eecs.tufts.edu/~eaftan/gcassertions-mspc-2008.pdf Andrei
The hook doesn't sound like a bad idea, but it raises a lot of issues with the implementation details. These are things I could figure out given plenty of time. I'd like weak refs, too. However, I don't think this makes the short list for D2 because: 1. Doing it at all properly requires a lot of thought about what a good design for such an API should be and how to implement it efficiently. 2. I think we still need an ArrayBuilder or something because, while the MRU would be reasonably efficient, it still wouldn't be as efficient as an ArrayBuilder, and would do nothing to solve the uniqueness problem. Therefore, I think fleshing out ArrayBuilder is a higher priority. I was thinking of a design something like this: abstract class Array { // A bunch of final methods for .length, opIndex, etc. // No .ptr or opSlice. } class UniqueArray : Array { // Still no .ptr or opSlice. Has .toImmutable, which allows // for conversion to immutability iff the elements are either // pure value types or themselves immutable. // // Also, can deterministically delete old arrays on reallocation, // since it owns a unique reference, leading to more GC-efficient // appending. } class ArrayBuilder : Array { // Add opSlice and .ptr. Appending doesn't deterministically // delete old arrays, even if the GC supports this. No guarantees // about uniqueness. }
Nov 19 2009
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 Steven Schveighoffer wrote:
 On Thu, 19 Nov 2009 12:01:25 -0500, dsimcha <dsimcha yahoo.com> wrote:

 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s
 article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s
article
 Yes, it will be because the book has a few failing unittests. In
fact, I
 was hoping I could talk you or David into doing it :o).
 Andrei
Unfortunately, I've come to hate the MRU idea because it would fail
miserably for
 large arrays.  I've explained this before, but not particularly
thoroughly, so
 I'll try to explain it more thoroughly here.  Let's say you have an
array that
 takes up more than half of the total memory you are using.  You try
to append to
 it and:

 1.  The GC runs.  The MRU cache is therefore cleared.

 2.  Your append succeeds, but the array is reallocated.

 3.  You try to append again.  Now, because you have a huge piece of
garbage that
 you just created by reallocating on the last append, the GC needs
to run again.
 The MRU cache is cleared again.

 4.  Goto 2.
This is not a matter of principles, but one of implementation. When you GC, you can adjust the cache instead of clearing it.
Technically true, but what is a matter of principles is whether the implementation of arrays should be very tightly coupled to the implementation of the GC. Fixing this issue would have massive ripple effects throughout the already spaghetti code-like GC, and might affect GC performance. For every single object the GC freed, it would have to look through the MRU cache and remove it from there if present, too.
You perform the lookup via MRU cache (after mark, before sweep). I see it as a single function call at the right place in the GC.
 The point is that this **can** be done, but we probably don't **want** to
 introduce this kind of coupling, especially if we want our GC model to
 be sane
 enough that people might actually come along and write us a better GC
 one day.
What about implementing it as a hook "do this between mark and sweep"? Then it becomes decoupled from the GC. -Steve
I think these are great ideas, but you'd need to transport certain information to the cache so it can adjust its pointers. Anyhow, I believe this is worth exploring because it can help with a great many other things such as weak pointers and similar checks and adjustments (there was a paper on GC assertions that I don't have time to dig right now. Aw what the heck, found it: http://www.eecs.tufts.edu/~eaftan/gcassertions-mspc-2008.pdf Andrei
The hook doesn't sound like a bad idea, but it raises a lot of issues with the implementation details. These are things I could figure out given plenty of time. I'd like weak refs, too. However, I don't think this makes the short list for D2 because: 1. Doing it at all properly requires a lot of thought about what a good design for such an API should be and how to implement it efficiently. 2. I think we still need an ArrayBuilder or something because, while the MRU would be reasonably efficient, it still wouldn't be as efficient as an ArrayBuilder, and would do nothing to solve the uniqueness problem. Therefore, I think fleshing out ArrayBuilder is a higher priority. I was thinking of a design something like this: abstract class Array { // A bunch of final methods for .length, opIndex, etc. // No .ptr or opSlice. } class UniqueArray : Array { // Still no .ptr or opSlice. Has .toImmutable, which allows // for conversion to immutability iff the elements are either // pure value types or themselves immutable. // // Also, can deterministically delete old arrays on reallocation, // since it owns a unique reference, leading to more GC-efficient // appending. } class ArrayBuilder : Array { // Add opSlice and .ptr. Appending doesn't deterministically // delete old arrays, even if the GC supports this. No guarantees // about uniqueness. }
What does .toImmutable return? As far as I can tell making UniqueArray a class can't work because by definition you give up controlling how many references to the array there could be in a program. Andrei
Nov 19 2009
parent reply dsimcha <dsimcha yahoo.com> writes:
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 Steven Schveighoffer wrote:
 On Thu, 19 Nov 2009 12:01:25 -0500, dsimcha <dsimcha yahoo.com> wrote:

 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s
 article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s
article
 Yes, it will be because the book has a few failing unittests. In
fact, I
 was hoping I could talk you or David into doing it :o).
 Andrei
Unfortunately, I've come to hate the MRU idea because it would fail
miserably for
 large arrays.  I've explained this before, but not particularly
thoroughly, so
 I'll try to explain it more thoroughly here.  Let's say you have an
array that
 takes up more than half of the total memory you are using.  You try
to append to
 it and:

 1.  The GC runs.  The MRU cache is therefore cleared.

 2.  Your append succeeds, but the array is reallocated.

 3.  You try to append again.  Now, because you have a huge piece of
garbage that
 you just created by reallocating on the last append, the GC needs
to run again.
 The MRU cache is cleared again.

 4.  Goto 2.
This is not a matter of principles, but one of implementation. When you GC, you can adjust the cache instead of clearing it.
Technically true, but what is a matter of principles is whether the implementation of arrays should be very tightly coupled to the implementation of the GC. Fixing this issue would have massive ripple effects throughout the already spaghetti code-like GC, and might affect GC performance. For every single object the GC freed, it would have to look through the MRU cache and remove it from there if present, too.
You perform the lookup via MRU cache (after mark, before sweep). I see it as a single function call at the right place in the GC.
 The point is that this **can** be done, but we probably don't **want** to
 introduce this kind of coupling, especially if we want our GC model to
 be sane
 enough that people might actually come along and write us a better GC
 one day.
What about implementing it as a hook "do this between mark and sweep"? Then it becomes decoupled from the GC. -Steve
I think these are great ideas, but you'd need to transport certain information to the cache so it can adjust its pointers. Anyhow, I believe this is worth exploring because it can help with a great many other things such as weak pointers and similar checks and adjustments (there was a paper on GC assertions that I don't have time to dig right now. Aw what the heck, found it: http://www.eecs.tufts.edu/~eaftan/gcassertions-mspc-2008.pdf Andrei
The hook doesn't sound like a bad idea, but it raises a lot of issues with the implementation details. These are things I could figure out given plenty of time. I'd like weak refs, too. However, I don't think this makes the short list for D2 because: 1. Doing it at all properly requires a lot of thought about what a good design for such an API should be and how to implement it efficiently. 2. I think we still need an ArrayBuilder or something because, while the MRU would be reasonably efficient, it still wouldn't be as efficient as an ArrayBuilder, and would do nothing to solve the uniqueness problem. Therefore, I think fleshing out ArrayBuilder is a higher priority. I was thinking of a design something like this: abstract class Array { // A bunch of final methods for .length, opIndex, etc. // No .ptr or opSlice. } class UniqueArray : Array { // Still no .ptr or opSlice. Has .toImmutable, which allows // for conversion to immutability iff the elements are either // pure value types or themselves immutable. // // Also, can deterministically delete old arrays on reallocation, // since it owns a unique reference, leading to more GC-efficient // appending. } class ArrayBuilder : Array { // Add opSlice and .ptr. Appending doesn't deterministically // delete old arrays, even if the GC supports this. No guarantees // about uniqueness. }
What does .toImmutable return? As far as I can tell making UniqueArray a class can't work because by definition you give up controlling how many references to the array there could be in a program. Andrei
Sorry, forgot to flesh out a few details. 1. .toImmutable() returns an immutable slice, but also sets UniqueArray's pointer member to null, so no instance of UniqueArray has a mutable reference any longer. After this is called, the UniqueArray object will be invalid unless reinitialized. 2. After thinking about this some more, the big issue I see is ref opIndex. We can either: a. Disallow it for both UniqueArray and ArrayBuilder. b. Allow it for both UniqueArray and ArrayBuilder and accept that a sufficiently dumb programmer can invalidate the guarantees of UniqueArray by taking the address of one of the elements and saving it somewhere. Probably a bad idea, since assumeUnique() already works for the careful programmer, and UniqueArray is supposed to provide ironclad guarantees. c. Don't define opIndex in the abstract base class at all, thus making Array almost useless as an abstract base class.
Nov 19 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 Steven Schveighoffer wrote:
 On Thu, 19 Nov 2009 12:01:25 -0500, dsimcha <dsimcha yahoo.com> wrote:

 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s
 article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s
article
 Yes, it will be because the book has a few failing unittests. In
fact, I
 was hoping I could talk you or David into doing it :o).
 Andrei
Unfortunately, I've come to hate the MRU idea because it would fail
miserably for
 large arrays.  I've explained this before, but not particularly
thoroughly, so
 I'll try to explain it more thoroughly here.  Let's say you have an
array that
 takes up more than half of the total memory you are using.  You try
to append to
 it and:

 1.  The GC runs.  The MRU cache is therefore cleared.

 2.  Your append succeeds, but the array is reallocated.

 3.  You try to append again.  Now, because you have a huge piece of
garbage that
 you just created by reallocating on the last append, the GC needs
to run again.
 The MRU cache is cleared again.

 4.  Goto 2.
This is not a matter of principles, but one of implementation. When you GC, you can adjust the cache instead of clearing it.
Technically true, but what is a matter of principles is whether the implementation of arrays should be very tightly coupled to the implementation of the GC. Fixing this issue would have massive ripple effects throughout the already spaghetti code-like GC, and might affect GC performance. For every single object the GC freed, it would have to look through the MRU cache and remove it from there if present, too.
You perform the lookup via MRU cache (after mark, before sweep). I see it as a single function call at the right place in the GC.
 The point is that this **can** be done, but we probably don't **want** to
 introduce this kind of coupling, especially if we want our GC model to
 be sane
 enough that people might actually come along and write us a better GC
 one day.
What about implementing it as a hook "do this between mark and sweep"? Then it becomes decoupled from the GC. -Steve
I think these are great ideas, but you'd need to transport certain information to the cache so it can adjust its pointers. Anyhow, I believe this is worth exploring because it can help with a great many other things such as weak pointers and similar checks and adjustments (there was a paper on GC assertions that I don't have time to dig right now. Aw what the heck, found it: http://www.eecs.tufts.edu/~eaftan/gcassertions-mspc-2008.pdf Andrei
The hook doesn't sound like a bad idea, but it raises a lot of issues with the implementation details. These are things I could figure out given plenty of time. I'd like weak refs, too. However, I don't think this makes the short list for D2 because: 1. Doing it at all properly requires a lot of thought about what a good design for such an API should be and how to implement it efficiently. 2. I think we still need an ArrayBuilder or something because, while the MRU would be reasonably efficient, it still wouldn't be as efficient as an ArrayBuilder, and would do nothing to solve the uniqueness problem. Therefore, I think fleshing out ArrayBuilder is a higher priority. I was thinking of a design something like this: abstract class Array { // A bunch of final methods for .length, opIndex, etc. // No .ptr or opSlice. } class UniqueArray : Array { // Still no .ptr or opSlice. Has .toImmutable, which allows // for conversion to immutability iff the elements are either // pure value types or themselves immutable. // // Also, can deterministically delete old arrays on reallocation, // since it owns a unique reference, leading to more GC-efficient // appending. } class ArrayBuilder : Array { // Add opSlice and .ptr. Appending doesn't deterministically // delete old arrays, even if the GC supports this. No guarantees // about uniqueness. }
What does .toImmutable return? As far as I can tell making UniqueArray a class can't work because by definition you give up controlling how many references to the array there could be in a program. Andrei
Sorry, forgot to flesh out a few details. 1. .toImmutable() returns an immutable slice, but also sets UniqueArray's pointer member to null, so no instance of UniqueArray has a mutable reference any longer. After this is called, the UniqueArray object will be invalid unless reinitialized.
Ok, so destructive extraction. Perfect.
 2.  After thinking about this some more, the big issue I see is ref opIndex. 
We
 can either:
     a.  Disallow it for both UniqueArray and ArrayBuilder.
     b.  Allow it for both UniqueArray and ArrayBuilder and accept
         that a sufficiently dumb programmer can invalidate the
         guarantees of UniqueArray by taking the address of one of the
         elements and saving it somewhere.  Probably a bad idea, since
         assumeUnique() already works for the careful programmer, and
         UniqueArray is supposed to provide ironclad guarantees.
     c.  Don't define opIndex in the abstract base class at all, thus
         making Array almost useless as an abstract base class.
Welcome to my demons :o). One possibility that I thought of for a long time would be to disallow taking the address of a ref. That reduces the scope of the problem but doesn't eliminate it: void main() { auto a = new UniqueArray(10); fun(a[0], a); } void fun(ref int a, UniqueArray b) { auto imm = b.toImmutable(); // here a is a mutable alias into an immutable array!!! } So that doesn't work, but I thought I'd mention it :o). Another possibility is to expose opIndex to return by value and also opIndexAssign that sets the value. That would be a no-no in C++ because copying is arbitrarily expensive, but I have a feeling that in D it is sensible to consider and foster that all objects should be defined to be cheap to copy (e.g. refcounting, COW etc.) If we go by the notion that in D we can always assume copy costs are reasonable, this last possibility would work. With the newfangled operators, it would even work beautifully because you can do all sorts of things like a[1] += 4 without ever exposing a ref to the user. Andrei
Nov 19 2009
next sibling parent reply dsimcha <dsimcha yahoo.com> writes:
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 2.  After thinking about this some more, the big issue I see is ref opIndex. 
We
 can either:
     a.  Disallow it for both UniqueArray and ArrayBuilder.
     b.  Allow it for both UniqueArray and ArrayBuilder and accept
         that a sufficiently dumb programmer can invalidate the
         guarantees of UniqueArray by taking the address of one of the
         elements and saving it somewhere.  Probably a bad idea, since
         assumeUnique() already works for the careful programmer, and
         UniqueArray is supposed to provide ironclad guarantees.
     c.  Don't define opIndex in the abstract base class at all, thus
         making Array almost useless as an abstract base class.
Welcome to my demons :o). One possibility that I thought of for a long time would be to disallow taking the address of a ref. That reduces the scope of the problem but doesn't eliminate it: void main() { auto a = new UniqueArray(10); fun(a[0], a); } void fun(ref int a, UniqueArray b) { auto imm = b.toImmutable(); // here a is a mutable alias into an immutable array!!! } So that doesn't work, but I thought I'd mention it :o). Another possibility is to expose opIndex to return by value and also opIndexAssign that sets the value. That would be a no-no in C++ because copying is arbitrarily expensive, but I have a feeling that in D it is sensible to consider and foster that all objects should be defined to be cheap to copy (e.g. refcounting, COW etc.) If we go by the notion that in D we can always assume copy costs are reasonable, this last possibility would work. With the newfangled operators, it would even work beautifully because you can do all sorts of things like a[1] += 4 without ever exposing a ref to the user. Andrei
I wonder if it would be feasible to allow overloading on ref vs. non-ref return. Technically this would be overloading on return type, but without many of the practical problems. If the return value is used as an lvalue, the ref return function gets called. If the return value is only used as an rvalue, the non-ref function gets called. This would allow return by value to be defined in the base class and return by reference to only be defined in ArrayBuilder.
Nov 19 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 2.  After thinking about this some more, the big issue I see is ref opIndex. 
We
 can either:
     a.  Disallow it for both UniqueArray and ArrayBuilder.
     b.  Allow it for both UniqueArray and ArrayBuilder and accept
         that a sufficiently dumb programmer can invalidate the
         guarantees of UniqueArray by taking the address of one of the
         elements and saving it somewhere.  Probably a bad idea, since
         assumeUnique() already works for the careful programmer, and
         UniqueArray is supposed to provide ironclad guarantees.
     c.  Don't define opIndex in the abstract base class at all, thus
         making Array almost useless as an abstract base class.
Welcome to my demons :o). One possibility that I thought of for a long time would be to disallow taking the address of a ref. That reduces the scope of the problem but doesn't eliminate it: void main() { auto a = new UniqueArray(10); fun(a[0], a); } void fun(ref int a, UniqueArray b) { auto imm = b.toImmutable(); // here a is a mutable alias into an immutable array!!! } So that doesn't work, but I thought I'd mention it :o). Another possibility is to expose opIndex to return by value and also opIndexAssign that sets the value. That would be a no-no in C++ because copying is arbitrarily expensive, but I have a feeling that in D it is sensible to consider and foster that all objects should be defined to be cheap to copy (e.g. refcounting, COW etc.) If we go by the notion that in D we can always assume copy costs are reasonable, this last possibility would work. With the newfangled operators, it would even work beautifully because you can do all sorts of things like a[1] += 4 without ever exposing a ref to the user. Andrei
I wonder if it would be feasible to allow overloading on ref vs. non-ref return. Technically this would be overloading on return type, but without many of the practical problems. If the return value is used as an lvalue, the ref return function gets called. If the return value is only used as an rvalue, the non-ref function gets called. This would allow return by value to be defined in the base class and return by reference to only be defined in ArrayBuilder.
It has been discussed. Overloading is not necessary - Walter said defining opIndex and opIndexLvalue and having the compiler call the appropriate one is possible. Andrei
Nov 19 2009
parent dsimcha <dsimcha yahoo.com> writes:
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 dsimcha wrote:
 == Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 2.  After thinking about this some more, the big issue I see is ref opIndex. 
We
 can either:
     a.  Disallow it for both UniqueArray and ArrayBuilder.
     b.  Allow it for both UniqueArray and ArrayBuilder and accept
         that a sufficiently dumb programmer can invalidate the
         guarantees of UniqueArray by taking the address of one of the
         elements and saving it somewhere.  Probably a bad idea, since
         assumeUnique() already works for the careful programmer, and
         UniqueArray is supposed to provide ironclad guarantees.
     c.  Don't define opIndex in the abstract base class at all, thus
         making Array almost useless as an abstract base class.
Welcome to my demons :o). One possibility that I thought of for a long time would be to disallow taking the address of a ref. That reduces the scope of the problem but doesn't eliminate it: void main() { auto a = new UniqueArray(10); fun(a[0], a); } void fun(ref int a, UniqueArray b) { auto imm = b.toImmutable(); // here a is a mutable alias into an immutable array!!! } So that doesn't work, but I thought I'd mention it :o). Another possibility is to expose opIndex to return by value and also opIndexAssign that sets the value. That would be a no-no in C++ because copying is arbitrarily expensive, but I have a feeling that in D it is sensible to consider and foster that all objects should be defined to be cheap to copy (e.g. refcounting, COW etc.) If we go by the notion that in D we can always assume copy costs are reasonable, this last possibility would work. With the newfangled operators, it would even work beautifully because you can do all sorts of things like a[1] += 4 without ever exposing a ref to the user. Andrei
I wonder if it would be feasible to allow overloading on ref vs. non-ref return. Technically this would be overloading on return type, but without many of the practical problems. If the return value is used as an lvalue, the ref return function gets called. If the return value is only used as an rvalue, the non-ref function gets called. This would allow return by value to be defined in the base class and return by reference to only be defined in ArrayBuilder.
It has been discussed. Overloading is not necessary - Walter said defining opIndex and opIndexLvalue and having the compiler call the appropriate one is possible. Andrei
Well, given that Walter's plate is pretty full, how hard would it be to implement this? I'll probably prototype my ArrayBuilder w/ pure value return and add in opIndexLvalue stuff if/when it becomes available. Also, are we ever going to do anything about vconst (DIP2)? For now, I've just been ignoring this issue when I write containers and writing them as if const does not exist, but this is obviously not the correct thing to do. http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP2
Nov 19 2009
prev sibling parent dsimcha <dsimcha yahoo.com> writes:
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article
 2.  After thinking about this some more, the big issue I see is ref opIndex. 
We
 can either:
     a.  Disallow it for both UniqueArray and ArrayBuilder.
     b.  Allow it for both UniqueArray and ArrayBuilder and accept
         that a sufficiently dumb programmer can invalidate the
         guarantees of UniqueArray by taking the address of one of the
         elements and saving it somewhere.  Probably a bad idea, since
         assumeUnique() already works for the careful programmer, and
         UniqueArray is supposed to provide ironclad guarantees.
     c.  Don't define opIndex in the abstract base class at all, thus
         making Array almost useless as an abstract base class.
Welcome to my demons :o). One possibility that I thought of for a long time would be to disallow taking the address of a ref. That reduces the scope of the problem but doesn't eliminate it: void main() { auto a = new UniqueArray(10); fun(a[0], a); } void fun(ref int a, UniqueArray b) { auto imm = b.toImmutable(); // here a is a mutable alias into an immutable array!!! } So that doesn't work, but I thought I'd mention it :o). Another possibility is to expose opIndex to return by value and also opIndexAssign that sets the value. That would be a no-no in C++ because copying is arbitrarily expensive, but I have a feeling that in D it is sensible to consider and foster that all objects should be defined to be cheap to copy (e.g. refcounting, COW etc.) If we go by the notion that in D we can always assume copy costs are reasonable, this last possibility would work. With the newfangled operators, it would even work beautifully because you can do all sorts of things like a[1] += 4 without ever exposing a ref to the user. Andrei
Eureka! This is one of those corner cases where non-virtual, non-final class member functions are tremendously useful. Fortunately, we can simulate them my making opIndex a template. abstract class ArrayBase(T) { T* ptr; T opIndex(I : size_t)(I index) { return ptr[index]; } } class ArrayBuilder(T) : ArrayBase!(T) { ref T opIndex(I : size_t)(I index) { return ptr[index]; } } void main() { auto ab = new ArrayBuilder!(float); auto num = ab[0]; // Yes, this segfaults, but we only care // that it compiles. } If the compile time type is an ArrayBase or any subtype of ArrayBase that doesn't override opIndex, then we get value return. If it's an ArrayBuilder, we get ref return. This happens regardless of what the runtime type is. Precisely what we wanted.
Nov 19 2009
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 19 Nov 2009 13:54:04 -0500, dsimcha <dsimcha yahoo.com> wrote:

 The hook doesn't sound like a bad idea, but it raises a lot of issues  
 with the
 implementation details.  These are things I could figure out given  
 plenty of time.
  I'd like weak refs, too.  However, I don't think this makes the short  
 list for D2
 because:

 1.  Doing it at all properly requires a lot of thought about what a good  
 design
 for such an API should be and how to implement it efficiently.
To have a hook or not to have a hook is not as important as fixing the bug. If we do it first without a public hook, that is fine, then we can refine it later.
 2.  I think we still need an ArrayBuilder or something because, while  
 the MRU
 would be reasonably efficient, it still wouldn't be as efficient as an
 ArrayBuilder, and would do nothing to solve the uniqueness problem.   
 Therefore, I
 think fleshing out ArrayBuilder is a higher priority.
Let's fix the bug first, then we can worry about efficiency. The fact that you can stomp on immutable memory in safeD is no good for D2. -Steve
Nov 23 2009
prev sibling parent Leandro Lucarella <llucax gmail.com> writes:
dsimcha, el 19 de noviembre a las 17:01 me escribiste:
 Unfortunately, I've come to hate the MRU idea because it would fail miserably
for
 large arrays.  I've explained this before, but not particularly thoroughly, so
 I'll try to explain it more thoroughly here.  Let's say you have an array that
 takes up more than half of the total memory you are using.  You try to append
to
 it and:

 1.  The GC runs.  The MRU cache is therefore cleared.

 2.  Your append succeeds, but the array is reallocated.

 3.  You try to append again.  Now, because you have a huge piece of garbage
that
 you just created by reallocating on the last append, the GC needs to run again.
 The MRU cache is cleared again.

 4.  Goto 2.
This is not a matter of principles, but one of implementation. When you GC, you can adjust the cache instead of clearing it.
Technically true, but what is a matter of principles is whether the implementation of arrays should be very tightly coupled to the implementation of the GC. Fixing this issue would have massive ripple effects throughout the already spaghetti code-like GC, and might affect GC performance. For every single object the GC freed, it would have to look through the MRU cache and remove it from there if present, too. The point is that this **can** be done, but we probably don't **want** to introduce this kind of coupling, especially if we want our GC model to be sane enough that people might actually come along and write us a better GC one day.
Amen! -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) ---------------------------------------------------------------------- now self-employed, concerned (but powerless), an empowered and informed member of society (pragmatism not idealism), will not cry in public, less chance of illness, tires that grip in the wet (shot of baby strapped in back seat),
Nov 19 2009
prev sibling next sibling parent reply Kyle <none dontemail.com> writes:
Andrei Alexandrescu Wrote:

 Yes; opBinary was just given as an example. Unary operators look like this:
 
 T opUnary(string op)();
Why not do T op(string op)(); // unary T op(string op)(T rhs); // binary
Nov 19 2009
parent Kyle <none dontemail.com> writes:
Kyle Wrote:

 Andrei Alexandrescu Wrote:
 
 Yes; opBinary was just given as an example. Unary operators look like this:
 
 T opUnary(string op)();
Why not do T op(string op)(); // unary T op(string op)(T rhs); // binary
Whoops T op(string operation)(); // unary T op(string operation)(T rhs); // binary Maybe I just discovered the reason :)
Nov 19 2009
prev sibling next sibling parent reply aarti_pl <aarti interia.pl> writes:
Andrei Alexandrescu pisze:
 We're entering the finale of D2 and I want to keep a short list of 
 things that must be done and integrated in the release. It is clearly 
 understood by all of us that there are many things that could and 
 probably should be done.
 
 1. Currently Walter and Don are diligently fixing the problems marked on 
 the current manuscript.
 
 2. User-defined operators must be revamped. Fortunately Don already put 
 in an important piece of functionality (opDollar). What we're looking at 
 is a two-pronged attack motivated by Don's proposal:
 
 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7
 
 The two prongs are:
 
 * Encode operators by compile-time strings. For example, instead of the 
 plethora of opAdd, opMul, ..., we'd have this:
 
 T opBinary(string op)(T rhs) { ... }
 
 The string is "+", "*", etc. We need to design what happens with 
 read-modify-write operators like "+=" (should they be dispatch to a 
 different function? etc.) and also what happens with index-and-modify 
 operators like "[]=", "[]+=" etc. Should we go with proxies? Absorb them 
 in opBinary? Define another dedicated method? etc.
 
 * Loop fusion that generalizes array-wise operations. This idea of 
 Walter is, I think, very good because it generalizes and democratizes 
 "magic". The idea is that, if you do
 
 a = b + c;
 
 and b + c does not make sense but b and c are ranges for which a.front = 
 b.front + c.front does make sense, to automatically add the iteration 
 paraphernalia.
 
 3. It was mentioned in this group that if getopt() does not work in 
 SafeD, then SafeD may as well pack and go home. I agree. We need to make 
 it work. Three ideas discussed with Walter:
 
 * Allow taking addresses of locals, but in that case switch allocation 
 from stack to heap, just like with delegates. If we only do that in 
 SafeD, behavior will be different than with regular D. In any case, it's 
 an inefficient proposition, particularly for getopt() which actually 
 does not need to escape the addresses - just fills them up.
 
 * Allow  trusted (and maybe even  safe) functions to receive addresses 
 of locals. Statically check that they never escape an address of a 
 parameter. I think this is very interesting because it enlarges the 
 common ground of D and SafeD.
 
 * Figure out a way to reconcile "ref" with variadics. This is the actual 
 reason why getopt chose to traffic in addresses, and fixing it is the 
 logical choice and my personal favorite.
 
 4. Allow private members inside a template using the eponymous trick:
 
 template wyda(int x) {
    private enum geeba = x / 2;
    alias geeba wyda;
 }
 
 The names inside an eponymous template are only accessible to the 
 current instantiation. For example, wyda!5 cannot access wyda!(4).geeba, 
 only its own geeba. That we we elegantly avoid the issue "where is this 
 symbol looked up?"
 
 5. Chain exceptions instead of having a recurrent exception terminate 
 the program. I'll dedicate a separate post to this.
 
 6. There must be many things I forgot to mention, or that cause grief to 
 many of us. Please add to/comment on this list.
 
 
 
 Andrei
I kinda like this proposal. But I would rather call template like below: T opInfix(string op)(T rhs) { ... } T opPrefix(string op)(T rhs) { ... } T opPostfix(string op)(T rhs) { ... } and allow user to define her own operators (though it doesn't have to be done now). I know that quite a few people here doesn't like to allow users to define their own operators, because it might obfuscate code. But it doesn't have to be like this. Someone here already mentioned here that it is not real problem for programs in C++. Good libraries don't abuse this functionality. User defined operators would allow easy definition of Domain Specific Languages in D. I was already writing about it some time ago: http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D&article_id=81026 http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D&article_id=81352 BR Marcin Kuszczak (aarti_pl)
Nov 19 2009
next sibling parent reply aarti_pl <aarti interia.pl> writes:
aarti_pl pisze:
 Andrei Alexandrescu pisze:
 2. User-defined operators must be revamped. Fortunately Don already 
 put in an important piece of functionality (opDollar). What we're 
 looking at is a two-pronged attack motivated by Don's proposal:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7

 The two prongs are:

 * Encode operators by compile-time strings. For example, instead of 
 the plethora of opAdd, opMul, ..., we'd have this:

 T opBinary(string op)(T rhs) { ... }

 The string is "+", "*", etc. We need to design what happens with 
 read-modify-write operators like "+=" (should they be dispatch to a 
 different function? etc.) and also what happens with index-and-modify 
 operators like "[]=", "[]+=" etc. Should we go with proxies? Absorb 
 them in opBinary? Define another dedicated method? etc.

 * Loop fusion that generalizes array-wise operations. This idea of 
 Walter is, I think, very good because it generalizes and democratizes 
 "magic". The idea is that, if you do

 a = b + c;

 and b + c does not make sense but b and c are ranges for which a.front 
 = b.front + c.front does make sense, to automatically add the 
 iteration paraphernalia.
(..)
 Andrei
I kinda like this proposal. But I would rather call template like below: T opInfix(string op)(T rhs) { ... } T opPrefix(string op)(T rhs) { ... } T opPostfix(string op)(T rhs) { ... } and allow user to define her own operators (though it doesn't have to be done now). I know that quite a few people here doesn't like to allow users to define their own operators, because it might obfuscate code. But it doesn't have to be like this. Someone here already mentioned here that it is not real problem for programs in C++. Good libraries don't abuse this functionality. User defined operators would allow easy definition of Domain Specific Languages in D. I was already writing about it some time ago: http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmar .D&article_id=81026 http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmar .D&article_id=81352 BR Marcin Kuszczak (aarti_pl)
Of course for opPrefix/opPostfix signatures will be different: T opPrefix(string op)() { ... } T opPostfix(string op)() { ... } Sorry for mistake. BR Marcin Kuszczak (aarti_pl)
Nov 19 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
aarti_pl wrote:
 aarti_pl pisze:
 Andrei Alexandrescu pisze:
 2. User-defined operators must be revamped. Fortunately Don already 
 put in an important piece of functionality (opDollar). What we're 
 looking at is a two-pronged attack motivated by Don's proposal:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7

 The two prongs are:

 * Encode operators by compile-time strings. For example, instead of 
 the plethora of opAdd, opMul, ..., we'd have this:

 T opBinary(string op)(T rhs) { ... }

 The string is "+", "*", etc. We need to design what happens with 
 read-modify-write operators like "+=" (should they be dispatch to a 
 different function? etc.) and also what happens with index-and-modify 
 operators like "[]=", "[]+=" etc. Should we go with proxies? Absorb 
 them in opBinary? Define another dedicated method? etc.

 * Loop fusion that generalizes array-wise operations. This idea of 
 Walter is, I think, very good because it generalizes and democratizes 
 "magic". The idea is that, if you do

 a = b + c;

 and b + c does not make sense but b and c are ranges for which 
 a.front = b.front + c.front does make sense, to automatically add the 
 iteration paraphernalia.
(..)
 Andrei
I kinda like this proposal. But I would rather call template like below: T opInfix(string op)(T rhs) { ... } T opPrefix(string op)(T rhs) { ... } T opPostfix(string op)(T rhs) { ... } and allow user to define her own operators (though it doesn't have to be done now). I know that quite a few people here doesn't like to allow users to define their own operators, because it might obfuscate code. But it doesn't have to be like this. Someone here already mentioned here that it is not real problem for programs in C++. Good libraries don't abuse this functionality. User defined operators would allow easy definition of Domain Specific Languages in D. I was already writing about it some time ago: http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmar .D&article_id=81026 http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmar .D&article_id=81352 BR Marcin Kuszczak (aarti_pl)
Of course for opPrefix/opPostfix signatures will be different: T opPrefix(string op)() { ... } T opPostfix(string op)() { ... } Sorry for mistake. BR Marcin Kuszczak (aarti_pl)
I think we'll solve postfix "++" without requiring the user to define it. Do you envision user-defined postfix operators? Andrei
Nov 19 2009
parent reply aarti_pl <aarti interia.pl> writes:
Andrei Alexandrescu pisze:
 aarti_pl wrote:
 aarti_pl pisze:
 Andrei Alexandrescu pisze:
 2. User-defined operators must be revamped. Fortunately Don already 
 put in an important piece of functionality (opDollar). What we're 
 looking at is a two-pronged attack motivated by Don's proposal:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7

 The two prongs are:

 * Encode operators by compile-time strings. For example, instead of 
 the plethora of opAdd, opMul, ..., we'd have this:

 T opBinary(string op)(T rhs) { ... }

 The string is "+", "*", etc. We need to design what happens with 
 read-modify-write operators like "+=" (should they be dispatch to a 
 different function? etc.) and also what happens with 
 index-and-modify operators like "[]=", "[]+=" etc. Should we go with 
 proxies? Absorb them in opBinary? Define another dedicated method? etc.

 * Loop fusion that generalizes array-wise operations. This idea of 
 Walter is, I think, very good because it generalizes and 
 democratizes "magic". The idea is that, if you do

 a = b + c;

 and b + c does not make sense but b and c are ranges for which 
 a.front = b.front + c.front does make sense, to automatically add 
 the iteration paraphernalia.
(..)
 Andrei
I kinda like this proposal. But I would rather call template like below: T opInfix(string op)(T rhs) { ... } T opPrefix(string op)(T rhs) { ... } T opPostfix(string op)(T rhs) { ... } and allow user to define her own operators (though it doesn't have to be done now). I know that quite a few people here doesn't like to allow users to define their own operators, because it might obfuscate code. But it doesn't have to be like this. Someone here already mentioned here that it is not real problem for programs in C++. Good libraries don't abuse this functionality. User defined operators would allow easy definition of Domain Specific Languages in D. I was already writing about it some time ago: http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmar .D&article_id=81026 http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmar .D&article_id=81352 BR Marcin Kuszczak (aarti_pl)
Of course for opPrefix/opPostfix signatures will be different: T opPrefix(string op)() { ... } T opPostfix(string op)() { ... } Sorry for mistake. BR Marcin Kuszczak (aarti_pl)
I think we'll solve postfix "++" without requiring the user to define it. Do you envision user-defined postfix operators? Andrei
Well, maybe something like below: auto a = 2²; //(quadratic power of 2) auto a = 5!; //factorial of 5 auto a = 2Ƴ + 3ɛ; //solving equations auto weight = 5kg; //units of measurement The point is that this covers whole scope of operators. In fact even built-in operators could be defined using it. Postfix operator ++ can be defined using prefix operator++ just by delegation and this can be default. Best Regards Marcin Kuszczak (aarti_pl)
Nov 19 2009
next sibling parent reply Justin Johansson <no spam.com> writes:
aarti_pl wrote:
 Andrei Alexandrescu pisze:
 aarti_pl wrote:
 aarti_pl pisze:
 Andrei Alexandrescu pisze:
 2. User-defined operators must be revamped. Fortunately Don already 
 put in an important piece of functionality (opDollar). What we're 
 looking at is a two-pronged attack motivated by Don's proposal:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7

 The two prongs are:

 * Encode operators by compile-time strings. For example, instead of 
 the plethora of opAdd, opMul, ..., we'd have this:

 T opBinary(string op)(T rhs) { ... }

 The string is "+", "*", etc. We need to design what happens with 
 read-modify-write operators like "+=" (should they be dispatch to a 
 different function? etc.) and also what happens with 
 index-and-modify operators like "[]=", "[]+=" etc. Should we go 
 with proxies? Absorb them in opBinary? Define another dedicated 
 method? etc.

 * Loop fusion that generalizes array-wise operations. This idea of 
 Walter is, I think, very good because it generalizes and 
 democratizes "magic". The idea is that, if you do

 a = b + c;

 and b + c does not make sense but b and c are ranges for which 
 a.front = b.front + c.front does make sense, to automatically add 
 the iteration paraphernalia.
(..)
 Andrei
I kinda like this proposal. But I would rather call template like below: T opInfix(string op)(T rhs) { ... } T opPrefix(string op)(T rhs) { ... } T opPostfix(string op)(T rhs) { ... } and allow user to define her own operators (though it doesn't have to be done now). I know that quite a few people here doesn't like to allow users to define their own operators, because it might obfuscate code. But it doesn't have to be like this. Someone here already mentioned here that it is not real problem for programs in C++. Good libraries don't abuse this functionality. User defined operators would allow easy definition of Domain Specific Languages in D. I was already writing about it some time ago: http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmar .D&article_id=81026 http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmar .D&article_id=81352 BR Marcin Kuszczak (aarti_pl)
Of course for opPrefix/opPostfix signatures will be different: T opPrefix(string op)() { ... } T opPostfix(string op)() { ... } Sorry for mistake. BR Marcin Kuszczak (aarti_pl)
I think we'll solve postfix "++" without requiring the user to define it. Do you envision user-defined postfix operators? Andrei
Well, maybe something like below: auto a = 2²; //(quadratic power of 2) auto a = 5!; //factorial of 5 auto a = 2Ƴ + 3ɛ; //solving equations auto weight = 5kg; //units of measurement The point is that this covers whole scope of operators. In fact even built-in operators could be defined using it. Postfix operator ++ can be defined using prefix operator++ just by delegation and this can be default. Best Regards Marcin Kuszczak (aarti_pl)
Marcin demonstrates a valid point. If there is going to be this feature creep, the feature should be complete with all the usual variants of operator arity and notation (i.e. prefix/postfix/infix). Otherwise it really it's only two-thirds baked. -- Justin Johansson
Nov 19 2009
parent reply Justin Johansson <no spam.com> writes:
Justin Johansson wrote:
 aarti_pl wrote:
 Andrei Alexandrescu pisze:
 aarti_pl wrote:
 aarti_pl pisze:
 Andrei Alexandrescu pisze:
 2. User-defined operators must be revamped. Fortunately Don 
 already put in an important piece of functionality (opDollar). 
 What we're looking at is a two-pronged attack motivated by Don's 
 proposal:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7

 The two prongs are:

 * Encode operators by compile-time strings. For example, instead 
 of the plethora of opAdd, opMul, ..., we'd have this:

 T opBinary(string op)(T rhs) { ... }

 The string is "+", "*", etc. We need to design what happens with 
 read-modify-write operators like "+=" (should they be dispatch to 
 a different function? etc.) and also what happens with 
 index-and-modify operators like "[]=", "[]+=" etc. Should we go 
 with proxies? Absorb them in opBinary? Define another dedicated 
 method? etc.

 * Loop fusion that generalizes array-wise operations. This idea of 
 Walter is, I think, very good because it generalizes and 
 democratizes "magic". The idea is that, if you do

 a = b + c;

 and b + c does not make sense but b and c are ranges for which 
 a.front = b.front + c.front does make sense, to automatically add 
 the iteration paraphernalia.
(..)
 Andrei
I kinda like this proposal. But I would rather call template like below: T opInfix(string op)(T rhs) { ... } T opPrefix(string op)(T rhs) { ... } T opPostfix(string op)(T rhs) { ... } and allow user to define her own operators (though it doesn't have to be done now). I know that quite a few people here doesn't like to allow users to define their own operators, because it might obfuscate code. But it doesn't have to be like this. Someone here already mentioned here that it is not real problem for programs in C++. Good libraries don't abuse this functionality. User defined operators would allow easy definition of Domain Specific Languages in D. I was already writing about it some time ago: http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmar .D&article_id=81026 http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmar .D&article_id=81352 BR Marcin Kuszczak (aarti_pl)
Of course for opPrefix/opPostfix signatures will be different: T opPrefix(string op)() { ... } T opPostfix(string op)() { ... } Sorry for mistake. BR Marcin Kuszczak (aarti_pl)
I think we'll solve postfix "++" without requiring the user to define it. Do you envision user-defined postfix operators? Andrei
Well, maybe something like below: auto a = 2²; //(quadratic power of 2) auto a = 5!; //factorial of 5 auto a = 2Ƴ + 3ɛ; //solving equations auto weight = 5kg; //units of measurement The point is that this covers whole scope of operators. In fact even built-in operators could be defined using it. Postfix operator ++ can be defined using prefix operator++ just by delegation and this can be default. Best Regards Marcin Kuszczak (aarti_pl)
Marcin demonstrates a valid point. If there is going to be this feature creep, the feature should be complete with all the usual variants of operator arity and notation (i.e. prefix/postfix/infix). Otherwise it really it's only two-thirds baked. -- Justin Johansson
I meant to say "Iff there is ..." as in "if and only if". Like others, I'm not completely sold on the idea at all. Also it's probably not possible to squeeze something as long as this on to a short list. All or nothing please.
Nov 19 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Justin Johansson wrote:
 Justin Johansson wrote:
 aarti_pl wrote:
 Andrei Alexandrescu pisze:
 aarti_pl wrote:
 aarti_pl pisze:
 Andrei Alexandrescu pisze:
 2. User-defined operators must be revamped. Fortunately Don 
 already put in an important piece of functionality (opDollar). 
 What we're looking at is a two-pronged attack motivated by Don's 
 proposal:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7

 The two prongs are:

 * Encode operators by compile-time strings. For example, instead 
 of the plethora of opAdd, opMul, ..., we'd have this:

 T opBinary(string op)(T rhs) { ... }

 The string is "+", "*", etc. We need to design what happens with 
 read-modify-write operators like "+=" (should they be dispatch to 
 a different function? etc.) and also what happens with 
 index-and-modify operators like "[]=", "[]+=" etc. Should we go 
 with proxies? Absorb them in opBinary? Define another dedicated 
 method? etc.

 * Loop fusion that generalizes array-wise operations. This idea 
 of Walter is, I think, very good because it generalizes and 
 democratizes "magic". The idea is that, if you do

 a = b + c;

 and b + c does not make sense but b and c are ranges for which 
 a.front = b.front + c.front does make sense, to automatically add 
 the iteration paraphernalia.
(..)
 Andrei
I kinda like this proposal. But I would rather call template like below: T opInfix(string op)(T rhs) { ... } T opPrefix(string op)(T rhs) { ... } T opPostfix(string op)(T rhs) { ... } and allow user to define her own operators (though it doesn't have to be done now). I know that quite a few people here doesn't like to allow users to define their own operators, because it might obfuscate code. But it doesn't have to be like this. Someone here already mentioned here that it is not real problem for programs in C++. Good libraries don't abuse this functionality. User defined operators would allow easy definition of Domain Specific Languages in D. I was already writing about it some time ago: http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmar .D&article_id=81026 http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmar .D&article_id=81352 BR Marcin Kuszczak (aarti_pl)
Of course for opPrefix/opPostfix signatures will be different: T opPrefix(string op)() { ... } T opPostfix(string op)() { ... } Sorry for mistake. BR Marcin Kuszczak (aarti_pl)
I think we'll solve postfix "++" without requiring the user to define it. Do you envision user-defined postfix operators? Andrei
Well, maybe something like below: auto a = 2²; //(quadratic power of 2) auto a = 5!; //factorial of 5 auto a = 2Ƴ + 3ɛ; //solving equations auto weight = 5kg; //units of measurement The point is that this covers whole scope of operators. In fact even built-in operators could be defined using it. Postfix operator ++ can be defined using prefix operator++ just by delegation and this can be default. Best Regards Marcin Kuszczak (aarti_pl)
Marcin demonstrates a valid point. If there is going to be this feature creep, the feature should be complete with all the usual variants of operator arity and notation (i.e. prefix/postfix/infix). Otherwise it really it's only two-thirds baked. -- Justin Johansson
I meant to say "Iff there is ..." as in "if and only if". Like others, I'm not completely sold on the idea at all. Also it's probably not possible to squeeze something as long as this on to a short list. All or nothing please.
I disagree with this false choice. http://en.wikipedia.org/wiki/False_dilemma#False_choice Andrei
Nov 19 2009
parent Justin Johansson <no spam.com> writes:
Andrei Alexandrescu wrote:
 Justin Johansson wrote:
 Justin Johansson wrote:
 aarti_pl wrote:
 Andrei Alexandrescu pisze:
 aarti_pl wrote:
 aarti_pl pisze:
 Andrei Alexandrescu pisze:
 2. User-defined operators must be revamped. Fortunately Don 
 already put in an important piece of functionality (opDollar). 
 What we're looking at is a two-pronged attack motivated by Don's 
 proposal:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7

 The two prongs are:

 * Encode operators by compile-time strings. For example, instead 
 of the plethora of opAdd, opMul, ..., we'd have this:

 T opBinary(string op)(T rhs) { ... }

 The string is "+", "*", etc. We need to design what happens with 
 read-modify-write operators like "+=" (should they be dispatch 
 to a different function? etc.) and also what happens with 
 index-and-modify operators like "[]=", "[]+=" etc. Should we go 
 with proxies? Absorb them in opBinary? Define another dedicated 
 method? etc.

 * Loop fusion that generalizes array-wise operations. This idea 
 of Walter is, I think, very good because it generalizes and 
 democratizes "magic". The idea is that, if you do

 a = b + c;

 and b + c does not make sense but b and c are ranges for which 
 a.front = b.front + c.front does make sense, to automatically 
 add the iteration paraphernalia.
(..)
 Andrei
I kinda like this proposal. But I would rather call template like below: T opInfix(string op)(T rhs) { ... } T opPrefix(string op)(T rhs) { ... } T opPostfix(string op)(T rhs) { ... } and allow user to define her own operators (though it doesn't have to be done now). I know that quite a few people here doesn't like to allow users to define their own operators, because it might obfuscate code. But it doesn't have to be like this. Someone here already mentioned here that it is not real problem for programs in C++. Good libraries don't abuse this functionality. User defined operators would allow easy definition of Domain Specific Languages in D. I was already writing about it some time ago: http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmar .D&article_id=81026 http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmar .D&article_id=81352 BR Marcin Kuszczak (aarti_pl)
Of course for opPrefix/opPostfix signatures will be different: T opPrefix(string op)() { ... } T opPostfix(string op)() { ... } Sorry for mistake. BR Marcin Kuszczak (aarti_pl)
I think we'll solve postfix "++" without requiring the user to define it. Do you envision user-defined postfix operators? Andrei
Well, maybe something like below: auto a = 2²; //(quadratic power of 2) auto a = 5!; //factorial of 5 auto a = 2Ƴ + 3ɛ; //solving equations auto weight = 5kg; //units of measurement The point is that this covers whole scope of operators. In fact even built-in operators could be defined using it. Postfix operator ++ can be defined using prefix operator++ just by delegation and this can be default. Best Regards Marcin Kuszczak (aarti_pl)
Marcin demonstrates a valid point. If there is going to be this feature creep, the feature should be complete with all the usual variants of operator arity and notation (i.e. prefix/postfix/infix). Otherwise it really it's only two-thirds baked. -- Justin Johansson
I meant to say "Iff there is ..." as in "if and only if". Like others, I'm not completely sold on the idea at all. Also it's probably not possible to squeeze something as long as this on to a short list. All or nothing please.
I disagree with this false choice. http://en.wikipedia.org/wiki/False_dilemma#False_choice Andrei
You are very well read :-)
Nov 19 2009
prev sibling parent KennyTM~ <kennytm gmail.com> writes:
On Nov 20, 09 04:31, aarti_pl wrote:
 Andrei Alexandrescu pisze:
 aarti_pl wrote:
 aarti_pl pisze:
 Andrei Alexandrescu pisze:
 2. User-defined operators must be revamped. Fortunately Don already
 put in an important piece of functionality (opDollar). What we're
 looking at is a two-pronged attack motivated by Don's proposal:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7

 The two prongs are:

 * Encode operators by compile-time strings. For example, instead of
 the plethora of opAdd, opMul, ..., we'd have this:

 T opBinary(string op)(T rhs) { ... }

 The string is "+", "*", etc. We need to design what happens with
 read-modify-write operators like "+=" (should they be dispatch to a
 different function? etc.) and also what happens with
 index-and-modify operators like "[]=", "[]+=" etc. Should we go
 with proxies? Absorb them in opBinary? Define another dedicated
 method? etc.

 * Loop fusion that generalizes array-wise operations. This idea of
 Walter is, I think, very good because it generalizes and
 democratizes "magic". The idea is that, if you do

 a = b + c;

 and b + c does not make sense but b and c are ranges for which
 a.front = b.front + c.front does make sense, to automatically add
 the iteration paraphernalia.
(..)
 Andrei
I kinda like this proposal. But I would rather call template like below: T opInfix(string op)(T rhs) { ... } T opPrefix(string op)(T rhs) { ... } T opPostfix(string op)(T rhs) { ... } and allow user to define her own operators (though it doesn't have to be done now). I know that quite a few people here doesn't like to allow users to define their own operators, because it might obfuscate code. But it doesn't have to be like this. Someone here already mentioned here that it is not real problem for programs in C++. Good libraries don't abuse this functionality. User defined operators would allow easy definition of Domain Specific Languages in D. I was already writing about it some time ago: http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D&article_id=81026 http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D&article_id=81352 BR Marcin Kuszczak (aarti_pl)
Of course for opPrefix/opPostfix signatures will be different: T opPrefix(string op)() { ... } T opPostfix(string op)() { ... } Sorry for mistake. BR Marcin Kuszczak (aarti_pl)
I think we'll solve postfix "++" without requiring the user to define it. Do you envision user-defined postfix operators? Andrei
Well, maybe something like below: auto a = 2²; //(quadratic power of 2) auto a = 5!; //factorial of 5 auto a = 2Ƴ + 3ɛ; //solving equations auto weight = 5kg; //units of measurement The point is that this covers whole scope of operators. In fact even built-in operators could be defined using it. Postfix operator ++ can be defined using prefix operator++ just by delegation and this can be default. Best Regards Marcin Kuszczak (aarti_pl)
It will make weird stuff like class G { G opPostfix(string op)() if (op == "%") { ... } G opCall(int x) { ... } } auto g = new G; g% (4+2); // what should it do?
Nov 19 2009
prev sibling next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
aarti_pl:

 T opInfix(string op)(T rhs) { ... }
 T opPrefix(string op)(T rhs) { ... }
 T opPostfix(string op)(T rhs) { ... }
So you can use opInfix to define operators like a ~~ b :-) Now you just need a way to define operator precedence level, with an int number in 0-14 and you're done (I don't think you need to specify operator associativity if left-to-right/right-to-left) :-) T opInfix(string op, int prec)(T rhs) { ... } There programming languages that allow to specify operator precedence level too, but I think this is overkill in D. Regarding operators, in D they are named according to their purpose and not according to their look, and I think this is a good idea. But opDollar doesn't follow that, so isn't a name like opEnd better? Bye, bearophile
Nov 19 2009
parent reply aarti_pl <aarti interia.pl> writes:
bearophile pisze:
 aarti_pl:
 
 T opInfix(string op)(T rhs) { ... }
 T opPrefix(string op)(T rhs) { ... }
 T opPostfix(string op)(T rhs) { ... }
So you can use opInfix to define operators like a ~~ b :-)
Exactly. But the question is if you *REALLY* need it :-) But IMHO the answer is up to designer.
 Now you just need a way to define operator precedence level, with an int
number in 0-14 and you're done (I don't think you need to specify operator
associativity if left-to-right/right-to-left) :-)
 T opInfix(string op, int prec)(T rhs) { ... }
 There programming languages that allow to specify operator precedence level
too, but I think this is overkill in D.
I agree. That's too much.
 Regarding operators, in D they are named according to their purpose and not
according to their look, and I think this is a good idea. But opDollar doesn't
follow that, so isn't a name like opEnd better?
I think that proposed names exactly reflect the meaning. So I would say it is perfectly consistent with D convention.
 
 Bye,
 bearophile
Nov 19 2009
parent reply bearophile <bearophileHUGS lycos.com> writes:
aarti_pl:

 I think that proposed names exactly reflect the meaning. So I would say 
 it is perfectly consistent with D convention.
Dollars are money, but opDollar is not a member function that returns the price of a struct. So I don't agree with you, or I don't understand what you mean. Bye, bearophile
Nov 19 2009
parent reply aarti_pl <aarti interia.pl> writes:
bearophile pisze:
 aarti_pl:
 
 I think that proposed names exactly reflect the meaning. So I would say 
 it is perfectly consistent with D convention.
Dollars are money, but opDollar is not a member function that returns the price of a struct. So I don't agree with you, or I don't understand what you mean. Bye, bearophile
I agree. opDollar is not particularly fitting to D language operator concept. opLength/opSize would fit better. In my post I was referring to my name propositions: opInfix, opPrefix, opPostfix. BR Marcin Kuszczak (aarti_pl)
Nov 19 2009
next sibling parent reply Don <nospam nospam.com> writes:
aarti_pl wrote:
 bearophile pisze:
 aarti_pl:

 I think that proposed names exactly reflect the meaning. So I would 
 say it is perfectly consistent with D convention.
Dollars are money, but opDollar is not a member function that returns the price of a struct. So I don't agree with you, or I don't understand what you mean. Bye, bearophile
I agree. opDollar is not particularly fitting to D language operator concept. opLength/opSize would fit better.
Unfortunately $ is not necessarily the length, nor the size. It might not even be an arithmetic type.
Nov 20 2009
next sibling parent reply "Simen kjaeraas" <simen.kjaras gmail.com> writes:
Don <nospam nospam.com> wrote:

 aarti_pl wrote:
 bearophile pisze:
 aarti_pl:

 I think that proposed names exactly reflect the meaning. So I would  
 say it is perfectly consistent with D convention.
Dollars are money, but opDollar is not a member function that returns the price of a struct. So I don't agree with you, or I don't understand what you mean. Bye, bearophile
I agree. opDollar is not particularly fitting to D language operator concept. opLength/opSize would fit better.
Unfortunately $ is not necessarily the length, nor the size. It might not even be an arithmetic type.
opEnd, then? -- Simen
Nov 20 2009
parent Don <nospam nospam.com> writes:
Simen kjaeraas wrote:
 Don <nospam nospam.com> wrote:
 
 aarti_pl wrote:
 bearophile pisze:
 aarti_pl:

 I think that proposed names exactly reflect the meaning. So I would 
 say it is perfectly consistent with D convention.
Dollars are money, but opDollar is not a member function that returns the price of a struct. So I don't agree with you, or I don't understand what you mean. Bye, bearophile
I agree. opDollar is not particularly fitting to D language operator concept. opLength/opSize would fit better.
Unfortunately $ is not necessarily the length, nor the size. It might not even be an arithmetic type.
opEnd, then?
That was the only other viable suggestion. But I don't think it's very intuitive -- eg, it sounds like there ought to be an opBegin(); why isn't it used for ranges?; etc. I'd go for opLength() if it really was a length, but it's not. The big thing in favour of opDollar() is that everyone instantly knows exactly what it means.
Nov 20 2009
prev sibling parent bearophile <bearophileHUGS lycos.con> writes:
Don:
 Unfortunately $ is not necessarily the length, nor the size. It might 
 not even be an arithmetic type.
Is opEnd a fitter name? Bye, bearophile
Nov 20 2009
prev sibling parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
aarti_pl wrote:
<snip>
 I agree. opDollar is not particularly fitting to D language operator 
 concept. opLength/opSize would fit better.
<snip> Why I believe opLength and opSize are also wrong names: http://www.digitalmars.com/d/archives/digitalmars/D/announce/Re_opDollar_12939.html http://d.puremagic.com/issues/show_bug.cgi?id=3474 Stewart.
Nov 20 2009
next sibling parent reply Justin Johansson <no spam.com> writes:
Stewart Gordon wrote:
 aarti_pl wrote:
 <snip>
 I agree. opDollar is not particularly fitting to D language operator 
 concept. opLength/opSize would fit better.
<snip> Why I believe opLength and opSize are also wrong names: http://www.digitalmars.com/d/archives/digitalmars/D/announce/Re opDollar_12939.html http://d.puremagic.com/issues/show_bug.cgi?id=3474 Stewart.
FWIW, another suggestion: opCount Though I'm unsure if that is also the wrong name by your criteria. -- Justin
Nov 20 2009
parent reply Don <nospam nospam.com> writes:
Justin Johansson wrote:
 Stewart Gordon wrote:
 aarti_pl wrote:
 <snip>
 I agree. opDollar is not particularly fitting to D language operator 
 concept. opLength/opSize would fit better.
<snip> Why I believe opLength and opSize are also wrong names: http://www.digitalmars.com/d/archives/digitalmars/D/announce/Re opDollar_12939.html http://d.puremagic.com/issues/show_bug.cgi?id=3474 Stewart.
FWIW, another suggestion: opCount Though I'm unsure if that is also the wrong name by your criteria. -- Justin
Like opSize(), opCount() only makes sense for integers.
Nov 20 2009
parent reply "Denis Koroskin" <2korden gmail.com> writes:
On Sat, 21 Nov 2009 09:06:53 +0300, Don <nospam nospam.com> wrote:

 Justin Johansson wrote:
 Stewart Gordon wrote:
 aarti_pl wrote:
 <snip>
 I agree. opDollar is not particularly fitting to D language operator  
 concept. opLength/opSize would fit better.
<snip> Why I believe opLength and opSize are also wrong names: http://www.digitalmars.com/d/archives/digitalmars/D/announce/Re opDollar_12939.html http://d.puremagic.com/issues/show_bug.cgi?id=3474 Stewart.
FWIW, another suggestion: opCount Though I'm unsure if that is also the wrong name by your criteria. -- Justin
Like opSize(), opCount() only makes sense for integers.
opDim(ension)?
Nov 21 2009
parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
Denis Koroskin wrote:
 On Sat, 21 Nov 2009 09:06:53 +0300, Don <nospam nospam.com> wrote:
 
 Justin Johansson wrote:
 Stewart Gordon wrote:
<snip>
 Why I believe opLength and opSize are also wrong names:

 http://www.digitalmars.com/d/archives/digitalmars/D/announce/Re
opDollar_12939.html 
 http://d.puremagic.com/issues/show_bug.cgi?id=3474

 Stewart.
FWIW, another suggestion: opCount Though I'm unsure if that is also the wrong name by your criteria. -- Justin
Like opSize(), opCount() only makes sense for integers.
opDim(ension)?
You've lost me.... Stewart.
Nov 21 2009
parent reply Justin Johansson <no spam.com> writes:
Stewart Gordon wrote:
 Denis Koroskin wrote:
 On Sat, 21 Nov 2009 09:06:53 +0300, Don <nospam nospam.com> wrote:

 Justin Johansson wrote:
 Stewart Gordon wrote:
<snip>
 Why I believe opLength and opSize are also wrong names:

 http://www.digitalmars.com/d/archives/digitalmars/D/announce/Re
opDollar_12939.html 
 http://d.puremagic.com/issues/show_bug.cgi?id=3474

 Stewart.
FWIW, another suggestion: opCount Though I'm unsure if that is also the wrong name by your criteria. -- Justin
Like opSize(), opCount() only makes sense for integers.
opDim(ension)?
You've lost me.... Stewart.
Me too. Another suggestion: opRational. I jest :-) --Justin
Nov 21 2009
parent Travis Boucher <boucher.travis gmail.com> writes:
Justin Johansson wrote:
 Stewart Gordon wrote:
 Denis Koroskin wrote:
 On Sat, 21 Nov 2009 09:06:53 +0300, Don <nospam nospam.com> wrote:

 Justin Johansson wrote:
 Stewart Gordon wrote:
<snip>
 Why I believe opLength and opSize are also wrong names:

 http://www.digitalmars.com/d/archives/digitalmars/D/announce/Re
opDollar_12939.html 
 http://d.puremagic.com/issues/show_bug.cgi?id=3474

 Stewart.
FWIW, another suggestion: opCount Though I'm unsure if that is also the wrong name by your criteria. -- Justin
Like opSize(), opCount() only makes sense for integers.
opDim(ension)?
You've lost me.... Stewart.
Me too. Another suggestion: opRational. I jest :-) --Justin
Is that something like opOp? The operation you define to define operations for new operations?
Nov 21 2009
prev sibling parent reply Gerrit Wichert <gw green-stores.de> writes:
how about opLimit ?
Nov 24 2009
parent reply "Denis Koroskin" <2korden gmail.com> writes:
On Tue, 24 Nov 2009 14:00:18 +0300, Gerrit Wichert <gw green-stores.de>  
wrote:

 how about opLimit ?
I recall that Visual Basic has UBound function that returns upper bound of a multi-dimensional array: Dim a(100, 5, 4) As Byte UBound(a, 1) -> 100 UBound(a, 2) -> 5 UBound(a, 3) -> 4 Works for single-dimensional arrays, too: Dim b(8) As Byte UBound(b) -> 8
Nov 24 2009
parent reply Travis Boucher <boucher.travis gmail.com> writes:
Denis Koroskin wrote:
 On Tue, 24 Nov 2009 14:00:18 +0300, Gerrit Wichert <gw green-stores.de> 
 wrote:
 
 how about opLimit ?
I recall that Visual Basic has UBound function that returns upper bound of a multi-dimensional array: Dim a(100, 5, 4) As Byte UBound(a, 1) -> 100 UBound(a, 2) -> 5 UBound(a, 3) -> 4 Works for single-dimensional arrays, too: Dim b(8) As Byte UBound(b) -> 8
See the length property. char[] a = "Hello, World!"; // a.length = 13
Nov 24 2009
parent reply "Denis Koroskin" <2korden gmail.com> writes:
On Wed, 25 Nov 2009 00:46:39 +0300, Travis Boucher  
<boucher.travis gmail.com> wrote:

 Denis Koroskin wrote:
 On Tue, 24 Nov 2009 14:00:18 +0300, Gerrit Wichert <gw green-stores.de>  
 wrote:

 how about opLimit ?
I recall that Visual Basic has UBound function that returns upper bound of a multi-dimensional array: Dim a(100, 5, 4) As Byte UBound(a, 1) -> 100 UBound(a, 2) -> 5 UBound(a, 3) -> 4 Works for single-dimensional arrays, too: Dim b(8) As Byte UBound(b) -> 8
See the length property. char[] a = "Hello, World!"; // a.length = 13
Thanks, but... This thread is actually about discussing different names for a $ operator. I brought a point that VB has a UBound function that does exactly what opDollar is supposed to do, so something like opUpperBound() might fit.
Nov 25 2009
parent reply Don <nospam nospam.com> writes:
Denis Koroskin wrote:
  I recall that Visual Basic has UBound function that returns upper 
 bound of a multi-dimensional array:
  Dim a(100, 5, 4) As Byte
  UBound(a, 1) -> 100
 UBound(a, 2) -> 5
 UBound(a, 3) -> 4
  Works for single-dimensional arrays, too:
  Dim b(8) As Byte
 UBound(b) -> 8
 I brought a point that VB has a UBound function that 
 does exactly what opDollar is supposed to do, so something like 
 opUpperBound() might fit.
Finally, a viable alternative to opDollar! I could live with opUpperBound.
Nov 25 2009
parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 11/25/2009 10:46 AM, Don wrote:
 Denis Koroskin wrote:
 I recall that Visual Basic has UBound function that returns upper
 bound of a multi-dimensional array:
 Dim a(100, 5, 4) As Byte
 UBound(a, 1) -> 100
 UBound(a, 2) -> 5
 UBound(a, 3) -> 4
 Works for single-dimensional arrays, too:
 Dim b(8) As Byte
 UBound(b) -> 8
 I brought a point that VB has a UBound function that does exactly what
 opDollar is supposed to do, so something like opUpperBound() might fit.
Finally, a viable alternative to opDollar! I could live with opUpperBound.
<nitpick> VB's ubound doesn't do exactly the same thing as $; in your code snippet b(0) b(8) are both valid elements. Does opUpperBound imply an opLowerBound? In VB you can declare things like dim a(20 to 100, 5, 1 to 4) as Byte LBound(a,1) -> 20 Yep. Visual Basic. Awesome language. *Cough*
Nov 25 2009
next sibling parent reply "Denis Koroskin" <2korden gmail.com> writes:
On Wed, 25 Nov 2009 21:11:48 +0300, Ellery Newcomer  
<ellery-newcomer utulsa.edu> wrote:

 On 11/25/2009 10:46 AM, Don wrote:
 Denis Koroskin wrote:
 I recall that Visual Basic has UBound function that returns upper
 bound of a multi-dimensional array:
 Dim a(100, 5, 4) As Byte
 UBound(a, 1) -> 100
 UBound(a, 2) -> 5
 UBound(a, 3) -> 4
 Works for single-dimensional arrays, too:
 Dim b(8) As Byte
 UBound(b) -> 8
 I brought a point that VB has a UBound function that does exactly what
 opDollar is supposed to do, so something like opUpperBound() might fit.
Finally, a viable alternative to opDollar! I could live with opUpperBound.
<nitpick> VB's ubound doesn't do exactly the same thing as $; in your code snippet b(0) b(8) are both valid elements. Does opUpperBound imply an opLowerBound? In VB you can declare things like dim a(20 to 100, 5, 1 to 4) as Byte LBound(a,1) -> 20 Yep. Visual Basic. Awesome language. *Cough*
Lower bound is always 0 in D, unlike VB where is can take an arbitrary value. As such, there is no need for opLowerBound in D.
Nov 25 2009
next sibling parent =?UTF-8?B?UGVsbGUgTcOlbnNzb24=?= <pelle.mansson gmail.com> writes:
Denis Koroskin wrote:
 On Wed, 25 Nov 2009 21:11:48 +0300, Ellery Newcomer 
 <ellery-newcomer utulsa.edu> wrote:
 
 On 11/25/2009 10:46 AM, Don wrote:
 Denis Koroskin wrote:
 I recall that Visual Basic has UBound function that returns upper
 bound of a multi-dimensional array:
 Dim a(100, 5, 4) As Byte
 UBound(a, 1) -> 100
 UBound(a, 2) -> 5
 UBound(a, 3) -> 4
 Works for single-dimensional arrays, too:
 Dim b(8) As Byte
 UBound(b) -> 8
 I brought a point that VB has a UBound function that does exactly what
 opDollar is supposed to do, so something like opUpperBound() might fit.
Finally, a viable alternative to opDollar! I could live with opUpperBound.
<nitpick> VB's ubound doesn't do exactly the same thing as $; in your code snippet b(0) b(8) are both valid elements. Does opUpperBound imply an opLowerBound? In VB you can declare things like dim a(20 to 100, 5, 1 to 4) as Byte LBound(a,1) -> 20 Yep. Visual Basic. Awesome language. *Cough*
Lower bound is always 0 in D, unlike VB where is can take an arbitrary value. As such, there is no need for opLowerBound in D.
Why does it make any sense that the lower bound of any arbitrary class needs to be 0? I'd say opUpperBound is as wrong as opEnd.
Nov 25 2009
prev sibling parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
Denis Koroskin wrote:
<snip>
 Lower bound is always 0 in D, unlike VB where is can take an arbitrary 
 value. As such, there is no need for opLowerBound in D.
Only for built-in linear arrays. Half the point is: What if somebody creates a type for which the lower bound is something different? Thinking about it, maybe we need some symbol like ^ to denote the lower bound, and opLowerBound to implement it. (I've picked ^ along the lines of regexps, from which $ is presumably derived. I *think* this doesn't lead to any ambiguity....) Stewart.
Nov 27 2009
parent reply bearophile <bearophileHUGS lycos.com> writes:
Stewart Gordon:
 Only for built-in linear arrays.  Half the point is: What if somebody 
 creates a type for which the lower bound is something different?
This can be useful. For example an array with indices in a .. z+1 :-)
 maybe we need some symbol like ^ to denote the lower 
 bound, and opLowerBound to implement it.  (I've picked ^ along the lines 
 of regexps, from which $ is presumably derived.  I *think* this doesn't 
 lead to any ambiguity....)
In Python you usually just omit the value: a[:5] === a[0 .. 5] a[5:] === a[5 .. $] Or you can also use None, this can useful because you can put None inside a variable, etc (while in D you can't put $ inside a variable to represent "the end of that array"): a[None:5] === a[0 .. 5] a[5:None] === a[5 .. $] Bye, bearophile
Nov 28 2009
parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
bearophile wrote:
<snip>
 In Python you usually just omit the value:
 a[:5] === a[0 .. 5]
 a[5:] === a[5 .. $]
Which doesn't accommodate anything equivalent to a[$-4 .. $-2].
 Or you can also use None, this can useful because you can put None 
 inside a variable, etc (while in D you can't put $ inside a variable 
 to represent "the end of that array"):
<snip> I don't understand this statement at all. Stewart.
Dec 07 2009
next sibling parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 12/07/2009 08:33 PM, Stewart Gordon wrote:
 bearophile wrote:
 <snip>
 In Python you usually just omit the value:
 a[:5] === a[0 .. 5]
 a[5:] === a[5 .. $]
Which doesn't accommodate anything equivalent to a[$-4 .. $-2].
you mean this? a[-4:-2]
 Or you can also use None, this can useful because you can put None
 inside a variable, etc (while in D you can't put $ inside a variable
 to represent "the end of that array"):
<snip> I don't understand this statement at all.
neither can I. I'll posit a guess though. c = None; assert a[1:c] == a[1:] not seeing any advantage over c = len(a)
 Stewart.
Dec 07 2009
parent reply "Simen kjaeraas" <simen.kjaras gmail.com> writes:
On Tue, 08 Dec 2009 04:02:21 +0100, Ellery Newcomer  
<ellery-newcomer utulsa.edu> wrote:

 On 12/07/2009 08:33 PM, Stewart Gordon wrote:
 bearophile wrote:
 <snip>
 In Python you usually just omit the value:
 a[:5] === a[0 .. 5]
 a[5:] === a[5 .. $]
Which doesn't accommodate anything equivalent to a[$-4 .. $-2].
you mean this? a[-4:-2]
That, however (like D) does not support arrays with negative indices. -- Simen
Dec 08 2009
parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
Simen kjaeraas wrote:
 On Tue, 08 Dec 2009 04:02:21 +0100, Ellery Newcomer 
 <ellery-newcomer utulsa.edu> wrote:
 
 On 12/07/2009 08:33 PM, Stewart Gordon wrote:
 bearophile wrote:
 <snip>
 In Python you usually just omit the value:
 a[:5] === a[0 .. 5]
 a[5:] === a[5 .. $]
Which doesn't accommodate anything equivalent to a[$-4 .. $-2].
you mean this? a[-4:-2]
That, however (like D) does not support arrays with negative indices.
But in D, you can use a negative index/key in an AA or custom array type. And is there any equivalent in Python to a[$/2] or anything fancy like that? Stewart.
Dec 10 2009
parent bearophile <bearophileHUGS lycos.com> writes:
Stewart Gordon:
 But in D, you can use a negative index/key in an AA or custom array type.
This is natural & easy to do in Python too.
 And is there any equivalent in Python to a[$/2] or anything fancy like that?
You have to do it in explicit way: a[len(a) / 2] Or today better (// is the integer division): a[len(a) // 2] Bye, bearophile
Dec 10 2009
prev sibling parent bearophile <bearophileHUGS lycos.com> writes:
Stewart Gordon:

Which doesn't accommodate anything equivalent to a[$-4 .. $-2].<
In Python:
 a = "abcdefgh"
 a[-4 : -2]
'ef'
 Or you can also use None, this can useful because you can put None
 inside a variable, etc (while in D you can't put $ inside a variable
 to represent "the end of that array"):
<snip>
I don't understand this statement at all.<
You are right and I am sorry, let's try again. In Python slices using None is the same as omitting the value: a[:5] === a[None : 5] a[5:] === a[5 : None] Omitting a value is handy, but nothing is not a value you can pass around and store in a variable, while None allows you to do that, so it gives more flexibility. This is a little example: def foo(start, stop): a = "abcdefgh" print a[start : stop] foo(1, 2) foo(None, 2) foo(2, None) foo(None, None) Output: b ab cdefgh abcdefgh You can't do that with omitted values, and $ too can't be moved around in a variable, so None is more flexible. (I know you can't do that in D, you need nullable integers, for example). I hope this explanation was a little more clear. Bye, bearophile
Dec 07 2009
prev sibling parent reply "Denis Koroskin" <2korden gmail.com> writes:
On Wed, 25 Nov 2009 21:11:48 +0300, Ellery Newcomer  
<ellery-newcomer utulsa.edu> wrote:

 On 11/25/2009 10:46 AM, Don wrote:
 Denis Koroskin wrote:
 I recall that Visual Basic has UBound function that returns upper
 bound of a multi-dimensional array:
 Dim a(100, 5, 4) As Byte
 UBound(a, 1) -> 100
 UBound(a, 2) -> 5
 UBound(a, 3) -> 4
 Works for single-dimensional arrays, too:
 Dim b(8) As Byte
 UBound(b) -> 8
 I brought a point that VB has a UBound function that does exactly what
 opDollar is supposed to do, so something like opUpperBound() might fit.
Finally, a viable alternative to opDollar! I could live with opUpperBound.
<nitpick> VB's ubound doesn't do exactly the same thing as $; in your code snippet b(0) b(8) are both valid elements.
IIRC lower bound is 1 by default in VB and therefore b(0) is illegal.
 Does opUpperBound imply an opLowerBound?

 In VB you can declare things like

 dim a(20 to 100, 5, 1 to 4) as Byte

 LBound(a,1) -> 20

 Yep. Visual Basic. Awesome language. *Cough*
-- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
Nov 25 2009
parent Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 11/25/2009 01:40 PM, Denis Koroskin wrote:
 IIRC lower bound is 1 by default in VB and therefore b(0) is illegal.
Nope. Dim b(8) as String is equivalent to Dim b(0 to 8) as String and therefore b(0) is very legal.
Nov 25 2009
prev sibling next sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
aarti_pl wrote:
 I know that quite a few people here doesn't like to allow users to 
 define their own operators, because it might obfuscate code. But it 
 doesn't have to be like this. Someone here already mentioned here that 
 it is not real problem for programs in C++. Good libraries don't abuse 
 this functionality.
The problem with user defined operators is: 1. User defined tokens - mixes up lexing with semantic analysis 2. User defined syntax - mixes up parsing with semantic analysis and then we're in C++ land :-( Unless such have a unique grammar that can be lexed and parsed: a :string: b where string is the user defined name, so you can do things like: a :^^: b and define your own pow operator. The problem with this approach is the sheer ugliness of it.
Nov 19 2009
next sibling parent reply KennyTM~ <kennytm gmail.com> writes:
On Nov 20, 09 05:26, Walter Bright wrote:
 aarti_pl wrote:
 I know that quite a few people here doesn't like to allow users to
 define their own operators, because it might obfuscate code. But it
 doesn't have to be like this. Someone here already mentioned here that
 it is not real problem for programs in C++. Good libraries don't abuse
 this functionality.
The problem with user defined operators is: 1. User defined tokens - mixes up lexing with semantic analysis 2. User defined syntax - mixes up parsing with semantic analysis and then we're in C++ land :-( Unless such have a unique grammar that can be lexed and parsed: a :string: b where string is the user defined name, so you can do things like: a :^^: b and define your own pow operator. The problem with this approach is the sheer ugliness of it.
a /pow/ b is already implementable... Speaking of which, how to map opDiv_r into its opBinary!()() equivalent?
Nov 19 2009
parent Bill Baxter <wbaxter gmail.com> writes:
On Thu, Nov 19, 2009 at 1:35 PM, KennyTM~ <kennytm gmail.com> wrote:

 Speaking of which, how to map opDiv_r into its opBinary!()() equivalent?
I would guess there will simply be an opBinary_r!("/") in addition to opBinary!("/"). --bb
Nov 19 2009
prev sibling next sibling parent reply "Adam D. Ruppe" <destructionator gmail.com> writes:
On Thu, Nov 19, 2009 at 01:26:41PM -0800, Walter Bright wrote:
 Unless such have a unique grammar that can be lexed and parsed:
 
     a :string: b
 
 where string is the user defined name, so you can do things like:
 
     a :^^: b
What if there was some magic defined at the top of the file or something, that could be pulled out without parsing the whole thing? pragma(DEFINE_BINARY_OPERATOR, "^^"); Then, later in that module, and any that import it, you'd treat the ^^ as a user defined operator token. -- Adam D. Ruppe http://arsdnet.net
Nov 19 2009
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Adam D. Ruppe wrote:
 What if there was some magic defined at the top of the file or something, that
 could be pulled out without parsing the whole thing?
 
 pragma(DEFINE_BINARY_OPERATOR, "^^");
 
 Then, later in that module, and any that import it, you'd treat the ^^ as
 a user defined operator token.
Still mixing up lexing, parsing, and semantic analysis. I know it's tempting, but it only leads to madness.
Nov 19 2009
parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Thu, Nov 19, 2009 at 04:00:25PM -0800, Walter Bright wrote:
 I know it's tempting, but it only leads to madness.
Besides, I think this is something that a lot of people might ask for, but then once they got it, they wouldn't actually use it. Imagine getting a library that uses a really cool character as an operator... just to find that you can't type that character on your keyboard anyway. Or maybe you can, but it is a big pain, so it isn't worth the effort. Or getting that cool character in source that you edit in a primitive editor or font that can't display it, so you just see gibberish. In theory, those aren't problems. But in practice, I think that would make the feature very rarely used. There just aren't that many easy to type / ubiquitous to display symbols that have meaning not already defined in the language. -- Adam D. Ruppe http://arsdnet.net
Nov 19 2009
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Thu, Nov 19, 2009 at 04:46:55PM -0500, Adam D. Ruppe wrote:
 What if there was some magic defined at the top of the file or something, that
 could be pulled out without parsing the whole thing?
I should add that I'm not really sold on the idea of user defined operators at all - I'm just curious about how it might be done in the compiler. -- Adam D. Ruppe http://arsdnet.net
Nov 19 2009
prev sibling next sibling parent retard <re tard.com.invalid> writes:
Thu, 19 Nov 2009 13:26:41 -0800, Walter Bright wrote:

 aarti_pl wrote:
 I know that quite a few people here doesn't like to allow users to
 define their own operators, because it might obfuscate code. But it
 doesn't have to be like this. Someone here already mentioned here that
 it is not real problem for programs in C++. Good libraries don't abuse
 this functionality.
The problem with user defined operators is: 1. User defined tokens - mixes up lexing with semantic analysis 2. User defined syntax - mixes up parsing with semantic analysis
Some languages have syntactic rule extensions which allows defining symbols constructed either from characters [a-zA-Z][a-zA-Z0-9]* (like C) or from special characters like <>:;,.-/\ etc. Another feature is that in those languages operators are just normal functions with extra syntax (fixity, associativity, etc.) This way even "built-in" operations on ints etc. can be defined in the library.
 and then we're in C++ land :-(
 
 Unless such have a unique grammar that can be lexed and parsed:
 
      a :string: b
 
 where string is the user defined name, so you can do things like:
 
      a :^^: b
 
 and define your own pow operator. The problem with this approach is the
 sheer ugliness of it.
Haskell uses a `myFun` b.
Nov 19 2009
prev sibling next sibling parent Bill Baxter <wbaxter gmail.com> writes:
On Thu, Nov 19, 2009 at 1:26 PM, Walter Bright
<newshound1 digitalmars.com> wrote:
 aarti_pl wrote:
 I know that quite a few people here doesn't like to allow users to defin=
e
 their own operators, because it might obfuscate code. But it doesn't hav=
e to
 be like this. Someone here already mentioned here that it is not real
 problem for programs in C++. Good libraries don't abuse this functionali=
ty.
 The problem with user defined operators is:

 =A01. User defined tokens - mixes up lexing with semantic analysis

 =A02. User defined syntax - mixes up parsing with semantic analysis

 and then we're in C++ land :-(

 Unless such have a unique grammar that can be lexed and parsed:

 =A0 =A0a :string: b

 where string is the user defined name, so you can do things like:

 =A0 =A0a :^^: b

 and define your own pow operator. The problem with this approach is the
 sheer ugliness of it.
You also have to wedge precedence in there somehow. Maybe instead of :string: you use +string+ or *string* and the first/last character tells the precedence. Oh wait, downs already has code that does this. The question, I think, is how to make it reasonably fast and less cumbersome to declare. Though I'm not sure it's really worth the effort. --bb
Nov 19 2009
prev sibling next sibling parent aarti_pl <aarti interia.pl> writes:
Walter Bright pisze:
 aarti_pl wrote:
 I know that quite a few people here doesn't like to allow users to 
 define their own operators, because it might obfuscate code. But it 
 doesn't have to be like this. Someone here already mentioned here that 
 it is not real problem for programs in C++. Good libraries don't abuse 
 this functionality.
The problem with user defined operators is: 1. User defined tokens - mixes up lexing with semantic analysis 2. User defined syntax - mixes up parsing with semantic analysis and then we're in C++ land :-(
"retard" probably already answered it. I was thinking about something like below: identifier = '[a-zA-Z][a-zA-Z0-9]*' //quick & dirty definition Prefix operator: 1. [symbols][whitespace][identifier] 2. [identifier]whitespace[identifier] Postfix operator: 1. [identifier][whitespace][symbols] 2. [identifier]whitespace[identifier] Infix operator: 1. [identifier][whitespace][symbols][whitespace][identifier] 2. [identifier]whitespace[identifier]whitespace[identifier] With precedence defined as above: first prefix, then postfix, then infix. Should work...
 Unless such have a unique grammar that can be lexed and parsed:
 
     a :string: b
 
 where string is the user defined name, so you can do things like:
 
     a :^^: b
 
 and define your own pow operator. The problem with this approach is the 
 sheer ugliness of it.
This might be also acceptable. Especially with `` syntax. (I know it's currently for strings). ---- I believe that user defined operators will have positive impact on D language. Please notice that always when you have more power, then you can make more bad things. But you can also do more good things! And doing good things or doing bad things is not something what should programming language care of. Programming language is IMHO just for giving good tools. User should decide how to use them. I think there aren't enough good reasons to introduce new syntax if it will not allow user defined operators. It will be rather source of confusion if we only allow to pass few symbols as strings: strings are about 'freedom' of content. So using it for passing just a few symbols is for me "corner case". BR Marcin Kuszczak (aarti_pl)
Nov 19 2009
prev sibling next sibling parent Michel Fortin <michel.fortin michelf.com> writes:
On 2009-11-19 16:26:41 -0500, Walter Bright <newshound1 digitalmars.com> said:

 Unless such have a unique grammar that can be lexed and parsed:
 
      a :string: b
 
 where string is the user defined name, so you can do things like:
 
      a :^^: b
 
 and define your own pow operator. The problem with this approach is the 
 sheer ugliness of it.
By the way, if you could omit the parenthesis for a one-argument function call, you could write things like this: a .pow b -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Nov 19 2009
prev sibling parent reply Don <nospam nospam.com> writes:
Walter Bright wrote:
 aarti_pl wrote:
 I know that quite a few people here doesn't like to allow users to 
 define their own operators, because it might obfuscate code. But it 
 doesn't have to be like this. Someone here already mentioned here that 
 it is not real problem for programs in C++. Good libraries don't abuse 
 this functionality.
The problem with user defined operators is: 1. User defined tokens - mixes up lexing with semantic analysis 2. User defined syntax - mixes up parsing with semantic analysis and then we're in C++ land :-( Unless such have a unique grammar that can be lexed and parsed: a :string: b where string is the user defined name, so you can do things like: a :^^: b and define your own pow operator. The problem with this approach is the sheer ugliness of it.
There's not many sensible operators anyway. opPow is the only missing one that's present in many other general-purpose languages. The only other ones I think are remotely interesting are dot and cross product. Anything beyond that, you generally want a full DSL, probably with different precendence and associativity rules. Eg, for regexp, you'd want postfix * and + operators. There are examples of clever things done in C++ with operator overloading, but I think that's just because it's the only way to do DSLs in C++. I don't think the applications are there.
Nov 20 2009
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Don wrote:
 There's not many sensible operators anyway. opPow is the only missing 
 one that's present in many other general-purpose languages. The only 
 other ones I think are remotely interesting are dot and cross product.
Yup.
 Anything beyond that, you generally want a full DSL, probably with 
 different precendence and associativity rules. Eg, for regexp, you'd 
 want postfix * and + operators. There are examples of clever things done 
 in C++  with operator overloading, but I think that's just because it's 
 the only way to do DSLs in C++.
I was enthralled with the way C++ did it for regex for a while, but when I think more about it, it's just too clever. I think it's more operator overloading abuse now.
 I don't think the applications are there.
I agree.
Nov 20 2009
parent reply aarti_pl <aarti interia.pl> writes:
Walter Bright pisze:
 Don wrote:
 There's not many sensible operators anyway. opPow is the only missing 
 one that's present in many other general-purpose languages. The only 
 other ones I think are remotely interesting are dot and cross product.
Yup.
 Anything beyond that, you generally want a full DSL, probably with 
 different precendence and associativity rules. Eg, for regexp, you'd 
 want postfix * and + operators. There are examples of clever things 
 done in C++  with operator overloading, but I think that's just 
 because it's the only way to do DSLs in C++.
I was enthralled with the way C++ did it for regex for a while, but when I think more about it, it's just too clever. I think it's more operator overloading abuse now.
 I don't think the applications are there.
I agree.
Well, I can understand your fear about operator abuse. And I agree that code might be awful when operator overloading will be abused. But I have in mind one very convincing example. I defined in D/Java SQL syntax. They are also other frameworks which do the same. What can I say about my experiences with using such framework: it is very, very powerful concept. It cuts time necessary to develop application, makes sql statements type safe and allows to pass around parts of sql statements inside application. It also makes easy refactoring of sql statement (especially in Java). Its huge win comparing it to defining DSL as strings. It's hard to explain just in few sentences all details. I have already done it long time ago, and in my first post I provided links. Problem with current approach is that I have to define SQL in D/Java in following way: auto statement = Select(visitcars.name).Where(And(More(visitcards.id, 100), Like(visitcards.surname, "A*"))); Please look at code in Where(). It's so awfuuuuulllllll! It would be so much better to write: auto statement = Select(visitcars.name).Where((visitcards.id `>` 100) `AND` (visitcards.surname `Like` "A*")); I used here syntax which you have proposed with delimiter ``. I think it is good enough solution for such purpose. But please, don't underestimate problem! Many DSL languages would never appear if languages would be good enough. As I said solution with delimiter is good enough for me. It has another advantage that it clearly shows in code that you have overloaded operator here, so no surprises here. Additionally when you implement template function: opInfix('AND')(val0, val1); you pass string into template. So I see it quite intuitive that you use string as operator: ``. Maybe there will be not necessary to change current behavior that `` defines string. I think we have good possibility to open this door now. It can be even implemented later, but I would wish just not to close this door now :-) BR Marcin Kuszczak (aarti_pl)
Nov 21 2009
parent "Phil Deets" <pjdeets2 gmail.com> writes:
On Sat, 21 Nov 2009 13:21:10 -0500, aarti_pl <aarti interia.pl> wrote:
 [snip]

 Problem with current approach is that I have to define SQL in D/Java in  
 following way:

 auto statement = Select(visitcars.name).Where(And(More(visitcards.id,  
 100), Like(visitcards.surname, "A*")));

 Please look at code in Where(). It's so awfuuuuulllllll!

 It would be so much better to write:
 auto statement = Select(visitcars.name).Where((visitcards.id `>` 100)  
 `AND` (visitcards.surname `Like` "A*"));

 I used here syntax which you have proposed with delimiter ``. I think it  
 is good enough solution for such purpose.

 [snip]
Would something like expression trees (http://msdn.microsoft.com/en-us/library/bb397951.aspx) suit your needs?
Nov 21 2009
prev sibling parent Travis Boucher <boucher.travis gmail.com> writes:
aarti_pl wrote:
 Andrei Alexandrescu pisze:
 We're entering the finale of D2 and I want to keep a short list of 
 things that must be done and integrated in the release. It is clearly 
 understood by all of us that there are many things that could and 
 probably should be done.

 1. Currently Walter and Don are diligently fixing the problems marked 
 on the current manuscript.

 2. User-defined operators must be revamped. Fortunately Don already 
 put in an important piece of functionality (opDollar). What we're 
 looking at is a two-pronged attack motivated by Don's proposal:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7

 The two prongs are:

 * Encode operators by compile-time strings. For example, instead of 
 the plethora of opAdd, opMul, ..., we'd have this:

 T opBinary(string op)(T rhs) { ... }

 The string is "+", "*", etc. We need to design what happens with 
 read-modify-write operators like "+=" (should they be dispatch to a 
 different function? etc.) and also what happens with index-and-modify 
 operators like "[]=", "[]+=" etc. Should we go with proxies? Absorb 
 them in opBinary? Define another dedicated method? etc.

 * Loop fusion that generalizes array-wise operations. This idea of 
 Walter is, I think, very good because it generalizes and democratizes 
 "magic". The idea is that, if you do

 a = b + c;

 and b + c does not make sense but b and c are ranges for which a.front 
 = b.front + c.front does make sense, to automatically add the 
 iteration paraphernalia.

 3. It was mentioned in this group that if getopt() does not work in 
 SafeD, then SafeD may as well pack and go home. I agree. We need to 
 make it work. Three ideas discussed with Walter:

 * Allow taking addresses of locals, but in that case switch allocation 
 from stack to heap, just like with delegates. If we only do that in 
 SafeD, behavior will be different than with regular D. In any case, 
 it's an inefficient proposition, particularly for getopt() which 
 actually does not need to escape the addresses - just fills them up.

 * Allow  trusted (and maybe even  safe) functions to receive addresses 
 of locals. Statically check that they never escape an address of a 
 parameter. I think this is very interesting because it enlarges the 
 common ground of D and SafeD.

 * Figure out a way to reconcile "ref" with variadics. This is the 
 actual reason why getopt chose to traffic in addresses, and fixing it 
 is the logical choice and my personal favorite.

 4. Allow private members inside a template using the eponymous trick:

 template wyda(int x) {
    private enum geeba = x / 2;
    alias geeba wyda;
 }

 The names inside an eponymous template are only accessible to the 
 current instantiation. For example, wyda!5 cannot access 
 wyda!(4).geeba, only its own geeba. That we we elegantly avoid the 
 issue "where is this symbol looked up?"

 5. Chain exceptions instead of having a recurrent exception terminate 
 the program. I'll dedicate a separate post to this.

 6. There must be many things I forgot to mention, or that cause grief 
 to many of us. Please add to/comment on this list.



 Andrei
I kinda like this proposal. But I would rather call template like below: T opInfix(string op)(T rhs) { ... } T opPrefix(string op)(T rhs) { ... } T opPostfix(string op)(T rhs) { ... } and allow user to define her own operators (though it doesn't have to be done now). I know that quite a few people here doesn't like to allow users to define their own operators, because it might obfuscate code. But it doesn't have to be like this. Someone here already mentioned here that it is not real problem for programs in C++. Good libraries don't abuse this functionality. User defined operators would allow easy definition of Domain Specific Languages in D. I was already writing about it some time ago: http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmar .D&article_id=81026 http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmar .D&article_id=81352 BR Marcin Kuszczak (aarti_pl)
Sweet, I've been waiting for a way to implement brainfuck using operators! auto bf = new BrainFuck(); bf++++++++++[>+++++++>++++++++++>+++>+<<<<-]>++.>+.+++++++..+++.>++.<<+++++++++++++++.>.+++.------.--------.>+.>.; writef(bf.toString()); // outputs "Hello World!\n"
Nov 19 2009
prev sibling next sibling parent reply =?ISO-8859-1?Q?Pelle_M=E5nsson?= <pelle.mansson gmail.com> writes:
Andrei Alexandrescu wrote:
 * Encode operators by compile-time strings. For example, instead of the 
 plethora of opAdd, opMul, ..., we'd have this:
 
 T opBinary(string op)(T rhs) { ... }
 
 The string is "+", "*", etc. We need to design what happens with 
 read-modify-write operators like "+=" (should they be dispatch to a 
 different function? etc.) and also what happens with index-and-modify 
 operators like "[]=", "[]+=" etc. Should we go with proxies? Absorb them 
 in opBinary? Define another dedicated method? etc.
 
 Andrei
What about pure, what about const? Will we need to pure T opBinary(string op)(T rhs) if (op == "+" || op == "-") { static if (op == "+") { /* ... */ } else static if (op == "-") { /* ... */ } } T opBinary(string op)(T rhs) if (op == "+=" || ...) { // more static if's } thereby duplicating every case?
Nov 20 2009
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Pelle Mnsson wrote:
 Andrei Alexandrescu wrote:
 * Encode operators by compile-time strings. For example, instead of 
 the plethora of opAdd, opMul, ..., we'd have this:

 T opBinary(string op)(T rhs) { ... }

 The string is "+", "*", etc. We need to design what happens with 
 read-modify-write operators like "+=" (should they be dispatch to a 
 different function? etc.) and also what happens with index-and-modify 
 operators like "[]=", "[]+=" etc. Should we go with proxies? Absorb 
 them in opBinary? Define another dedicated method? etc.

 Andrei
What about pure, what about const? Will we need to pure T opBinary(string op)(T rhs) if (op == "+" || op == "-") { static if (op == "+") { /* ... */ } else static if (op == "-") { /* ... */ } } T opBinary(string op)(T rhs) if (op == "+=" || ...) { // more static if's } thereby duplicating every case?
Well you're better off than before anyway - you get to group operators in pure and impure. Andrei
Nov 20 2009
prev sibling next sibling parent Stewart Gordon <smjg_1998 yahoo.com> writes:
Andrei Alexandrescu wrote:
 We're entering the finale of D2 and I want to keep a short list of 
 things that must be done and integrated in the release. It is clearly 
 understood by all of us that there are many things that could and 
 probably should be done.
What do you mean by finale, exactly? <snip>
 * Encode operators by compile-time strings. For example, instead of the 
 plethora of opAdd, opMul, ..., we'd have this:
 
 T opBinary(string op)(T rhs) { ... }
 
 The string is "+", "*", etc. We need to design what happens with 
 read-modify-write operators like "+=" (should they be dispatch to a 
 different function? etc.)
<snip> Perhaps something like T opModify(string op)(T rhs) so that a += b would be equivalent to the first of these to be valid: a = a.opModify!("+")(b) a = (a.opModify!("+")(b), a) (if opModify returns void) a = a.opBinary!("+")(b) a = b.opBinary!("+")(a) The idea is that an opModify method would likely modify the object in-place and return the modified object, but may return a new object to be assigned to a. That said, I'm not sure if there's any real use case for it being a new object distinct from simple a + b. But maybe there are cases where it may variously modify in-place or reallocate. (Is this still how ~= works in current D2?) But whatever we do, we shouldn't allow opModify to return something if that something isn't going to be assigned to a. Stewart.
Nov 20 2009
prev sibling parent reply yigal chripun <yigal100 gmail.com> writes:
aarti_pl Wrote:

 Walter Bright pisze:
 Don wrote:
 There's not many sensible operators anyway. opPow is the only missing 
 one that's present in many other general-purpose languages. The only 
 other ones I think are remotely interesting are dot and cross product.
Yup.
 Anything beyond that, you generally want a full DSL, probably with 
 different precendence and associativity rules. Eg, for regexp, you'd 
 want postfix * and + operators. There are examples of clever things 
 done in C++  with operator overloading, but I think that's just 
 because it's the only way to do DSLs in C++.
I was enthralled with the way C++ did it for regex for a while, but when I think more about it, it's just too clever. I think it's more operator overloading abuse now.
 I don't think the applications are there.
I agree.
Well, I can understand your fear about operator abuse. And I agree that code might be awful when operator overloading will be abused. But I have in mind one very convincing example. I defined in D/Java SQL syntax. They are also other frameworks which do the same. What can I say about my experiences with using such framework: it is very, very powerful concept. It cuts time necessary to develop application, makes sql statements type safe and allows to pass around parts of sql statements inside application. It also makes easy refactoring of sql statement (especially in Java). Its huge win comparing it to defining DSL as strings. It's hard to explain just in few sentences all details. I have already done it long time ago, and in my first post I provided links. Problem with current approach is that I have to define SQL in D/Java in following way: auto statement = Select(visitcars.name).Where(And(More(visitcards.id, 100), Like(visitcards.surname, "A*"))); Please look at code in Where(). It's so awfuuuuulllllll! It would be so much better to write: auto statement = Select(visitcars.name).Where((visitcards.id `>` 100) `AND` (visitcards.surname `Like` "A*")); I used here syntax which you have proposed with delimiter ``. I think it is good enough solution for such purpose. But please, don't underestimate problem! Many DSL languages would never appear if languages would be good enough. As I said solution with delimiter is good enough for me. It has another advantage that it clearly shows in code that you have overloaded operator here, so no surprises here. Additionally when you implement template function: opInfix('AND')(val0, val1); you pass string into template. So I see it quite intuitive that you use string as operator: ``. Maybe there will be not necessary to change current behavior that `` defines string. I think we have good possibility to open this door now. It can be even implemented later, but I would wish just not to close this door now :-) BR Marcin Kuszczak (aarti_pl)
There's nothing more hideous than all those frameworks in Java/C++ that try to re-enginer SQL into functions, templates, LINQ, whatever. SQL *is* a perfectly designed language for its purpose and it doesn't need to be redisnged! The only problem with this is the type-safety when embedding sql as string in a host language. the solution is two-phased: string like: "select * from table where :a > 42", the :name is a place holder for the host-language variable, and you call an API to bind those :names to variables in a type-safe way. the downside is that it's verbose. phase b is what Nemerle does with the above - it has an AST macro to wrap the above so you can write your query directly and it is checked as compile-time. No operators were abused in implementing this.
Nov 23 2009
next sibling parent reply Don <nospam nospam.com> writes:
yigal chripun wrote:
 aarti_pl Wrote:
 
 Walter Bright pisze:
 Don wrote:
 There's not many sensible operators anyway. opPow is the only missing 
 one that's present in many other general-purpose languages. The only 
 other ones I think are remotely interesting are dot and cross product.
Yup.
 Anything beyond that, you generally want a full DSL, probably with 
 different precendence and associativity rules. Eg, for regexp, you'd 
 want postfix * and + operators. There are examples of clever things 
 done in C++  with operator overloading, but I think that's just 
 because it's the only way to do DSLs in C++.
I was enthralled with the way C++ did it for regex for a while, but when I think more about it, it's just too clever. I think it's more operator overloading abuse now.
 I don't think the applications are there.
I agree.
Well, I can understand your fear about operator abuse. And I agree that code might be awful when operator overloading will be abused. But I have in mind one very convincing example. I defined in D/Java SQL syntax. They are also other frameworks which do the same. What can I say about my experiences with using such framework: it is very, very powerful concept. It cuts time necessary to develop application, makes sql statements type safe and allows to pass around parts of sql statements inside application. It also makes easy refactoring of sql statement (especially in Java). Its huge win comparing it to defining DSL as strings. It's hard to explain just in few sentences all details. I have already done it long time ago, and in my first post I provided links. Problem with current approach is that I have to define SQL in D/Java in following way: auto statement = Select(visitcars.name).Where(And(More(visitcards.id, 100), Like(visitcards.surname, "A*"))); Please look at code in Where(). It's so awfuuuuulllllll! It would be so much better to write: auto statement = Select(visitcars.name).Where((visitcards.id `>` 100) `AND` (visitcards.surname `Like` "A*")); I used here syntax which you have proposed with delimiter ``. I think it is good enough solution for such purpose. But please, don't underestimate problem! Many DSL languages would never appear if languages would be good enough. As I said solution with delimiter is good enough for me. It has another advantage that it clearly shows in code that you have overloaded operator here, so no surprises here. Additionally when you implement template function: opInfix('AND')(val0, val1); you pass string into template. So I see it quite intuitive that you use string as operator: ``. Maybe there will be not necessary to change current behavior that `` defines string. I think we have good possibility to open this door now. It can be even implemented later, but I would wish just not to close this door now :-) BR Marcin Kuszczak (aarti_pl)
There's nothing more hideous than all those frameworks in Java/C++ that try to re-enginer SQL into functions, templates, LINQ, whatever. SQL *is* a perfectly designed language for its purpose and it doesn't need to be redisnged! The only problem with this is the type-safety when embedding sql as string in a host language. the solution is two-phased: string like:
 "select * from table where :a > 42", the :name is a place holder for the
host-language variable, and you call an API to bind those :names to variables
in a type-safe way. the downside is that it's verbose. 
  
 phase b is what Nemerle does with the above - it has an AST macro to wrap the
above so you can write your query directly and it is checked as compile-time. 
 
 No operators were abused in implementing this. 
I quite agree. What we can do already is: auto statement = db.execute!(`select $a from table where $b > 100 && $c Like "A*"`)(visitcars.name,visitcars.id, visitcars.surname); which I personally like much better than the proposed goal:
 It would be so much better to write:
 auto statement = Select(visitcars.name).Where((visitcards.id `>` 100)
 `AND` (visitcards.surname `Like` "A*"));
(Replace $a with your preferred method for defining placeholder variables). And the question then is, can we improve the existing solution? And if so, how? I just don't think the solution involves overloading operators. I think this a great example of why we *don't* want arbitrary operator overloading: there's no point overloading && and > if you can't make 'from', 'where', and 'like' to all be infix operators, as well!
Nov 23 2009
parent reply Chad J <chadjoan __spam.is.bad__gmail.com> writes:
Don wrote:
 yigal chripun wrote:
 aarti_pl Wrote:

 Walter Bright pisze:
 Don wrote:
 There's not many sensible operators anyway. opPow is the only
 missing one that's present in many other general-purpose languages.
 The only other ones I think are remotely interesting are dot and
 cross product.
Yup.
 Anything beyond that, you generally want a full DSL, probably with
 different precendence and associativity rules. Eg, for regexp,
 you'd want postfix * and + operators. There are examples of clever
 things done in C++  with operator overloading, but I think that's
 just because it's the only way to do DSLs in C++.
I was enthralled with the way C++ did it for regex for a while, but when I think more about it, it's just too clever. I think it's more operator overloading abuse now.
 I don't think the applications are there.
I agree.
Well, I can understand your fear about operator abuse. And I agree that code might be awful when operator overloading will be abused. But I have in mind one very convincing example. I defined in D/Java SQL syntax. They are also other frameworks which do the same. What can I say about my experiences with using such framework: it is very, very powerful concept. It cuts time necessary to develop application, makes sql statements type safe and allows to pass around parts of sql statements inside application. It also makes easy refactoring of sql statement (especially in Java). Its huge win comparing it to defining DSL as strings. It's hard to explain just in few sentences all details. I have already done it long time ago, and in my first post I provided links. Problem with current approach is that I have to define SQL in D/Java in following way: auto statement = Select(visitcars.name).Where(And(More(visitcards.id, 100), Like(visitcards.surname, "A*"))); Please look at code in Where(). It's so awfuuuuulllllll! It would be so much better to write: auto statement = Select(visitcars.name).Where((visitcards.id `>` 100) `AND` (visitcards.surname `Like` "A*")); I used here syntax which you have proposed with delimiter ``. I think it is good enough solution for such purpose. But please, don't underestimate problem! Many DSL languages would never appear if languages would be good enough. As I said solution with delimiter is good enough for me. It has another advantage that it clearly shows in code that you have overloaded operator here, so no surprises here. Additionally when you implement template function: opInfix('AND')(val0, val1); you pass string into template. So I see it quite intuitive that you use string as operator: ``. Maybe there will be not necessary to change current behavior that `` defines string. I think we have good possibility to open this door now. It can be even implemented later, but I would wish just not to close this door now :-) BR Marcin Kuszczak (aarti_pl)
There's nothing more hideous than all those frameworks in Java/C++ that try to re-enginer SQL into functions, templates, LINQ, whatever. SQL *is* a perfectly designed language for its purpose and it doesn't need to be redisnged! The only problem with this is the type-safety when embedding sql as string in a host language. the solution is two-phased: query is one string like:
 "select * from table where :a > 42", the :name is a place holder for
 the host-language variable, and you call an API to bind those :names
 to variables in a type-safe way. the downside is that it's verbose.  
 phase b is what Nemerle does with the above - it has an AST macro to
 wrap the above so you can write your query directly and it is checked
 as compile-time.
 No operators were abused in implementing this. 
I quite agree. What we can do already is: auto statement = db.execute!(`select $a from table where $b > 100 && $c Like "A*"`)(visitcars.name,visitcars.id, visitcars.surname); which I personally like much better than the proposed goal:
 It would be so much better to write:
 auto statement = Select(visitcars.name).Where((visitcards.id `>` 100)
 `AND` (visitcards.surname `Like` "A*"));
(Replace $a with your preferred method for defining placeholder variables). And the question then is, can we improve the existing solution? And if so, how? I just don't think the solution involves overloading operators. I think this a great example of why we *don't* want arbitrary operator overloading: there's no point overloading && and > if you can't make 'from', 'where', and 'like' to all be infix operators, as well!
This sounds like a job for better mixin syntax. Then you can do $visitcars.id > 100 && $visitcars.surname Like "A*"`);
Nov 23 2009
parent reply Don <nospam nospam.com> writes:
Chad J wrote:
 Don wrote:
 yigal chripun wrote:
 aarti_pl Wrote:

 Walter Bright pisze:
 Don wrote:
 There's not many sensible operators anyway. opPow is the only
 missing one that's present in many other general-purpose languages.
 The only other ones I think are remotely interesting are dot and
 cross product.
Yup.
 Anything beyond that, you generally want a full DSL, probably with
 different precendence and associativity rules. Eg, for regexp,
 you'd want postfix * and + operators. There are examples of clever
 things done in C++  with operator overloading, but I think that's
 just because it's the only way to do DSLs in C++.
I was enthralled with the way C++ did it for regex for a while, but when I think more about it, it's just too clever. I think it's more operator overloading abuse now.
 I don't think the applications are there.
I agree.
Well, I can understand your fear about operator abuse. And I agree that code might be awful when operator overloading will be abused. But I have in mind one very convincing example. I defined in D/Java SQL syntax. They are also other frameworks which do the same. What can I say about my experiences with using such framework: it is very, very powerful concept. It cuts time necessary to develop application, makes sql statements type safe and allows to pass around parts of sql statements inside application. It also makes easy refactoring of sql statement (especially in Java). Its huge win comparing it to defining DSL as strings. It's hard to explain just in few sentences all details. I have already done it long time ago, and in my first post I provided links. Problem with current approach is that I have to define SQL in D/Java in following way: auto statement = Select(visitcars.name).Where(And(More(visitcards.id, 100), Like(visitcards.surname, "A*"))); Please look at code in Where(). It's so awfuuuuulllllll! It would be so much better to write: auto statement = Select(visitcars.name).Where((visitcards.id `>` 100) `AND` (visitcards.surname `Like` "A*")); I used here syntax which you have proposed with delimiter ``. I think it is good enough solution for such purpose. But please, don't underestimate problem! Many DSL languages would never appear if languages would be good enough. As I said solution with delimiter is good enough for me. It has another advantage that it clearly shows in code that you have overloaded operator here, so no surprises here. Additionally when you implement template function: opInfix('AND')(val0, val1); you pass string into template. So I see it quite intuitive that you use string as operator: ``. Maybe there will be not necessary to change current behavior that `` defines string. I think we have good possibility to open this door now. It can be even implemented later, but I would wish just not to close this door now :-) BR Marcin Kuszczak (aarti_pl)
There's nothing more hideous than all those frameworks in Java/C++ that try to re-enginer SQL into functions, templates, LINQ, whatever. SQL *is* a perfectly designed language for its purpose and it doesn't need to be redisnged! The only problem with this is the type-safety when embedding sql as string in a host language. the solution is two-phased: query is one string like: "select * from table where :a > 42", the :name is a place holder for the host-language variable, and you call an API to bind those :names to variables in a type-safe way. the downside is that it's verbose. phase b is what Nemerle does with the above - it has an AST macro to wrap the above so you can write your query directly and it is checked as compile-time. No operators were abused in implementing this.
I quite agree. What we can do already is: auto statement = db.execute!(`select $a from table where $b > 100 && $c Like "A*"`)(visitcars.name,visitcars.id, visitcars.surname); which I personally like much better than the proposed goal:
 It would be so much better to write:
 auto statement = Select(visitcars.name).Where((visitcards.id `>` 100)
 `AND` (visitcards.surname `Like` "A*"));
(Replace $a with your preferred method for defining placeholder variables). And the question then is, can we improve the existing solution? And if so, how? I just don't think the solution involves overloading operators. I think this a great example of why we *don't* want arbitrary operator overloading: there's no point overloading && and > if you can't make 'from', 'where', and 'like' to all be infix operators, as well!
This sounds like a job for better mixin syntax. Then you can do $visitcars.id > 100 && $visitcars.surname Like "A*"`);
Yeah, something like that. Or it could mixin automatically. eg if macro foo(args...) foo(args) meant mixin(foo(args)). then the syntax would be: db.execute(`select $visitcars.name from table where $visitcars.id > 100 && $visitcars.surname Like "A*"`); which has advantages and disadvantages. So there's quite a bit of flexibility. A lot of potential for brainstorming!
Nov 23 2009
next sibling parent yigal chripun <yigal100 gmail.com> writes:
Don Wrote:

 Chad J wrote:
 Don wrote:
 I quite agree. What we can do already is:

 auto statement = db.execute!(`select $a from table where $b > 100 && $c
 Like "A*"`)(visitcars.name,visitcars.id, visitcars.surname);

 which I personally like much better than the proposed goal:

 It would be so much better to write:
 auto statement = Select(visitcars.name).Where((visitcards.id `>` 100)
 `AND` (visitcards.surname `Like` "A*"));
(Replace $a with your preferred method for defining placeholder variables). And the question then is, can we improve the existing solution? And if so, how? I just don't think the solution involves overloading operators. I think this a great example of why we *don't* want arbitrary operator overloading: there's no point overloading && and > if you can't make 'from', 'where', and 'like' to all be infix operators, as well!
This sounds like a job for better mixin syntax. Then you can do $visitcars.id > 100 && $visitcars.surname Like "A*"`);
Yeah, something like that. Or it could mixin automatically. eg if macro foo(args...) foo(args) meant mixin(foo(args)). then the syntax would be: db.execute(`select $visitcars.name from table where $visitcars.id > 100 && $visitcars.surname Like "A*"`); which has advantages and disadvantages. So there's quite a bit of flexibility. A lot of potential for brainstorming!
a few points I want to add: 1) I though that :name was in some version of the SQL standard or a know extension so if we use this in APIs for D we should use the standard notation (can anyone verify this?) 2) I don't want to mix this discussion with infix functions and operator overloading. I'm not sure I want to limit these and perhaps there are other legitimate uses for general purpose infix functions. In this post I just pointed out that SQL is *not* a legitimate use case for that. 3) the Nemerle macros for SQL allow for: db.execute(`select $visitcars.name from table where $visitcars.id > 100 && $visitcars.surname Like "A*"`); the $ in Nemerle is used for controled breaking of hygene. Their Macro translates such a query into a sql string with :names and calls for the bind API to connect those with the given variables in a type-safe way. IIRC, they use the db connection object to conncet to the DB at compile-time and check the syntax and also existence of the objects (tables, columns, etc) in the DB schema.
Nov 23 2009
prev sibling parent reply Bill Baxter <wbaxter gmail.com> writes:
On Mon, Nov 23, 2009 at 3:12 AM, Don <nospam nospam.com> wrote:

 This sounds like a job for better mixin syntax.
.


 Then you can do


 $visitcars.id > 100 && $visitcars.surname Like "A*"`);
Yeah, something like that. Or it could mixin automatically. eg if macro foo(args...) foo(args) meant =A0mixin(foo(args)).
This is what I've been thinking too. But we might want there to be more to a macro than that. For instance most macro parameters are strings and the '(string this, string that, string theOther)', list of parameters doesn't look so great. And requiring the strings to be quoted on the calling side, also makes the call side rather ugly. It may be better to have arguments automatically convert to some sort of AST type which supports some introspection, and can be converted back to a string of code easily. So I think jumping on just making macro mean "mixin automatically" right now may limit our future choices too much. --bb
Nov 23 2009
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 23 Nov 2009 07:38:32 -0500, Bill Baxter <wbaxter gmail.com> wrote:

 On Mon, Nov 23, 2009 at 3:12 AM, Don <nospam nospam.com> wrote:

 This sounds like a job for better mixin syntax.
 .


 Then you can do


 $visitcars.id > 100 && $visitcars.surname Like "A*"`);
Yeah, something like that. Or it could mixin automatically. eg if macro foo(args...) foo(args) meant mixin(foo(args)).
This is what I've been thinking too. But we might want there to be more to a macro than that. For instance most macro parameters are strings and the '(string this, string that, string theOther)', list of parameters doesn't look so great. And requiring the strings to be quoted on the calling side, also makes the call side rather ugly. It may be better to have arguments automatically convert to some sort of AST type which supports some introspection, and can be converted back to a string of code easily. So I think jumping on just making macro mean "mixin automatically" right now may limit our future choices too much.
What about this: http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D.learn&article_id=17853 -Steve
Nov 23 2009
parent reply Bill Baxter <wbaxter gmail.com> writes:
On Mon, Nov 23, 2009 at 5:32 AM, Steven Schveighoffer
<schveiguy yahoo.com> wrote:
 On Mon, 23 Nov 2009 07:38:32 -0500, Bill Baxter <wbaxter gmail.com> wrote=
:
 On Mon, Nov 23, 2009 at 3:12 AM, Don <nospam nospam.com> wrote:

 This sounds like a job for better mixin syntax.
 .


 Then you can do


e
 $visitcars.id > 100 && $visitcars.surname Like "A*"`);
Yeah, something like that. Or it could mixin automatically. eg if macro foo(args...) foo(args) meant =A0mixin(foo(args)).
This is what I've been thinking too. But we might want there to be more to a macro than that. =A0For instance most macro parameters are strings and the '(string this, string that, string theOther)', list of parameters doesn't look so great. =A0And requiring the strings to be quoted on the calling side, also makes the call side rather ugly. =A0It may be better to have arguments automatically convert to some sort of AST type which supports some introspection, and can be converted back to a string of code easily. So I think jumping on just making macro mean "mixin automatically" right now may limit our future choices too much.
What about this: http://www.digitalmars.com/webnews/newsgroups.php?art_group=3Ddigitalmars=
.D.learn&article_id=3D17853 Quoting from there: """ macro doit(x, y, z) mixin("x" ~ "y" ~ "z"); // allow easy syntax for quoting parameters, since mixins are all about stringification. doit(a, b, c) =3D> mixin("abc"); """ What happens if you want to have the string "x", just a plain x, in the mix= in? Other than that, I think I don't know enough about the lessons to be learned from Lisp, Scheme, and Nemerle. I'd like to study more how macros work in those systems before I'd feel comfortable committing to any particular design. For instance what's the best way to handle hygene issues mentioned here: http://en.wikipedia.org/wiki/Hygienic_macro . There are probably some hygene issues there that should be considered for template mixins too. For instance it's a long-standing issue that if you use a mixin from a library, you must import all the modules that mixin depends on, which breaks a kind of encapsulation of the mixin. Mixins are probably also subject to the kind of environmental hygene problem described on that page, where a common function redefined in the current scope alters the intended behavior of the mixin. I think Walter had intended template mixins to take the place of macros. They offer some features of macros but not all. So once real macros exist, I wonder if there will be any real reason for them to continue existing. --bb
Nov 23 2009
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 23 Nov 2009 12:49:01 -0500, Bill Baxter <wbaxter gmail.com> wrote:

 On Mon, Nov 23, 2009 at 5:32 AM, Steven Schveighoffer
 <schveiguy yahoo.com> wrote:
 On Mon, 23 Nov 2009 07:38:32 -0500, Bill Baxter <wbaxter gmail.com>  
 wrote:

 On Mon, Nov 23, 2009 at 3:12 AM, Don <nospam nospam.com> wrote:

 This sounds like a job for better mixin syntax.
 .


 Then you can do


 $visitcars.id > 100 && $visitcars.surname Like "A*"`);
Yeah, something like that. Or it could mixin automatically. eg if macro foo(args...) foo(args) meant mixin(foo(args)).
This is what I've been thinking too. But we might want there to be more to a macro than that. For instance most macro parameters are strings and the '(string this, string that, string theOther)', list of parameters doesn't look so great. And requiring the strings to be quoted on the calling side, also makes the call side rather ugly. It may be better to have arguments automatically convert to some sort of AST type which supports some introspection, and can be converted back to a string of code easily. So I think jumping on just making macro mean "mixin automatically" right now may limit our future choices too much.
What about this: http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D.learn&article_id=17853
Quoting from there: """ macro doit(x, y, z) mixin("x" ~ "y" ~ "z"); // allow easy syntax for quoting parameters, since mixins are all about stringification. doit(a, b, c) => mixin("abc"); """ What happens if you want to have the string "x", just a plain x, in the mixin?
Don't name your parameter x. The author of the macro is charge of the names of its parameters, if you need a certain string, don't use it as a parameter name. I would say that I wouldn't expect a global search and replace, only matching symbols would be replaced, for example: macro doit(x) mixin("xx = x;"); doit(hi); would result in xx = hi;
 Other than that, I think I don't know enough about the lessons to be
 learned from Lisp, Scheme, and Nemerle.  I'd like to study more how
 macros work in those systems before I'd feel comfortable committing to
 any particular design.  For instance what's the best way to handle
 hygene issues mentioned here:
 http://en.wikipedia.org/wiki/Hygienic_macro .

 There are probably some hygene issues there that should be considered
 for template mixins too.  For instance it's a long-standing issue that
 if you use a mixin from a library, you must import all the modules
 that mixin depends on, which breaks a kind of encapsulation of the
 mixin.  Mixins are probably also subject to the kind of environmental
 hygene problem described on that page, where a common function
 redefined in the current scope alters the intended behavior of the
 mixin.
Yes, my scheme would suffer from those issues also. For those who are interested in a problem case, here is the first one identified on that page written in the proposed form: macro INCI(i) mixin("{int a = 0; i++;}"); void main() { int a = 0; int b = 0; INCI(a); // translates to {int a = 0; a++;} INCI(b); // translates to {int a = 0; b++;} } Now a won't be incremented as expected because the local a declared by the macro shadows the behavior. I'll put it this way -- any problem that mixins suffer from, macros would suffer from as well. If we can fix those problems in mixins, then macros also become fixed. I don't see it worthwhile to propose a fixed macro system when mixins still suffer from the same issue.
 I think Walter had intended template mixins to take the place of
 macros.  They offer some features of macros but not all.  So once real
 macros exist, I wonder if there will be any real reason for them to
 continue existing.
The thing I think macros give you over mixins is their usage is simple and looks like part of the API. Other than that, I don't think mixins are any less powerful. For example, instead of writing: log.logError("bad error occurred with object: " ~ expensiveObjectStringification(obj)); I can do the correct thing via a mixin with: mixin(doLog("log", "error", "\"bad error occurred with object: \" ~ expensiveObjectStringification(obj))")); Where doLog is a CTFE function that rewrites the code as the macro does. But the first looks *sooo* much better :) -Steve
Nov 23 2009
parent reply Bill Baxter <wbaxter gmail.com> writes:
On Mon, Nov 23, 2009 at 12:11 PM, Steven Schveighoffer
<schveiguy yahoo.com> wrote:
 On Mon, 23 Nov 2009 12:49:01 -0500, Bill Baxter <wbaxter gmail.com> wrote=
:
 On Mon, Nov 23, 2009 at 5:32 AM, Steven Schveighoffer
 <schveiguy yahoo.com> wrote:
 On Mon, 23 Nov 2009 07:38:32 -0500, Bill Baxter <wbaxter gmail.com>
 wrote:

 On Mon, Nov 23, 2009 at 3:12 AM, Don <nospam nospam.com> wrote:

 This sounds like a job for better mixin syntax.
 .


 Then you can do


ere
 $visitcars.id > 100 && $visitcars.surname Like "A*"`);
Yeah, something like that. Or it could mixin automatically. eg if macro foo(args...) foo(args) meant =A0mixin(foo(args)).
This is what I've been thinking too. But we might want there to be more to a macro than that. =A0For instan=
ce
 most macro parameters are strings and the '(string this, string that,
 string theOther)', list of parameters doesn't look so great. =A0And
 requiring the strings to be quoted on the calling side, also makes the
 call side rather ugly. =A0It may be better to have arguments
 automatically convert to some sort of AST type which supports some
 introspection, and can be converted back to a string of code easily.

 So I think jumping on just making macro mean "mixin automatically"
 right now may limit our future choices too much.
What about this: http://www.digitalmars.com/webnews/newsgroups.php?art_group=3Ddigitalma=
rs.D.learn&article_id=3D17853
 Quoting from there:
 """
 macro doit(x, y, z) mixin("x" ~ "y" ~ "z"); // allow easy syntax for
 quoting parameters, since mixins are all about stringification.

 doit(a, b, c) =3D> mixin("abc");
 """

 What happens if you want to have the string "x", just a plain x, in the
 mixin?
Don't name your parameter x. =A0The author of the macro is charge of the =
names
 of its parameters, if you need a certain string, don't use it as a parame=
ter
 name. =A0I would say that I wouldn't expect a global search and replace, =
only
 matching symbols would be replaced, for example:

 macro doit(x) mixin("xx =3D x;");

 doit(hi);

 would result in xx =3D hi;

 Other than that, I think I don't know enough about the lessons to be
 learned from Lisp, Scheme, and Nemerle. =A0I'd like to study more how
 macros work in those systems before I'd feel comfortable committing to
 any particular design. =A0For instance what's the best way to handle
 hygene issues mentioned here:
 http://en.wikipedia.org/wiki/Hygienic_macro .

 There are probably some hygene issues there that should be considered
 for template mixins too. =A0For instance it's a long-standing issue that
 if you use a mixin from a library, you must import all the modules
 that mixin depends on, which breaks a kind of encapsulation of the
 mixin. =A0Mixins are probably also subject to the kind of environmental
 hygene problem described on that page, where a common function
 redefined in the current scope alters the intended behavior of the
 mixin.
Yes, my scheme would suffer from those issues also. =A0For those who are interested in a problem case, here is the first one identified on that pa=
ge
 written in the proposed form:

 macro INCI(i) mixin("{int a =3D 0; i++;}");

 void main()
 {
 =A0int a =3D 0;
 =A0int b =3D 0;
 =A0INCI(a); // translates to {int a =3D 0; a++;}
 =A0INCI(b); // translates to {int a =3D 0; b++;}
 }

 Now a won't be incremented as expected because the local a declared by th=
e
 macro shadows the behavior.

 I'll put it this way -- any problem that mixins suffer from, macros would
 suffer from as well. =A0If we can fix those problems in mixins, then macr=
os
 also become fixed. =A0I don't see it worthwhile to propose a fixed macro
 system when mixins still suffer from the same issue.

 I think Walter had intended template mixins to take the place of
 macros. =A0They offer some features of macros but not all. =A0So once re=
al
 macros exist, I wonder if there will be any real reason for them to
 continue existing.
The thing I think macros give you over mixins is their usage is simple an=
d
 looks like part of the API. =A0Other than that, I don't think mixins are =
any
 less powerful.
Note that I was talking about template mixins, not string mixins. But really if we have macros, both template mixins and string mixins may be redundant. --bb
Nov 23 2009
parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 23 Nov 2009 15:22:35 -0500, Bill Baxter <wbaxter gmail.com> wrote:

 On Mon, Nov 23, 2009 at 12:11 PM, Steven Schveighoffer
 <schveiguy yahoo.com> wrote:
 On Mon, 23 Nov 2009 12:49:01 -0500, Bill Baxter <wbaxter gmail.com>  
 wrote:
 I think Walter had intended template mixins to take the place of
 macros.  They offer some features of macros but not all.  So once real
 macros exist, I wonder if there will be any real reason for them to
 continue existing.
The thing I think macros give you over mixins is their usage is simple and looks like part of the API. Other than that, I don't think mixins are any less powerful.
Note that I was talking about template mixins, not string mixins.
Yes, template mixins seem to be less useful because you cannot simply write statements inside a template block.
 But really if we have macros, both template mixins and string mixins
 may be redundant.
The nice thing about having macros be a simpler syntax to do mixins is that mixins are a proven entity that work to do just about anything. Some of the wizardry I've seen is amazing! I'm unsure that a) macros could make mixins obsolete and b) macros should be significantly different than mixins. -Steve
Nov 23 2009
prev sibling parent reply "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
Bill Baxter wrote:
 On Mon, Nov 23, 2009 at 3:12 AM, Don <nospam nospam.com> wrote:
 
 This sounds like a job for better mixin syntax.
 .


 Then you can do


 $visitcars.id > 100 && $visitcars.surname Like "A*"`);
Yeah, something like that. Or it could mixin automatically. eg if macro foo(args...) foo(args) meant mixin(foo(args)).
This is what I've been thinking too. But we might want there to be more to a macro than that. For instance most macro parameters are strings and the '(string this, string that, string theOther)', list of parameters doesn't look so great. And requiring the strings to be quoted on the calling side, also makes the call side rather ugly. It may be better to have arguments automatically convert to some sort of AST type which supports some introspection, and can be converted back to a string of code easily. So I think jumping on just making macro mean "mixin automatically" right now may limit our future choices too much.
I think the community has come to expect a lot more from the macro keyword. I, at least, would be disappointed if this is what comes out of it. :) How about auto template MixMeInAutomatically(T) { ... } void main() { MixMeInAutomatically!int; ... } Would this be ambiguous? It would be using auto for something completely new, but it would at least make sense. (More so than macro, in my opinion.) -Lars
Nov 23 2009
next sibling parent Don <nospam nospam.com> writes:
Lars T. Kyllingstad wrote:
 Bill Baxter wrote:
 On Mon, Nov 23, 2009 at 3:12 AM, Don <nospam nospam.com> wrote:

 This sounds like a job for better mixin syntax.
 .


 Then you can do


 $visitcars.id > 100 && $visitcars.surname Like "A*"`);
Yeah, something like that. Or it could mixin automatically. eg if macro foo(args...) foo(args) meant mixin(foo(args)).
This is what I've been thinking too. But we might want there to be more to a macro than that. For instance most macro parameters are strings and the '(string this, string that, string theOther)', list of parameters doesn't look so great. And requiring the strings to be quoted on the calling side, also makes the call side rather ugly. It may be better to have arguments automatically convert to some sort of AST type which supports some introspection, and can be converted back to a string of code easily. So I think jumping on just making macro mean "mixin automatically" right now may limit our future choices too much.
I think the community has come to expect a lot more from the macro keyword. I, at least, would be disappointed if this is what comes out of it. :)
Oh, me too. But, this establishes a minimum. We've got a very long time to get macros right.
 
 How about
 
     auto template MixMeInAutomatically(T) { ... }
 
     void main()
     {
         MixMeInAutomatically!int;
         ...
     }
 
 Would this be ambiguous? It would be using auto for something completely 
 new, but it would at least make sense. (More so than macro, in my opinion.)
 
 -Lars
Nov 23 2009
prev sibling parent "Simen kjaeraas" <simen.kjaras gmail.com> writes:
Lars T. Kyllingstad <public kyllingen.nospamnet> wrote:

 How about

      auto template MixMeInAutomatically(T) { ... }

      void main()
      {
          MixMeInAutomatically!int;
          ...
      }

 Would this be ambiguous? It would be using auto for something completely  
 new, but it would at least make sense. (More so than macro, in my  
 opinion.)

 -Lars
I believe I have suggested this syntax earlier: mixin template foo( ) { } struct S { foo!(); } I'm unsure if there would be ambiguities, though. -- Simen
Nov 23 2009
prev sibling next sibling parent reply Lutger <lutger.blijdestijn gmail.com> writes:
yigal chripun wrote:

 aarti_pl Wrote:
...
 
 There's nothing more hideous than all those frameworks in Java/C++ that
 try to re-enginer SQL into functions, templates, LINQ, whatever. SQL *is*
 a perfectly designed language for its purpose and it doesn't need to be
 redisnged! The only problem with this is the type-safety when embedding
 sql as string in a host language. the solution is two-phased:
I disagree. There are a couple of problems with using Sql strings inside regular code: - no help from the IDE whatsoever. Imagine writing your D program in notepad, then instead of compiling it in advance, you compile it at runtime: losing all benefit from the static type checking and whatever the IDE offers you (highlighting and autocomplete). This would be completely absurd yet you prefer it with sql? Of course you can write queries in a specialized environment, somewhat better but it also sucks. - mapping relational data to objects is a sucky, boring, lengthy and error prone task. In my experience a simple ORM tool such as Linq-to-Sql makes life so much easier. RoR has this too, it's a blessing. Now if you want to do a big part of your logic on the database instead of simply using it as storage, that may be a different story. (I still think it sucks though)

 one string like: "select * from table where :a > 42", the :name is a place
 holder for the host-language variable, and you call an API to bind those
 :names to variables in a type-safe way. the downside is that it's verbose.
  
 phase b is what Nemerle does with the above - it has an AST macro to wrap
 the above so you can write your query directly and it is checked as
 compile-time.
 
 No operators were abused in implementing this.
I fail to see how this is any different than Linq. Granted, you just reproduce the SQL language and ignore any mapping to Objects, but other than that it is just the same but with a different technology. (Linq is not only about abstracting sql by the way, it does a lot more)
Nov 23 2009
parent reply aarti_pl <aarti interia.pl> writes:
Lutger pisze:
 yigal chripun wrote:
 
 aarti_pl Wrote:
....
 There's nothing more hideous than all those frameworks in Java/C++ that
 try to re-enginer SQL into functions, templates, LINQ, whatever. SQL *is*
 a perfectly designed language for its purpose and it doesn't need to be
 redisnged! The only problem with this is the type-safety when embedding
 sql as string in a host language. the solution is two-phased:
I disagree. There are a couple of problems with using Sql strings inside regular code: - no help from the IDE whatsoever. Imagine writing your D program in notepad, then instead of compiling it in advance, you compile it at runtime: losing all benefit from the static type checking and whatever the IDE offers you (highlighting and autocomplete). This would be completely absurd yet you prefer it with sql? Of course you can write queries in a specialized environment, somewhat better but it also sucks.
Thanks for this post. It is basically answer to other (rather dismissive) posts. In addition to what you said, you get from IDE also possibility to refactor your database columns/names/types etc. All with static checks in your code. After change you know immediately what should be changed. (*it is possible at least in my framework, I don't know if it is possible in other frameworks).
 - mapping relational data to objects is a sucky, boring, lengthy and error 
 prone task.
 In my experience a simple ORM tool such as Linq-to-Sql makes life so much 
 easier. RoR has this too, it's a blessing.
 
 Now if you want to do a big part of your logic on the database instead of 
 simply using it as storage, that may be a different story. (I still think it 
 sucks though)
Well, I have different view on this. Personally I think that it is much better (faster, easier) to pass tables in the application. But after defining SQL query (in object oriented way) you can do whatever you want: you can change your SQL object into table of data or into objects. You can also use these object oriented queries to generate different SQL dialects for different databases or you can generate hashes for using them as keys in your cache. Additionally in my framework you can pass around parts of SQL e.g. : WhereExpression exp = Where(More(visitcards.id, 100)); You basically *CAN NOT* do it when just using strings. So I think such a way of defining DSLs in mother language have its merits. String is not enough.

 one string like: "select * from table where :a > 42", the :name is a place
 holder for the host-language variable, and you call an API to bind those
 :names to variables in a type-safe way. the downside is that it's verbose.
  
 phase b is what Nemerle does with the above - it has an AST macro to wrap
 the above so you can write your query directly and it is checked as
 compile-time.

 No operators were abused in implementing this.
I fail to see how this is any different than Linq. Granted, you just reproduce the SQL language and ignore any mapping to Objects, but other than that it is just the same but with a different technology. (Linq is not only about abstracting sql by the way, it does a lot more)
I would like to add that probably many of DSL languages would never be created if mother languages would be expressive enough. And additionally, can any of opponents explain what is so wrong with operator overloading suggested by me in last emails? (Pseudocode - I didn't check exact syntax): RES opInfix(alias str)(T val) { ... } ... auto expr = Where((visitcards.id `>` 100) `AND` (visitcards.surname `Like` "A*")); (Please note that when overloaded operator is used it is denoted with `` (it can be probably any definition of string). I think it would be good enough solution and thanks to this operator overloading will be clearly visible in calling code). And dear opponents please propose how to solve above mentioned problems with DSL languages as strings ;-) If there will be a way to solve all those problems cleanly - I am all for not using operator overloading :-) Best Regards Marcin Kuszczak
Nov 23 2009
parent reply Don <nospam nospam.com> writes:
aarti_pl wrote:
 Lutger pisze:
 yigal chripun wrote:

 aarti_pl Wrote:
....
 There's nothing more hideous than all those frameworks in Java/C++ that
 try to re-enginer SQL into functions, templates, LINQ, whatever. SQL 
 *is*
 a perfectly designed language for its purpose and it doesn't need to be
 redisnged! The only problem with this is the type-safety when embedding
 sql as string in a host language. the solution is two-phased:
I disagree. There are a couple of problems with using Sql strings inside regular code: - no help from the IDE whatsoever. Imagine writing your D program in notepad, then instead of compiling it in advance, you compile it at runtime: losing all benefit from the static type checking and whatever the IDE offers you (highlighting and autocomplete). This would be completely absurd yet you prefer it with sql? Of course you can write queries in a specialized environment, somewhat better but it also sucks.
Thanks for this post. It is basically answer to other (rather dismissive) posts. In addition to what you said, you get from IDE also possibility to refactor your database columns/names/types etc. All with static checks in your code. After change you know immediately what should be changed. (*it is possible at least in my framework, I don't know if it is possible in other frameworks).
Yes, I think IDE integration is one of the biggest issues facing macros.
 Additionally in my framework you can pass around parts of SQL e.g. :
 WhereExpression exp = Where(More(visitcards.id, 100));
 
 You basically *CAN NOT* do it when just using strings. 
Of course you can define a where clause using strings. (I'm not sure when you'd want to, though. I suspect that the fact that you can do it in your framework, is an implementation detail rather than a design goal. It's easy to think of uses for run-time generated naked WHERE clauses, but much harder for compile-time ones).

 query is
 one string like: "select * from table where :a > 42", the :name is a 
 place
 holder for the host-language variable, and you call an API to bind those
 :names to variables in a type-safe way. the downside is that it's 
 verbose.
  
 phase b is what Nemerle does with the above - it has an AST macro to 
 wrap
 the above so you can write your query directly and it is checked as
 compile-time.

 No operators were abused in implementing this.
I fail to see how this is any different than Linq. Granted, you just reproduce the SQL language and ignore any mapping to Objects, but other than that it is just the same but with a different technology.
That *is* the difference! It's exactly what you'd do in a run-time API, but it is checked at compile time. IMHO, the ideal for metaprogramming is that you can hardly tell that you are using it. (BTW, that's why I'm a huge fan of CTFE).
 (Linq is not only about abstracting sql by the way, it does a lot more)
I would like to add that probably many of DSL languages would never be created if mother languages would be expressive enough.
Yes, I've worked with some DSLs for which that was definitely true.
 And additionally, can any of opponents explain what is so wrong with 
 operator overloading suggested by me in last emails? (Pseudocode - I 
 didn't check exact syntax):
 
 RES opInfix(alias str)(T val) {
 ...
 }
 ...
 auto expr = Where((visitcards.id `>` 100) `AND` (visitcards.surname 
 `Like` "A*"));
 
 (Please note that when overloaded operator is used it is denoted with `` 
 (it can be probably any definition of string). I think it would be good 
 enough solution and thanks to this operator overloading will be clearly 
 visible in calling code).
`AND` is not operator overloading in the traditional sense. This is something else. You're proposing a whole new infix notation, and it's difficult to evaluate it without knowing how it would work. I don't understand why `Like` is an operator, but `Where` is not. You haven't addressed operator precedence. And I still don't think it looks terribly good for SQL. With: id `>` 100 users would always be thinking, "why can't I just write id > 100 ? That ` is so annoying!"
Nov 24 2009
parent reply aarti_pl <aarti interia.pl> writes:
Don pisze:
 
 Additionally in my framework you can pass around parts of SQL e.g. :
 WhereExpression exp = Where(More(visitcards.id, 100));

 You basically *CAN NOT* do it when just using strings. 
Of course you can define a where clause using strings. (I'm not sure when you'd want to, though. I suspect that the fact that you can do it in your framework, is an implementation detail rather than a design goal. It's easy to think of uses for run-time generated naked WHERE clauses, but much harder for compile-time ones).
Not exactly. At least it was not the case last time we talked about it: *I will give here only NG post numbers* 79272 Domain Specific Languages in D; was: C++, D: Dinosaurs? 81026 Operator overloading 81078 81393 And you finally agreed with me that it is not possible. So I take it proven :-) 81424 Possibility to split whole SQL statement into parts is useful. You can mix & match these parts together as you want. It is similar as you want to split you big chunk of code into smaller parts. You can for example make separate expressions objects and then put them one by one into Where().
 `AND` is not operator overloading in the traditional sense. This is 
 something else. You're proposing a whole new infix notation, and it's 
 difficult to evaluate it without knowing how it would work.
Well, with Andrei's proposition it is obvious for me that it should be generalized. I stated it already in: 81392 (almost one year ago) I will cite part of this e-mail: *<citation>* Exactly. My first thoughts was to allow defining infix keywords/operators in a similar way as today we can define prefix operators: myFunc a b or just: myFunc(a, b); Why? Because they are use cases where infix notation is much, much more natural than prefix notation. So we still need: 1. a myFunc b - infix operator 2. a b myFunc - postfix operator *</citation>* After reading my old e-mail it seems that it can be further generalized: prefix(tuple_args) (tuple_args)postfix (tuple_args)infix(tuple_args) Ok. Just a thought :-) Some more about more precise definition of this feature (proposals, to think about it): 1. I don't agree that it is something different than traditional operator overloading. It is just generalization. 2. After including Andrei's proposal the biggest part of this feature will be already in language. 3. The only think to define is just how the caller side should like: a. Always demand `` when using operator overloading; (Pros: no abuse of operators - you always know when operator overloading is involved; Cons: doesn't look so good, especially in cases where you would expect plain symbols) b. Don't demand `` in case of some standard operators like <>+-*/ etc. In other cases demand ``(Pros: Looks better; Cons: you will not always know that operator overloading is involved) c. Demand `` always when there is standard operator defined or operator is not symbol, but identifier. d. Don't allow redefinition of same operator Above concepts must be discussed, but it seems that there is not enough courage in NG to change current situation. Well it's strange that the feature is called 'operator overloading' - it's quite obvious that it's about defining some special functions. And IMHO some other languages got it right.
 I don't understand why `Like` is an operator, but `Where` is not. You 
 haven't addressed operator precedence.
'Like' against 'Where': it's definitely implementation detail of my framework. I don't think we should discuss it.
 
 And I still don't think it looks terribly good for SQL. With:
 id `>` 100
 users would always be thinking, "why can't I just write id > 100 ? That 
 ` is so annoying!"
Please believe me: it's 3000 times better than current situation. Additionally using such framework is pure pleasure and you want miss string SQLs even in current, bad situation :-) Best Regards Marcin Kuszczak (aarti_pl)
Nov 24 2009
parent reply Don <nospam nospam.com> writes:
aarti_pl wrote:
 Don pisze:
 Additionally in my framework you can pass around parts of SQL e.g. :
 WhereExpression exp = Where(More(visitcards.id, 100));

 You basically *CAN NOT* do it when just using strings. 
Of course you can define a where clause using strings. (I'm not sure when you'd want to, though. I suspect that the fact that you can do it in your framework, is an implementation detail rather than a design goal. It's easy to think of uses for run-time generated naked WHERE clauses, but much harder for compile-time ones).
Not exactly. At least it was not the case last time we talked about it: *I will give here only NG post numbers* 79272 Domain Specific Languages in D; was: C++, D: Dinosaurs? 81026 Operator overloading 81078 81393 And you finally agreed with me that it is not possible. So I take it proven :-) 81424
I believe I was agreeing that it wasn't possible to do it with a single SQL ctfe function.
 
 Possibility to split whole SQL statement into parts is useful. You can 
 mix & match these parts together as you want. It is similar as you want 
 to split you big chunk of code into smaller parts. You can for example 
 make separate expressions objects and then put them one by one into 
 Where().
 
 `AND` is not operator overloading in the traditional sense. This is 
 something else. You're proposing a whole new infix notation, and it's 
 difficult to evaluate it without knowing how it would work.
Well, with Andrei's proposition it is obvious for me that it should be generalized. I stated it already in: 81392 (almost one year ago) I will cite part of this e-mail: *<citation>* Exactly. My first thoughts was to allow defining infix keywords/operators in a similar way as today we can define prefix operators: myFunc a b or just: myFunc(a, b); Why? Because they are use cases where infix notation is much, much more natural than prefix notation. So we still need: 1. a myFunc b - infix operator 2. a b myFunc - postfix operator *</citation>* After reading my old e-mail it seems that it can be further generalized: prefix(tuple_args) (tuple_args)postfix (tuple_args)infix(tuple_args) Ok. Just a thought :-)
To quote something I said in that last post: "My point is simply: creating syntax trees is what you're using operator overloading for. You're actually not creating operations."
 Some more about more precise definition of this feature (proposals, to 
 think about it):
 1. I don't agree that it is something different than traditional 
 operator overloading. It is just generalization.
 2. After including Andrei's proposal the biggest part of this feature 
 will be already in language.
 3. The only think to define is just how the caller side should like:
 a. Always demand `` when using operator overloading; (Pros: no abuse of 
 operators - you always know when operator overloading is involved; Cons: 
 doesn't look so good, especially in cases where you would expect plain 
 symbols)
 b. Don't demand `` in case of some standard operators like <>+-*/ etc. 
 In other cases demand ``(Pros: Looks better; Cons: you will not always 
 know that operator overloading is involved)
 c. Demand `` always when there is standard operator defined or operator 
 is not symbol, but identifier.
 d. Don't allow redefinition of same operator
 
 Above concepts must be discussed, but it seems that there is not enough 
 courage in NG to change current situation.
Please. It's been very clearly stated, long ago, that : MACROS ARE DEFERRED TO D3.
 Well it's strange that the feature is called 'operator overloading' - 
 it's quite obvious that it's about defining some special functions. And 
 IMHO some other languages got it right.
IMHO, equating 'operator overloading' with special functions is a colossal mistake. I'm convinced that C++ got it badly wrong.
 And I still don't think it looks terribly good for SQL. With:
 id `>` 100
 users would always be thinking, "why can't I just write id > 100 ? 
 That ` is so annoying!"
Please believe me: it's 3000 times better than current situation.
I don't disagree with that. I just think we should aim higher.
Nov 25 2009
parent reply Don <nospam nospam.com> writes:
Don wrote:
 aarti_pl wrote:
 Don pisze:
 Additionally in my framework you can pass around parts of SQL e.g. :
 WhereExpression exp = Where(More(visitcards.id, 100));

 You basically *CAN NOT* do it when just using strings. 
Of course you can define a where clause using strings.
Even today, it's easy to take a CTFE/string mixin front-end and create a wrapper over your framework! So it's trivially easy to prove it can be done. You can literally do anything. Functionality is not the problem -- the challenge is to make it nice. The CTFE/mixin combo is not nice, but it definitely has the power. I'm thinking something like: auto result = SQL{ SELECT * FROM :localvar WHERE name LIKE 'abc*' AND price > 300 ORDER BY date DESC } where SQL is a DSL macro, and the {} are delimiters indicating that it's a DSL, not normal D code. That's a syntax which is currently available.
Nov 25 2009
parent reply "Denis Koroskin" <2korden gmail.com> writes:
On Wed, 25 Nov 2009 11:42:53 +0300, Don <nospam nospam.com> wrote:

 Don wrote:
 aarti_pl wrote:
 Don pisze:
 Additionally in my framework you can pass around parts of SQL e.g. :
 WhereExpression exp = Where(More(visitcards.id, 100));

 You basically *CAN NOT* do it when just using strings.
Of course you can define a where clause using strings.
Even today, it's easy to take a CTFE/string mixin front-end and create a wrapper over your framework! So it's trivially easy to prove it can be done. You can literally do anything. Functionality is not the problem -- the challenge is to make it nice. The CTFE/mixin combo is not nice, but it definitely has the power. I'm thinking something like: auto result = SQL{ SELECT * FROM :localvar WHERE name LIKE 'abc*' AND price > 300 ORDER BY date DESC } where SQL is a DSL macro, and the {} are delimiters indicating that it's a DSL, not normal D code. That's a syntax which is currently available.
Interesting! Then q{ Foo } syntax would be just a simple macro that returns a string, not a built-in language construct.
Nov 25 2009
parent Don <nospam nospam.com> writes:
Denis Koroskin wrote:
 On Wed, 25 Nov 2009 11:42:53 +0300, Don <nospam nospam.com> wrote:
 
 Don wrote:
 aarti_pl wrote:
 Don pisze:
 Additionally in my framework you can pass around parts of SQL e.g. :
 WhereExpression exp = Where(More(visitcards.id, 100));

 You basically *CAN NOT* do it when just using strings.
Of course you can define a where clause using strings.
Even today, it's easy to take a CTFE/string mixin front-end and create a wrapper over your framework! So it's trivially easy to prove it can be done. You can literally do anything. Functionality is not the problem -- the challenge is to make it nice. The CTFE/mixin combo is not nice, but it definitely has the power. I'm thinking something like: auto result = SQL{ SELECT * FROM :localvar WHERE name LIKE 'abc*' AND price > 300 ORDER BY date DESC } where SQL is a DSL macro, and the {} are delimiters indicating that it's a DSL, not normal D code. That's a syntax which is currently available.
Interesting! Then q{ Foo } syntax would be just a simple macro that returns a string, not a built-in language construct.
Yup. I'm just brainstorming, though.
Nov 25 2009
prev sibling parent aarti_pl <aarti interia.pl> writes:
yigal chripun pisze:
 aarti_pl Wrote:
 
 Walter Bright pisze:
 Don wrote:
 There's not many sensible operators anyway. opPow is the only missing 
 one that's present in many other general-purpose languages. The only 
 other ones I think are remotely interesting are dot and cross product.
Yup.
 Anything beyond that, you generally want a full DSL, probably with 
 different precendence and associativity rules. Eg, for regexp, you'd 
 want postfix * and + operators. There are examples of clever things 
 done in C++  with operator overloading, but I think that's just 
 because it's the only way to do DSLs in C++.
I was enthralled with the way C++ did it for regex for a while, but when I think more about it, it's just too clever. I think it's more operator overloading abuse now.
 I don't think the applications are there.
I agree.
Well, I can understand your fear about operator abuse. And I agree that code might be awful when operator overloading will be abused. But I have in mind one very convincing example. I defined in D/Java SQL syntax. They are also other frameworks which do the same. What can I say about my experiences with using such framework: it is very, very powerful concept. It cuts time necessary to develop application, makes sql statements type safe and allows to pass around parts of sql statements inside application. It also makes easy refactoring of sql statement (especially in Java). Its huge win comparing it to defining DSL as strings. It's hard to explain just in few sentences all details. I have already done it long time ago, and in my first post I provided links. Problem with current approach is that I have to define SQL in D/Java in following way: auto statement = Select(visitcars.name).Where(And(More(visitcards.id, 100), Like(visitcards.surname, "A*"))); Please look at code in Where(). It's so awfuuuuulllllll! It would be so much better to write: auto statement = Select(visitcars.name).Where((visitcards.id `>` 100) `AND` (visitcards.surname `Like` "A*")); I used here syntax which you have proposed with delimiter ``. I think it is good enough solution for such purpose. But please, don't underestimate problem! Many DSL languages would never appear if languages would be good enough. As I said solution with delimiter is good enough for me. It has another advantage that it clearly shows in code that you have overloaded operator here, so no surprises here. Additionally when you implement template function: opInfix('AND')(val0, val1); you pass string into template. So I see it quite intuitive that you use string as operator: ``. Maybe there will be not necessary to change current behavior that `` defines string. I think we have good possibility to open this door now. It can be even implemented later, but I would wish just not to close this door now :-) BR Marcin Kuszczak (aarti_pl)
There's nothing more hideous than all those frameworks in Java/C++ that try to re-enginer SQL into functions, templates, LINQ, whatever. SQL *is* a perfectly designed language for its purpose and it doesn't need to be redisnged! The only problem with this is the type-safety when embedding sql as string in a host language. the solution is two-phased: string like:
 "select * from table where :a > 42", the :name is a place holder for the
host-language variable, and you call an API to bind those :names to variables
in a type-safe way. the downside is that it's verbose. 
  
 phase b is what Nemerle does with the above - it has an AST macro to wrap the
above so you can write your query directly and it is checked as compile-time. 
 
 No operators were abused in implementing this. 
Please see for my comments into answer to Lutger post. BR Marcin Kuszczak (aarti_pl)
Nov 23 2009