www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DIP58: ".." as a Binary Operator

reply "Mason McGill" <mmcgill caltech.edu> writes:
I just wrote a DIP aimed at improving slicing and range 
construction syntax while maintaining backwards compatibility, 
and I'd like to hear your opinions!
http://wiki.dlang.org/DIP58

It can be thought of as an elaboration on the approach discussed 
here:
http://forum.dlang.org/thread/mailman.551.1365290408.4724.digitalmars-d-learn puremagic.com

And an alternative to the approach discussed here:
http://forum.dlang.org/thread/upzdamhmxrrlsexgcdva forum.dlang.org#post-ncwqaixkgbgycybvpkgj:40forum.dlang.org

I think the issue of appealing to numerical programmers is worth 
some attention because there's a distinct niche that D is 
frustratingly close to filling.  In my field, researchers will 
often write scripts in a dynamic language, publish, iterate, and 
eventually re-write their software in C++ and release it as a 
library.  The re-writing step is a large time investment, but 
it's important because

   - Dynamic languages are either slow (MATLAB/Python/R) or 
immature (Julia).
   - Other researchers may prefer another dynamic language, but 
every relevant dynamic language can interface with native 
libraries.

D already has the speed and modeling power of C++, GC for clean 
API design, and reflection for automatic bindings, but it's 
missing a few key features required to make something like NumPy 
or the Julia standard library possible in D.  I believe DIP58 
provides those features, and accepting DIP58 will make D a 
competitive alternative to the prototype/test/rewrite/release 
cycle.

On another note, I'm pretty new to D and the community, so let me 
know if there's any protocol I should follow with respect to DIPs 
and I'll get on it!

Cheers,
Mason
Mar 17 2014
next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Mason McGill:


 http://wiki.dlang.org/DIP58


Seems nice. But the syntax a..b..step is not very nice.

Bye,
bearophile
Mar 17 2014
parent reply "Mason McGill" <mmcgill caltech.edu> writes:
On Monday, 17 March 2014 at 17:41:16 UTC, bearophile wrote:

 Mason McGill:


 http://wiki.dlang.org/DIP58


 Seems nice.


Thanks.  There are a few awkward parts to maintain compatibility, 
but that seems to be the only way to go.


 But the syntax a..b..step is not very nice.


Do you not like the order?  Because it was actually a..step..b 
(like MATLAB/Julia, not like Python).  Or do you not like the 
"verbosity" of all those dots (a:step:b would be better)?  Or is 
it the readability issues if floating point literals were mixed 
in there?


 Bye,
 bearophile
Mar 17 2014
next sibling parent reply "Chris Williams" <yoreanon-chrisw yahoo.co.jp> writes:
On Monday, 17 March 2014 at 19:07:45 UTC, Mason McGill wrote:

 On Monday, 17 March 2014 at 17:41:16 UTC, bearophile wrote:

 Mason McGill:


 http://wiki.dlang.org/DIP58


 Seems nice.


 Thanks.  There are a few awkward parts to maintain 
 compatibility, but that seems to be the only way to go.


 But the syntax a..b..step is not very nice.


 Do you not like the order?  Because it was actually a..step..b 
 (like MATLAB/Julia, not like Python).  Or do you not like the 
 "verbosity" of all those dots (a:step:b would be better)?  Or 
 is it the readability issues if floating point literals were 
 mixed in there?


 Bye,
 bearophile




Random thought, but treating step as a template argument would 
allow for some more interesting changes to the iterations, though 
I can't think of any particular syntax that would look good. And 
if you did add such a thing, then other operators would want it 
as well.

int[] foo = a ..!"a += 10" b;
bool equals = dbl1 ==."abs(a - b) < 0.01" dbl2
Mar 17 2014
parent reply "Mason McGill" <mmcgill caltech.edu> writes:
 Random thought, but treating step as a template argument would 
 allow for some more interesting changes to the iterations, 
 though I can't think of any particular syntax that would look 
 good. And if you did add such a thing, then other operators 
 would want it as well.


Interesting, though I feel like operator syntax really shines 
when either
   1) It's significantly shorter/simpler than the equivalent 
function calls.
   2) It appeals to domain-specific intuitions.

In terms of (1) and (2), I think better spellings for these might 
be


 int[] foo = a ..!"a += 10" b;


import std.array: array;
import std.range: iota;

int[] foo = array(iota(a, b, 10));
// Or something with recurrence, in the general case.


 bool equals = dbl1 ==."abs(a - b) < 0.01" dbl2


import std.math: abs;

struct Approximation(A)
{
     A a;
     alias a this;

     bool opEquals(B)(B b) const
       { return abs(a - b) < 0.01; }
}

auto approximate(A)(A a)
   { return Approximation!A(a); }

bool equals = approximate(a) == b;
Mar 17 2014
parent "Chris Williams" <yoreanon-chrisw yahoo.co.jp> writes:
On Monday, 17 March 2014 at 20:13:20 UTC, Mason McGill wrote:

 Interesting, though I feel like operator syntax really shines 
 when either
   1) It's significantly shorter/simpler than the equivalent 
 function calls.
   2) It appeals to domain-specific intuitions.


In general, I agree. Though I think that my examples suffer from 
being poor examples and for allowing the most complete 
overridability (short of using lambdas). The .. operator could be 
written to only accept a constant step, like:

a ..!5 b;

The place where I was thinking that templating normal operators 
might be handy is actually in games, where you might want to (for 
example) treat a 4x4 matrix in the input as a 4x3 matrix during 
multiplication.

a *!M43 b;

Again, by restricting the allowed template parameters, rather 
than exposing code, you do end up with a shorter syntax, than a 
function call.
Mar 17 2014
prev sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Mar 17, 2014 at 07:07:44PM +0000, Mason McGill wrote:

 On Monday, 17 March 2014 at 17:41:16 UTC, bearophile wrote:

Mason McGill:


http://wiki.dlang.org/DIP58


Seems nice.


 
 Thanks.  There are a few awkward parts to maintain compatibility,
 but that seems to be the only way to go.
 

But the syntax a..b..step is not very nice.


 
 Do you not like the order?  Because it was actually a..step..b (like
 MATLAB/Julia, not like Python).  Or do you not like the "verbosity"
 of all those dots (a:step:b would be better)?  Or is it the
 readability issues if floating point literals were mixed in there?


[...]

Not speaking for bearophile here, but I don't like using .. to mean two
different things (endpoints vs. step).


T

-- 
Study gravitation, it's a field with a lot of potential.
Mar 17 2014
parent "Mason McGill" <mmcgill caltech.edu> writes:
On Monday, 17 March 2014 at 19:58:40 UTC, H. S. Teoh wrote:

 On Mon, Mar 17, 2014 at 07:07:44PM +0000, Mason McGill wrote:

 On Monday, 17 March 2014 at 17:41:16 UTC, bearophile wrote:

Mason McGill:


http://wiki.dlang.org/DIP58


Seems nice.


 
 Thanks.  There are a few awkward parts to maintain 
 compatibility,
 but that seems to be the only way to go.
 

But the syntax a..b..step is not very nice.


 
 Do you not like the order?  Because it was actually a..step..b 
 (like
 MATLAB/Julia, not like Python).  Or do you not like the 
 "verbosity"
 of all those dots (a:step:b would be better)?  Or is it the
 readability issues if floating point literals were mixed in 
 there?


 [...]

 Not speaking for bearophile here, but I don't like using .. to 
 mean two
 different things (endpoints vs. step).


That's fair.  The rationale was the precedent set by MATLAB, 
Python, and Julia.  One of the benefits of making (a..b) an 
expression is the ability to define functions that operate on the 
result.

Examples:
---------
stride(a..b, 2);
(a..b).by(2);    // I happen to like this spelling.

(a..b).reverse;

(a..b).square; // To define a 2D grid.

(a..b).window(c); // c-length windows of a..b.
                   // Useful for indexing video frames
                   // in a multi-frame analysis.
Mar 17 2014
prev sibling next sibling parent "Asman01" <jckj33 gmail.com> writes:
On Monday, 17 March 2014 at 07:56:20 UTC, Mason McGill wrote:

 I just wrote a DIP aimed at improving slicing and range 
 construction syntax while maintaining backwards compatibility, 
 and I'd like to hear your opinions!
 http://wiki.dlang.org/DIP58

 It can be thought of as an elaboration on the approach 
 discussed here:
 http://forum.dlang.org/thread/mailman.551.1365290408.4724.digitalmars-d-learn puremagic.com

 And an alternative to the approach discussed here:
 http://forum.dlang.org/thread/upzdamhmxrrlsexgcdva forum.dlang.org#post-ncwqaixkgbgycybvpkgj:40forum.dlang.org

 I think the issue of appealing to numerical programmers is 
 worth some attention because there's a distinct niche that D is 
 frustratingly close to filling.  In my field, researchers will 
 often write scripts in a dynamic language, publish, iterate, 
 and eventually re-write their software in C++ and release it as 
 a library.  The re-writing step is a large time investment, but 
 it's important because

   - Dynamic languages are either slow (MATLAB/Python/R) or 
 immature (Julia).
   - Other researchers may prefer another dynamic language, but 
 every relevant dynamic language can interface with native 
 libraries.

 D already has the speed and modeling power of C++, GC for clean 
 API design, and reflection for automatic bindings, but it's 
 missing a few key features required to make something like 
 NumPy or the Julia standard library possible in D.  I believe 
 DIP58 provides those features, and accepting DIP58 will make D 
 a competitive alternative to the prototype/test/rewrite/release 
 cycle.

 On another note, I'm pretty new to D and the community, so let 
 me know if there's any protocol I should follow with respect to 
 DIPs and I'll get on it!

 Cheers,
 Mason


Looks like Pascal stuff to me...
Mar 17 2014
prev sibling next sibling parent reply "Meta" <jared771 gmail.com> writes:
On Monday, 17 March 2014 at 07:56:20 UTC, Mason McGill wrote:

 I just wrote a DIP aimed at improving slicing and range 
 construction syntax while maintaining backwards compatibility, 
 and I'd like to hear your opinions!
 http://wiki.dlang.org/DIP58

 It can be thought of as an elaboration on the approach 
 discussed here:
 http://forum.dlang.org/thread/mailman.551.1365290408.4724.digitalmars-d-learn puremagic.com

 And an alternative to the approach discussed here:
 http://forum.dlang.org/thread/upzdamhmxrrlsexgcdva forum.dlang.org#post-ncwqaixkgbgycybvpkgj:40forum.dlang.org

 I think the issue of appealing to numerical programmers is 
 worth some attention because there's a distinct niche that D is 
 frustratingly close to filling.  In my field, researchers will 
 often write scripts in a dynamic language, publish, iterate, 
 and eventually re-write their software in C++ and release it as 
 a library.  The re-writing step is a large time investment, but 
 it's important because

   - Dynamic languages are either slow (MATLAB/Python/R) or 
 immature (Julia).
   - Other researchers may prefer another dynamic language, but 
 every relevant dynamic language can interface with native 
 libraries.

 D already has the speed and modeling power of C++, GC for clean 
 API design, and reflection for automatic bindings, but it's 
 missing a few key features required to make something like 
 NumPy or the Julia standard library possible in D.  I believe 
 DIP58 provides those features, and accepting DIP58 will make D 
 a competitive alternative to the prototype/test/rewrite/release 
 cycle.

 On another note, I'm pretty new to D and the community, so let 
 me know if there's any protocol I should follow with respect to 
 DIPs and I'll get on it!

 Cheers,
 Mason


It's a nice proposal, but what happens if you want a range of 
floats, e.g., 1.0..2.0? I don't know if this causes any ambiguity 
or not. Also, as others said, the step syntax is a bit weird, and 
could probably be delegated to a library function quite easily, 
such as in one of your examples:

const evenEntries = vector[(1..11).by(2)];
Mar 17 2014
parent "Mason McGill" <mmcgill caltech.edu> writes:
 It's a nice proposal, but what happens if you want a range of 
 floats, e.g., 1.0..2.0?


Correct me if I'm wrong, parsing would look like:

----------------------------------------
1.0..2.0
===      Found a literal.

1.0..2.0
    ==    Found a binary operator.

1.0..2.0
      === Found a literal.
----------------------------------------

Something like `5....5` could also compile (since there's only 1 
valid parsing), though such a thing is clearly an abomination, so 
I'd write it `5. .. .5` or `(5.)..(.5)`.  The whole ordeal is 
probably a bit of insight into why other languages use the ':' 
character for slicing.
Mar 17 2014
prev sibling next sibling parent reply Robert Schadek <realburner gmx.de> writes:
replace .. with : to make lexing easier, please
Mar 17 2014
parent "deadalnix" <deadalnix gmail.com> writes:
On Monday, 17 March 2014 at 22:21:46 UTC, Robert Schadek wrote:

 replace .. with : to make lexing easier, please


+1
Mar 17 2014
prev sibling next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Mar 17, 2014 at 11:16:12PM +0100, Robert Schadek wrote:

 replace .. with : to make lexing easier, please


	auto b = arr[(someCondition) ? w:x : y:z];

:-(


T

-- 
Computers aren't intelligent; they only think they are.
Mar 17 2014
parent "Mason McGill" <mmcgill caltech.edu> writes:
On Monday, 17 March 2014 at 22:26:05 UTC, H. S. Teoh wrote:

 On Mon, Mar 17, 2014 at 11:16:12PM +0100, Robert Schadek wrote:

 replace .. with : to make lexing easier, please


 	auto b = arr[(someCondition) ? w:x : y:z];

 :-(


 T


Julia has both the ternary conditional and ":" as an operator.  
In practice, they rarely come up in the same expressions, and 
when they do, parentheses are an easy solution.

     auto b = arr[someCondition ? (w:x) : (y:z)];

However, I think it's worth mentioning that--unless I've missed 
something--DIP58 doesn't introduce any parsing issues that aren't 
already present inside a SliceExpression:

     struct S { void opSlice(real a, real b) {} }

     S s;
     s[5. .. .5]; // Compiles.
     s[5... .5]; // Error.
     s[5. ...5]; // Error.
     s[5....5]; // Error.

This seems like reasonable behavior, in my opinion.

It seems like ":" and ".." both have their pros and cons, but 
".." allows backwards compatibility to be preserved, and I don't 
see how ":" can be worked in elegantly without breaking lots of 
code.
Mar 17 2014
prev sibling next sibling parent reply Robert Schadek <realburner gmx.de> writes:
On 03/17/2014 11:24 PM, H. S. Teoh wrote:

 On Mon, Mar 17, 2014 at 11:16:12PM +0100, Robert Schadek wrote:

 replace .. with : to make lexing easier, please


 	auto b = arr[(someCondition) ? w:x : y:z];

 :-(


 T


Thats a parsing problem.

1..1 makes you think you got a prefix of a float 1. but actually you got
an int slice_operator int. If there where no .. flex like generator
could handle D, at least as far as I can see it.
Mar 17 2014
parent Rainer Schuetze <r.sagitario gmx.de> writes:
On 17.03.2014 23:33, Robert Schadek wrote:

 On 03/17/2014 11:24 PM, H. S. Teoh wrote:

 On Mon, Mar 17, 2014 at 11:16:12PM +0100, Robert Schadek wrote:

 replace .. with : to make lexing easier, please


 	auto b = arr[(someCondition) ? w:x : y:z];

 :-(


 T


 Thats a parsing problem.

 1..1 makes you think you got a prefix of a float 1. but actually you got
 an int slice_operator int. If there where no .. flex like generator
 could handle D, at least as far as I can see it.


".." is a tiny problem regarding lexing in comparison to all the string 
types in D. How do you deal with DelimitedString and TokenString 
(http://dlang.org/lex#DelimitedString) when using a lexer generator?
Mar 18 2014
prev sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Mar 17, 2014 at 11:33:38PM +0100, Robert Schadek wrote:

 On 03/17/2014 11:24 PM, H. S. Teoh wrote:

 On Mon, Mar 17, 2014 at 11:16:12PM +0100, Robert Schadek wrote:

 replace .. with : to make lexing easier, please


 	auto b = arr[(someCondition) ? w:x : y:z];

 :-(


 T


 Thats a parsing problem.
 
 1..1 makes you think you got a prefix of a float 1. but actually you
 got an int slice_operator int. If there where no .. flex like
 generator could handle D, at least as far as I can see it.


Point.

But still, syntax that makes lexing or parsing hard -- that scores
against this DIP.


T

-- 
Why can't you just be a nonconformist like everyone else? -- YHL
Mar 17 2014
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 03/17/2014 11:37 PM, H. S. Teoh wrote:

1..1 makes you think you got a prefix of a float 1. but actually you
got an int slice_operator int. If there where no .. flex like
generator could handle D, at least as far as I can see it.




 Point.

 But still, syntax that makes lexing or parsing hard -- that scores
 against this DIP.


No, it does not. The lexer does not need to change.
Mar 17 2014
parent reply Robert Schadek <realburner gmx.de> writes:
On 03/17/2014 11:43 PM, Timon Gehr wrote:

 No, it does not. The lexer does not need to change.


you're right, but at some distant  future I would like .. to be replaced
by :
Mar 17 2014
parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Monday, 17 March 2014 at 23:03:14 UTC, Robert Schadek wrote:

 On 03/17/2014 11:43 PM, Timon Gehr wrote:

 No, it does not. The lexer does not need to change.


 you're right, but at some distant  future I would like .. to be 
 replaced by :


Yes. I also like the .. notation in Dart where you keep holding 
onto a reference:

return new X() ..setSomething(1) ..answer=42 ..name="Somebody";

It is very handy sugar when dealing with DOMs.
Mar 18 2014