www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Example of Rust code

reply "bearophile" <bearophileHUGS lycos.com> writes:
(Repost from D.learn.)

Through Reddit I've found a page that shows a small example of 
Rust code:

http://www.reddit.com/r/programming/comments/xyfqg/playing_with_rust/
https://gist.github.com/3299083

The code:
https://gist.github.com/3307450

-----------------------------

So I've tried to translate this first part of the Rust code to D 
(I have not run it, but it looks correct):


enum expr {
     val(int),
     plus(&expr, &expr),
     minus(&expr, &expr)
}

fn eval(e: &expr) -> int {
     alt *e {
       val(i) => i,
       plus(a, b) => eval(a) + eval(b),
       minus(a, b) => eval(a) - eval(b)
     }
}

fn main() {
     let x = eval(
         &minus(&val(5),
                &plus(&val(3), &val(1))));

     io::println(#fmt("val: %i", x));
}


A comment from the little article:


putting a & in front of a expression allocates it on the stack 
and gives you a reference to it. so the lifetime of this tree is 
to the end of the run [main] function.<


-----------------------------

The first D version is easy enough to write, but:
- It uses classes, I think each class instance uses more memory 
than what's used in the original Rust code.
- All the class instances here are allocated on the Heap. This is 
less efficient than the Rust code, where all the data is 
stack-allocated.
- This code contains boilerplate, it's long.
- Writing eval() is easy, but in the first version of eval() 
there were two bugs.
- The assert(0) in eval() is not nice. There is no compile-time 
safety.
- The several dynamic casts in eval() are slow.


interface Expr {}

class Val : Expr {
     const int v;
     this(in int v_) pure nothrow {
         this.v = v_;
     }
}

class Plus : Expr {
     const Expr x, y;
     this(in Expr x_, in Expr y_) pure nothrow {
         this.x = x_;
         this.y = y_;
     }
}

class Minus : Expr {
     const Expr x, y;
     this(in Expr x_, in Expr y_) pure nothrow {
         this.x = x_;
         this.y = y_;
     }
}

int eval(in Expr e) pure nothrow {
     if (Val ve = cast(Val)e)
         return ve.v;
     else if (Plus pe = cast(Plus)e)
         return eval(pe.x) + eval(pe.y);
     else if (Minus me = cast(Minus)e)
         return eval(me.x) - eval(me.y);
     else
         assert(0);
}

void main() {
     auto ex = new Minus(new Val(5),
                         new Plus(new Val(3),
                                  new Val(1)));

     import std.stdio;
     writeln("Val: ", eval(ex));
}

-----------------------------

This second D version uses the same class definitions, but 
allocates the class instances on the stack. The code is bug prone 
and ugly. The other disadvantages are unchanged:


void main() {
     import std.stdio;
     import std.conv: emplace;
     import core.stdc.stdlib: alloca;

     enum size_t size_Val = __traits(classInstanceSize, Val);
     enum size_t size_Plus = __traits(classInstanceSize, Plus);
     enum size_t size_Minus = __traits(classInstanceSize, Minus);

     Val e1 = emplace!Val(alloca(size_Val)[0 .. size_Val], 5);
     Val e2 = emplace!Val(alloca(size_Val)[0 .. size_Val], 3);
     Val e3 = emplace!Val(alloca(size_Val)[0 .. size_Val], 1);
     Plus e4 = emplace!Plus(alloca(size_Plus)[0 .. size_Plus], e2, 
e3);
     Minus ex2 = emplace!Minus(alloca(size_Minus)[0 .. 
size_Minus], e1, e4);

     writeln("Val: ", eval(ex2));
}

-----------------------------

A third D version, using tagged structs:
- It doesn't look nice, and it's long.
- Class references can be null, so I have added tests at runtime 
in the pre-conditions. In the Rust code the "references" can't be 
null.
- The structs are stack-allocated but the main() code is not nice.
- The tags can't const or immutable, otherwise the compiler 
doesn't read the actual value of the various tags, assuming it's 
always Tag.none.
- Too many casts make this code bug-prone.


import std.stdio;

enum Tag { none, val, plus, minus }

struct Expr {
     Tag tag = Tag.none;
}

struct Val {
     Tag tag = Tag.val;
     immutable int v;

     this(int v_) pure nothrow {
         this.v = v_;
     }
}

struct Plus {
     Tag tag = Tag.plus;
     const Expr* x, y;

     this(in Expr* x_, in Expr* y_) pure nothrow
     in {
         assert(x_ != null);
         assert(y_ != null);
     } body {
         this.x = x_;
         this.y = y_;
     }
}

struct Minus {
     Tag tag = Tag.minus;
     const Expr* x, y;

     this(in Expr* x_, in Expr* y_) pure nothrow
     in {
         assert(x_ != null);
         assert(y_ != null);
     } body {
         this.x = x_;
         this.y = y_;
     }
}

int eval(in Expr* e) pure nothrow
in {
     assert(e);
} body {
     final switch (e.tag) {
         case Tag.none:
             assert(0);
         case Tag.val:
             return (cast(Val*)e).v;
         case Tag.plus:
             auto pe = cast(Plus*)e;
             return eval(pe.x) + eval(pe.y);
         case Tag.minus:
             auto me = cast(Minus*)e;
             return eval(me.x) - eval(me.y);
     }
}

void main() {
     const e1 = Val(5);
     const e2 = Val(3);
     const e3 = Val(1);
     const e4 = Plus(cast(Expr*)&e2, cast(Expr*)&e3);
     const ex = Minus(cast(Expr*)&e1, cast(Expr*)&e4);

     writeln("Val: ", eval(cast(Expr*)&ex));
}


Probably there are ways to improve my D versions, or to write 
better versions.

Bye,
bearophile
Aug 10 2012
next sibling parent reply "Tove" <tove fransson.se> writes:
On Friday, 10 August 2012 at 12:32:28 UTC, bearophile wrote:

 This second D version uses the same class definitions, but 
 allocates the class instances on the stack. The code is bug 
 prone and ugly. The other disadvantages are unchanged:


 void main() {
     import std.stdio;
     import std.conv: emplace;
     import core.stdc.stdlib: alloca;

     enum size_t size_Val = __traits(classInstanceSize, Val);
     enum size_t size_Plus = __traits(classInstanceSize, Plus);
     enum size_t size_Minus = __traits(classInstanceSize, Minus);

     Val e1 = emplace!Val(alloca(size_Val)[0 .. size_Val], 5);
     Val e2 = emplace!Val(alloca(size_Val)[0 .. size_Val], 3);
     Val e3 = emplace!Val(alloca(size_Val)[0 .. size_Val], 1);
     Plus e4 = emplace!Plus(alloca(size_Plus)[0 .. size_Plus], 
 e2, e3);
     Minus ex2 = emplace!Minus(alloca(size_Minus)[0 .. 
 size_Minus], e1, e4);

     writeln("Val: ", eval(ex2));
 }

 Probably there are ways to improve my D versions, or to write 
 better versions.

 Bye,
 bearophile


I think version 2 would be the easiest one to improve, by 
including a combined emplace/alloca convenience function in 
Phobos for this common use-case.

See the technique used in:
http://www.digitalmars.com/d/archives/digitalmars/D/run-time_stack-based_allocation_166305.html

"auto Create(void* buf=alloca(frame_size))"
Aug 10 2012
parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Tove:


 I think version 2 would be the easiest one to improve, by 
 including a combined emplace/alloca convenience function in 
 Phobos for this common use-case.

 See the technique used in:
 http://www.digitalmars.com/d/archives/digitalmars/D/run-time_stack-based_allocation_166305.html

 "auto Create(void* buf=alloca(frame_size))"


I see, thank you for the suggestion, seems interesting.
And thank you to Timon Gehr for his compacted code.

Walter's code seems to miss the point, but maybe he's trying to 
tell me something about very small demo programs.

Bye,
bearophile
Aug 10 2012
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/10/2012 3:46 PM, bearophile wrote:

 Walter's code seems to miss the point, but maybe he's trying to tell me
 something about very small demo programs.


If you want something allocated on the stack, us a struct, not a class.
It's what structs are for.

You can also use templates with overloading to get stack allocated parametric 
polymorphism and zero runtime overhead.

What I mean is:

1. If you write FORTRAN code in D, it will not work as well as writing FORTRAN 
in FORTRAN.

2. If you write C code in D, it will not work as well as writing C in C.

3. If you write Rust code in D, it will not work as well as writing Rust in
Rust.

If you want D code to perform, you gotta write it in D. Not in Rust, C, or Java.
Aug 10 2012
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/10/2012 4:19 PM, Walter Bright wrote:

 You can also use templates with overloading to get stack allocated parametric
 polymorphism and zero runtime overhead.


It appears that Rust does not have function overloading. Is this correct?
Aug 10 2012
parent reply =?ISO-8859-1?Q?Jos=E9_Armando_Garc=EDa_Sancio?= <jsancio gmail.com> writes:
On Fri, Aug 10, 2012 at 4:35 PM, Walter Bright
<newshound2 digitalmars.com> wrote:

 On 8/10/2012 4:19 PM, Walter Bright wrote:

 You can also use templates with overloading to get stack allocated
 parametric
 polymorphism and zero runtime overhead.



 It appears that Rust does not have function overloading. Is this correct?


That is correct.
Aug 10 2012
parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/10/2012 5:02 PM, José Armando García Sancio wrote:

 On Fri, Aug 10, 2012 at 4:35 PM, Walter Bright

 It appears that Rust does not have function overloading. Is this correct?


 That is correct.



Well, the type class thing looks like a lame substitute. Sorry.
Aug 10 2012
prev sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Walter Bright:

Thank you for the answer.


 3. If you write Rust code in D, it will not work as well as 
 writing Rust in Rust.

 If you want D code to perform, you gotta write it in D. Not in 
 Rust, C, or Java.


I agree. Every language has its strengths and its specific 
qualities, so you can't ask for a perfect translation from code 
in language X to language Y. On the other hand when X and Y 
languages are meant to be used for similar computing tasks, it's 
good to have some ways to translate the purposes of X code well 
enough to Y.

Regarding the problem David Piepgrass has explained what was the 
point of that Rust code, ans why your code was missing the point.

The main point of my little comparison was to show the usefulness 
of some Rust features that were discussed for D too, like pattern 
matching, tagged recursive structures (in Phobos there is 
std.variant.Algebraic, but it's currently not usable to write 
that code), and the original nice way of allocating a struct on 
the stack and return a reference to it, to build that expression 
tree.

I suggest to welcome future comparisons between D with Rust in 
this D newsgroup, because full ignorance of Rust will _not_ help 
D growth.

Bye,
bearophile
Aug 10 2012
prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
One simple possibility is

import std.stdio;
struct Expr{
     enum Tag { val, plus, minus }
     union{int i;struct{Expr* a, b;}}
     Tag tag;
}
Expr val(int i){ Expr e;e.tag=Expr.Tag.val;e.i=i;return e;}
Expr plus(Expr* a, Expr* b){Expr e;e.tag=Expr.Tag.plus; e.a=a; 
e.b=b;return e;}
Expr minus(Expr* a, Expr* b){Expr e;e.tag=Expr.Tag.minus; e.a=a; 
e.b=b;return e;}

int eval(scope Expr* e){
     final switch(e.tag) with(Expr.Tag){
         case val:   return e.i;
         case plus:  return eval(e.a) + eval(e.b);
         case minus: return eval(e.a) - eval(e.b);
     }
}
void main(){
     auto five = val(5), three = val(3), one = val(1);
     auto add  = plus(&three, &one);
     auto sub  = minus(&five, &add);
     auto x    = eval(&sub);
     writeln("val: ",x);
}

It would of course be better if D supported ADTs.
Aug 10 2012
parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Fri, 10 Aug 2012 15:56:53 +0200
schrieb Timon Gehr <timon.gehr gmx.ch>:


 int eval(scope Expr* e){
      final switch(e.tag) with(Expr.Tag){
          case val:   return e.i;
          case plus:  return eval(e.a) + eval(e.b);
          case minus: return eval(e.a) - eval(e.b);
      }
 }


Can you quickly explain the use of scope here? Does that mean "I wont keep a
reference to e"?
What are the implications? Does scope change the method signature? Does the
compiler enforce something? Will generated code differ? Does it prevent bugs or
is it documentation for the user of the function?
Thanks in advance for some insight!

-- 
Marco
Aug 11 2012
next sibling parent reply simendsjo <simendsjo gmail.com> writes:
On Sat, 11 Aug 2012 13:24:12 +0200, Marco Leise <Marco.Leise gmx.de> wrote:


 Am Fri, 10 Aug 2012 15:56:53 +0200
 schrieb Timon Gehr <timon.gehr gmx.ch>:


 int eval(scope Expr* e){
      final switch(e.tag) with(Expr.Tag){
          case val:   return e.i;
          case plus:  return eval(e.a) + eval(e.b);
          case minus: return eval(e.a) - eval(e.b);
      }
 }


 Can you quickly explain the use of scope here? Does that mean "I wont  
 keep a reference to e"?
 What are the implications? Does scope change the method signature? Does  
 the compiler enforce something? Will generated code differ? Does it  
 prevent bugs or is it documentation for the user of the function?
 Thanks in advance for some insight!


If I'm not mistaken, scope will enforce that the reference never escapes  
the function.
So you cannot pass it to other functions that might keep it's reference or  
store it in any way.
Aug 11 2012
parent "David Nadlinger" <see klickverbot.at> writes:
On Saturday, 11 August 2012 at 11:47:43 UTC, simendsjo wrote:

 If I'm not mistaken, scope will enforce that the reference 
 never escapes the function.
 So you cannot pass it to other functions that might keep it's 
 reference or store it in any way.


It _should_ enforce that, but its implementation is lacking at 
this point.

David
Aug 11 2012
prev sibling next sibling parent "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Saturday, 11 August 2012 at 11:24:35 UTC, Marco Leise wrote:

 Can you quickly explain the use of scope here? Does that mean 
 "I wont keep a reference to e"?
 What are the implications? Does scope change the method 
 signature? Does the compiler enforce something? Will generated 
 code differ? Does it prevent bugs or is it documentation for 
 the user of the function?
 Thanks in advance for some insight!


The generated code is different when the parameter is a delegate 
(no closure is allocated in cases of anonymous functions/lamdas 
or expressions like &myNestedFunction). It's supposed to be 
enforced by the compiler that no references escape, but currently 
it's just documentation beyond the case of delegates.
Aug 11 2012
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 08/11/2012 01:24 PM, Marco Leise wrote:

 Am Fri, 10 Aug 2012 15:56:53 +0200
 schrieb Timon Gehr<timon.gehr gmx.ch>:


 int eval(scope Expr* e){
       final switch(e.tag) with(Expr.Tag){
           case val:   return e.i;
           case plus:  return eval(e.a) + eval(e.b);
           case minus: return eval(e.a) - eval(e.b);
       }
 }


 Can you quickly explain the use of scope here?
 Does that mean "I wont keep a reference to e"?


It means "I won't keep a reference to *e", but I assume that is what
was meant.


 What are the implications?


The caller has some confidence that passing a pointer to stack-
allocated data is safe.


 Does scope change the method signature?


Yes. It is eg. impossible to override a method that has a scope
parameter with a method that does not have a scope parameter.


 Does the compiler enforce something?


In this case and currently, it is merely documentation.
I think it should be enforced and cast(scope) should be added
to allow non- safe code to escape the conservative analysis.


 Will generated code differ?


Only the mangled symbol name will differ. (unlike when scope is used on
delegate parameters, in this case it prevents closure allocation at the
call site.)


 Does it prevent bugs or is it documentation for the user of the function?


It is just documentation, both for the user and the maintainer.


 Thanks in advance for some insight!
Aug 11 2012
next sibling parent reply "David Nadlinger" <see klickverbot.at> writes:
On Saturday, 11 August 2012 at 22:17:44 UTC, Timon Gehr wrote:

 Will generated code differ?


 Only the mangled symbol name will differ. (unlike when scope is 
 used on
 delegate parameters, in this case it prevents closure 
 allocation at the call site.)


The code for callee stays the same, yes, but the code for the 
caller might change as the optimizer is free to take advantage of 
the fact that any reference in the parameters will not be escaped 
by the function. For example, LDC will stack-allocate dynamic 
arrays and objects if they are local to the function. [1]

David


[1] The fine print: We currently don't take advantage of "scope" 
parameters for this yet, though (it seems too dangerous with the 
related analysis not being implemented in the frontend), and for 
a completely unrelated reason, the code which performs the 
mentioned optimization is disabled in current master (but will be 
re-enabled in the near future, before the September release).
Aug 11 2012
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 08/12/2012 12:34 AM, David Nadlinger wrote:

 On Saturday, 11 August 2012 at 22:17:44 UTC, Timon Gehr wrote:

 Will generated code differ?


 Only the mangled symbol name will differ. (unlike when scope is used on
 delegate parameters, in this case it prevents closure allocation at
 the call site.)


 The code for callee stays the same, yes, but the code for the caller
 might change as the optimizer is free to take advantage of the fact that
 any reference in the parameters will not be escaped by the function. For
 example, LDC will stack-allocate dynamic arrays and objects if they are
 local to the function. [1]

 David


 [1] The fine print: We currently don't take advantage of "scope"
 parameters for this yet, though (it seems too dangerous with the related
 analysis not being implemented in the frontend),  and for a completely
 unrelated reason, the code which performs the mentioned optimization is
 disabled in current master (but will be re-enabled in the near future,
 before the September release).


Is there an upper bound on the amount of allocated memory? Implicit
stack-allocation of arbitrarily-sized dynamic arrays seems dangerous.
Aug 11 2012
prev sibling next sibling parent Marco Leise <Marco.Leise gmx.de> writes:
Am Sun, 12 Aug 2012 00:17:44 +0200
schrieb Timon Gehr <timon.gehr gmx.ch>:


 On 08/11/2012 01:24 PM, Marco Leise wrote:

 Am Fri, 10 Aug 2012 15:56:53 +0200
 schrieb Timon Gehr<timon.gehr gmx.ch>:


 int eval(scope Expr* e){
       final switch(e.tag) with(Expr.Tag){
           case val:   return e.i;
           case plus:  return eval(e.a) + eval(e.b);
           case minus: return eval(e.a) - eval(e.b);
       }
 }


 Can you quickly explain the use of scope here?
 Does that mean "I wont keep a reference to e"?


 
 It means "I won't keep a reference to *e", but I assume that is what
 was meant.
 

 What are the implications?


 
 The caller has some confidence that passing a pointer to stack-
 allocated data is safe.
 

 Does scope change the method signature?


 
 Yes. It is eg. impossible to override a method that has a scope
 parameter with a method that does not have a scope parameter.
 

 Does the compiler enforce something?


 
 In this case and currently, it is merely documentation.
 I think it should be enforced and cast(scope) should be added
 to allow non- safe code to escape the conservative analysis.
 

 Will generated code differ?


 
 Only the mangled symbol name will differ. (unlike when scope is used on
 delegate parameters, in this case it prevents closure allocation at the
 call site.)
 

 Does it prevent bugs or is it documentation for the user of the function?


 
 It is just documentation, both for the user and the maintainer.
 

 Thanks in advance for some insight!




Now that looks like a good 'scope' FAQ. thx

-- 
Marco
Aug 12 2012
prev sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Sun, 12 Aug 2012 00:17:44 +0200
schrieb Timon Gehr <timon.gehr gmx.ch>:


 On 08/11/2012 01:24 PM, Marco Leise wrote:

 Am Fri, 10 Aug 2012 15:56:53 +0200
 schrieb Timon Gehr<timon.gehr gmx.ch>:


 int eval(scope Expr* e){
       final switch(e.tag) with(Expr.Tag){
           case val:   return e.i;
           case plus:  return eval(e.a) + eval(e.b);
           case minus: return eval(e.a) - eval(e.b);
       }
 }


 Can you quickly explain the use of scope here?
 Does that mean "I wont keep a reference to e"?


 
 It means "I won't keep a reference to *e", but I assume that is what
 was meant.
 

 What are the implications?


 
 The caller has some confidence that passing a pointer to stack-
 allocated data is safe.
 

 Does scope change the method signature?


 
 Yes. It is eg. impossible to override a method that has a scope
 parameter with a method that does not have a scope parameter.
 

 Does the compiler enforce something?


 
 In this case and currently, it is merely documentation.
 I think it should be enforced and cast(scope) should be added
 to allow non- safe code to escape the conservative analysis.


There are probably some more, less known use cases. For example this
recent thread on stackoverflow shows how scope might be necessary to
initialize an immutable variable.
http://stackoverflow.com/questions/11860584/changing-immutable-members-inside-the-constructor
Aug 12 2012
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 08/12/2012 10:17 AM, Johannes Pfau wrote:

 There are probably some more, less known use cases. For example this
 recent thread on stackoverflow shows how scope might be necessary to
 initialize an immutable variable.
 http://stackoverflow.com/questions/11860584/changing-immutable-members-inside-the-constructor


Such rules are not part of the current language. The comment that says
that they are is wrong. scope does not currently influence type
checking at method call boundaries.
Aug 12 2012
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/10/2012 5:32 AM, bearophile wrote:

 Through Reddit I've found a page that shows a small example of Rust code:


Here's the D version:
-----------------------------------------
import std.stdio;

struct expr {
     int val;
     int eval() { return val; }
}

expr plus (expr a, expr b) { return expr(a.val + b.val); }
expr minus(expr a, expr b) { return expr(a.val - b.val); }

void main() {
     auto x = minus(expr(5), plus(expr(3), expr(1))).eval();
     writeln("val: ", x);
}
------------------------------------------

And the generated code:

------------------------------------------
__Dmain comdat
         assume  CS:__Dmain
L0:             push    EAX
                 mov     EAX,offset FLAT:_D3std5stdio6stdoutS3std5stdio4File
                 push    dword ptr FLAT:_DATA[0Ch]
                 push    dword ptr FLAT:_DATA[08h]
                 push    1
                 push    0Ah
                 call    near ptr 
_D3std5stdio4File18__T5writeTAyaTiTaZ5writeMFAyaiaZv
                 xor     EAX,EAX
                 pop     ECX
                 ret
----------------------------------------

I'd say we're doing all right.
Aug 10 2012
parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Walter Bright:


 I'd say we're doing all right.


Are you serious?

Bye,
bearophile
Aug 10 2012
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/10/2012 3:42 PM, bearophile wrote:

 Walter Bright:


 I'd say we're doing all right.


 Are you serious?


Yes. What's wrong with my D version? It's short and to the point, works, and 
produces optimal code.
Aug 10 2012
parent reply "David Piepgrass" <qwertie256 gmail.com> writes:
 I'd say we're doing all right.


 Are you serious?


 Yes. What's wrong with my D version? It's short and to the 
 point, works, and produces optimal code.


Your version is basically a very long-winded way to say "auto x = 
5 - (3 + 1);" so it really has nothing to do with the example.

The point of the example was to represent a simple AST and store 
it on the stack, not to represent + and - operators as plus() and 
minus() functions.

(I must say though, that while ADTs are useful for simple ASTs, I 
am not convinced that they scale to big and complex ASTs, let 
alone extensible ASTs, which I care about more. Nevertheless ADTs 
are at least useful for rapid prototyping, and pattern matching 
is really nice too. I'm sure somebody could at least write a D 
mixin for ADTs, if not pattern matching.)


1. If you write FORTRAN code in D, it will not work as well as 
writing
 FORTRAN in FORTRAN.
2. If you write C code in D, it will not work as well as writing 
C in C.


Really? And here I genuinely thought D was good enough for all 
the things C and FORTRAN are used for.


3. If you write Rust code in D, it will not work as well as 
writing Rust in Rust.


I hope someday to have a programming system whose features are 
not limited to whatever features the language designers saw fit 
to include -- a language where the users can add their own 
features, all the while maintaining "native efficiency" like D. 
That language would potentially allow Rust-like code, D-like 
code, Ruby-like code and even ugly C-like code.

I guess you don't want to be the one to kickstart that PL. I've 
been planning to do it myself, but so far the task seems just too 
big for one person.
Aug 10 2012
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/10/2012 5:19 PM, David Piepgrass wrote:

 Your version is basically a very long-winded way to say "auto x = 5 - (3 + 1);"
 so it really has nothing to do with the example.

 The point of the example was to represent a simple AST and store it on the
 stack, not to represent + and - operators as plus() and minus() functions.


I see that now, and I presented a D way of doing it using "expression 
templates", a technique pioneered by C++ programmers.

Expression templates have been used before in D, in particular to implement a 
regular expression engine. Dmitry Olshansky has since shown how fantastic this 
is for generating incredibly fast regex engines.

As far as I know, only C++ and D have sufficiently powerful abstraction 
mechanisms to make this possible.



 Really? And here I genuinely thought D was good enough for all the things C and
 FORTRAN are used for.


If you're going to write C code, use a C compiler. It's possible to use D as a
C 
compiler, but kinda pointless except as a transitory state towards using D 
capabilities.



 I hope someday to have a programming system whose features are not limited to
 whatever features the language designers saw fit to include -- a language where
 the users can add their own features, all the while maintaining "native
 efficiency" like D. That language would potentially allow Rust-like code,
D-like
 code, Ruby-like code and even ugly C-like code.

 I guess you don't want to be the one to kickstart that PL. I've been planning
to
 do it myself, but so far the task seems just too big for one person.


Andrei originally proposed to me a language like that. I talked him out of it
:-)
Aug 10 2012
prev sibling next sibling parent reply Russel Winder <russel winder.org.uk> writes:
On Sat, 2012-08-11 at 02:19 +0200, David Piepgrass wrote:
[=E2=80=A6]

 I hope someday to have a programming system whose features are=20
 not limited to whatever features the language designers saw fit=20
 to include -- a language where the users can add their own=20
 features, all the while maintaining "native efficiency" like D.=20
 That language would potentially allow Rust-like code, D-like=20
 code, Ruby-like code and even ugly C-like code.
=20
 I guess you don't want to be the one to kickstart that PL. I've=20
 been planning to do it myself, but so far the task seems just too=20
 big for one person.


<quasi-troll>
Isn't that language Lisp?
</quasi-troll>

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.n=
et
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Aug 11 2012
parent reply "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Saturday, 11 August 2012 at 14:45:55 UTC, Russel Winder wrote:

 On Sat, 2012-08-11 at 02:19 +0200, David Piepgrass wrote:
 […]

 I hope someday to have a programming system whose features are 
 not limited to whatever features the language designers saw 
 fit to include -- a language where the users can add their own 
 features, all the while maintaining "native efficiency" like 
 D. That language would potentially allow Rust-like code, 
 D-like code, Ruby-like code and even ugly C-like code.
 
 I guess you don't want to be the one to kickstart that PL. 
 I've been planning to do it myself, but so far the task seems 
 just too big for one person.


 <quasi-troll>
 Isn't that language Lisp?
 </quasi-troll>


You missed the native efficiency part :-)

I think XL is the closest thing that currently exists.

http://en.wikipedia.org/wiki/XL_(programming_language)
Aug 11 2012
next sibling parent reply "Paulo Pinto" <pjmlp progtools.org> writes:
On Saturday, 11 August 2012 at 16:12:14 UTC, Peter Alexander 
wrote:

 On Saturday, 11 August 2012 at 14:45:55 UTC, Russel Winder 
 wrote:

 On Sat, 2012-08-11 at 02:19 +0200, David Piepgrass wrote:
 […]

 I hope someday to have a programming system whose features 
 are not limited to whatever features the language designers 
 saw fit to include -- a language where the users can add 
 their own features, all the while maintaining "native 
 efficiency" like D. That language would potentially allow 
 Rust-like code, D-like code, Ruby-like code and even ugly 
 C-like code.
 
 I guess you don't want to be the one to kickstart that PL. 
 I've been planning to do it myself, but so far the task seems 
 just too big for one person.


 <quasi-troll>
 Isn't that language Lisp?
 </quasi-troll>


 You missed the native efficiency part :-)


You mean like the Common Lisp compilers that are able to beat 
FORTRAN compilers
in floating point computations?

http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.54.5725

--
Paulo
Aug 11 2012
next sibling parent reply "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Saturday, 11 August 2012 at 18:04:29 UTC, Paulo Pinto wrote:

 On Saturday, 11 August 2012 at 16:12:14 UTC, Peter Alexander 
 wrote:

 On Saturday, 11 August 2012 at 14:45:55 UTC, Russel Winder 
 wrote:

 On Sat, 2012-08-11 at 02:19 +0200, David Piepgrass wrote:
 […]

 I hope someday to have a programming system whose features 
 are not limited to whatever features the language designers 
 saw fit to include -- a language where the users can add 
 their own features, all the while maintaining "native 
 efficiency" like D. That language would potentially allow 
 Rust-like code, D-like code, Ruby-like code and even ugly 
 C-like code.
 
 I guess you don't want to be the one to kickstart that PL. 
 I've been planning to do it myself, but so far the task 
 seems just too big for one person.


 <quasi-troll>
 Isn't that language Lisp?
 </quasi-troll>


 You missed the native efficiency part :-)


 You mean like the Common Lisp compilers that are able to beat 
 FORTRAN compilers
 in floating point computations?

 http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.54.5725

 --
 Paulo


Not sure where you read that in the paper.

 From the conclusion:

"We have demonstrated that the speed of compiled Common Lisp 
code, though today somewhat slower than that of the best compiled 
Fortran, could probably be as efficient, and in some ways 
superior."

Probably is the operative word there.


 Most modern Lisp implementations employ JITing one way or 
 another, so
 you do get native code. Just not on the first run through a bit 
 of code.


JIT has its limits. A dynamically typed language is still 
dynamically typed once compiled. Sure the JIT may be able to 
deduce the types in some cases, but not all.

I do see your point, but in general it's still not as fast as 
optimised C.
Aug 11 2012
parent reply Paulo Pinto <pjmlp progtools.org> writes:
Am 11.08.2012 20:37, schrieb Peter Alexander:

 On Saturday, 11 August 2012 at 18:04:29 UTC, Paulo Pinto wrote:

 On Saturday, 11 August 2012 at 16:12:14 UTC, Peter Alexander wrote:

 On Saturday, 11 August 2012 at 14:45:55 UTC, Russel Winder wrote:

 On Sat, 2012-08-11 at 02:19 +0200, David Piepgrass wrote:
 […]

 I hope someday to have a programming system whose features are not
 limited to whatever features the language designers saw fit to
 include -- a language where the users can add their own features,
 all the while maintaining "native efficiency" like D. That language
 would potentially allow Rust-like code, D-like code, Ruby-like code
 and even ugly C-like code.

 I guess you don't want to be the one to kickstart that PL. I've
 been planning to do it myself, but so far the task seems just too
 big for one person.


 <quasi-troll>
 Isn't that language Lisp?
 </quasi-troll>


 You missed the native efficiency part :-)


 You mean like the Common Lisp compilers that are able to beat FORTRAN
 compilers
 in floating point computations?

 http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.54.5725

 --
 Paulo


 Not sure where you read that in the paper.

  From the conclusion:

 "We have demonstrated that the speed of compiled Common Lisp code,
 though today somewhat slower than that of the best compiled Fortran,
 could probably be as efficient, and in some ways superior."

 Probably is the operative word there.


Should have re-read the paper. It has been a few years since I fully 
read it. Still I think Common Lisp code is pretty efficient.


 Most modern Lisp implementations employ JITing one way or another, so
 you do get native code. Just not on the first run through a bit of code.


 JIT has its limits. A dynamically typed language is still dynamically
 typed once compiled. Sure the JIT may be able to deduce the types in
 some cases, but not all.

 I do see your point, but in general it's still not as fast as optimised C.


I imagine you wanted to answer to Russel's post.

On Lisp's case, most systems available today only JIT when using the 
REPL, as you can always compile to native code.

Type annotations help improve the speed in hotspot areas of your code,
in case you really need the extra speed for the application use case.

As for the speed of native code produced by JITs for dynamic languages,
I think Cog(Smalltalk), Self(Smaltalk ended up becoming JVM Hotspot),
PyPy (Python), V8(JavaScript), LLVM(Julia) prove that you can get pretty 
close to C for the majority of use cases that matter to the common user.

Actually a running assumption among the dynamic languages advocates is that
if dynamic languages had had as much money and research support as the
static languages field, the compilers would be much better by now.

--
Paulo
Aug 12 2012
parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Paulo Pinto:


 As for the speed of native code produced by JITs for dynamic 
 languages,
 I think Cog(Smalltalk), Self(Smaltalk ended up becoming JVM 
 Hotspot),
 PyPy (Python), V8(JavaScript), LLVM(Julia) prove that you can 
 get pretty close to C for the majority of use cases that matter 
 to the common user.


Among V8 developers thee are some ex Self developers.
I think at the moment there aren't enough Julia benchmarks to 
allow us to judge its performance well enough.
PyPy is surely not close to C speeds when it JITs Python code 
(PyPy developers need to stop using just their few benchmarks and 
try to optimize many other programs).
And you miss the best of the bunch, the Lua JIT.

One problem with JITs is that they give you a good performance if 
they are well implemented and if the the GC is good. While older 
languages produce decent performance even with a simple compiler.

Bye,
bearophile
Aug 12 2012
parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Sunday, 12 August 2012 at 11:28:28 UTC, bearophile wrote:

 Paulo Pinto:


 As for the speed of native code produced by JITs for dynamic 
 languages,
 I think Cog(Smalltalk), Self(Smaltalk ended up becoming JVM 
 Hotspot),
 PyPy (Python), V8(JavaScript), LLVM(Julia) prove that you can 
 get pretty close to C for the majority of use cases that 
 matter to the common user.


 Among V8 developers thee are some ex Self developers.
 I think at the moment there aren't enough Julia benchmarks to 
 allow us to judge its performance well enough.
 PyPy is surely not close to C speeds when it JITs Python code 
 (PyPy developers need to stop using just their few benchmarks 
 and try to optimize many other programs).
 And you miss the best of the bunch, the Lua JIT.


Yeah, silly me forgeting about LuaJIT.


 One problem with JITs is that they give you a good performance 
 if they are well implemented and if the the GC is good. While 
 older languages produce decent performance even with a simple 
 compiler.

 Bye,
 bearophile



True, this year's Google IO V8 talk has lots of informations on 
how
you write JavaScript code can influence the code quality 
generated by V8.

 From the compiler design geek I used to be, I find very 
interesting to
get myself informed about this area, and I think that dynamic 
languages JITs still have a lot of room to improve.

Although personally I prefer static languages with native 
compilers for my
own coding projects.

--
Paulo
Aug 12 2012
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/11/2012 11:04 AM, Paulo Pinto wrote:

 On Saturday, 11 August 2012 at 16:12:14 UTC, Peter Alexander wrote:

 You missed the native efficiency part :-)


 You mean like the Common Lisp compilers that are able to beat FORTRAN compilers
 in floating point computations?


Floating point code is a rather specialized subset of what a good native 
compiler can do.

For example, with Java, doing well with floating point has no relevance to the 
lack of user defined value types in Java, and the lack of efficiency that
entails.
Aug 11 2012
prev sibling parent Russel Winder <russel winder.org.uk> writes:
On Sat, 2012-08-11 at 18:12 +0200, Peter Alexander wrote:

 On Saturday, 11 August 2012 at 14:45:55 UTC, Russel Winder wrote:


[=E2=80=A6]

 <quasi-troll>
 Isn't that language Lisp?
 </quasi-troll>


=20
 You missed the native efficiency part :-)


Most modern Lisp implementations employ JITing one way or another, so
you do get native code. Just not on the first run through a bit of code.

[=E2=80=A6]

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.n=
et
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Aug 11 2012
prev sibling parent Philippe Sigaud <philippe.sigaud gmail.com> writes:
On Sat, Aug 11, 2012 at 2:19 AM, David Piepgrass <qwertie256 gmail.com> wrote:


 I must say though, that while ADTs are useful for simple ASTs, I am not
 convinced that they scale to big and complex ASTs, let alone extensible
 ASTs, which I care about more.


You mean AST for D code?


 Nevertheless ADTs are at least useful for
 rapid prototyping, and pattern matching is really nice too. I'm sure
 somebody could at least write a D mixin for ADTs, if not pattern matching.)


I did it, maybe 2 years ago. I worked for recursive ADT too (lists,
trees) and automatically generated small matchers, and maybe specific
map/reduce.
That would be easier today, with this CTFE++ we now have.

IIRC, it generated an abstract class with internal subtypes and a tag
to distinguish the state. I guess I could have used a union for the
fields, like Timon did further upthread. Hmm, does a union allow for
recursive fields?

I never tried to do generic pattern matchers, that would work also on
any struct and class, by using .tupleof. I daydreamed about it a few
times, tough, but never found a palatable syntax. Maybe with the new
() => syntax, that'd be better.



 I hope someday to have a programming system whose features are not limited
 to whatever features the language designers saw fit to include -- a language
 where the users can add their own features, all the while maintaining
 "native efficiency" like D. That language would potentially allow Rust-like
 code, D-like code, Ruby-like code and even ugly C-like code.

 I guess you don't want to be the one to kickstart that PL. I've been
 planning to do it myself, but so far the task seems just too big for one
 person.


Well, we are not far from having an official D lexer.
Then, an official D parser.


From this, adding user-defined extensions is not *that* complicated


(not simple, mind you, but doable).

* define lowerings (aka, translations from your extended syntax to D
syntax), maybe by snatching the unused macro keyword
* code a small wrapper around dmd, rdmd-like: given a file, it
extracts the macros, parses the extended code, transforms the
extensions, does that as many times as necessary, if some macros call
other macros.
* Discard the macros and then pass the transformed file to dmd.

It looks like C macros and preprocessor-based programming, but since
it knows the D grammar, it's nearer Lisp macros, I think.
Aug 11 2012
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/10/2012 3:42 PM, bearophile wrote:

 Walter Bright:


 I'd say we're doing all right.


 Are you serious?


You can also do things called "expression templates" in D:


import std.stdio;

auto val(T)(T v) { static struct S { T v; int eval() { return v; }} auto s = 
S(v); return s; }

auto plus(A,B)(A a, B b) { static struct S { A a; B b; int eval() { return 
a.eval() + b.eval(); }} auto s = S(a,b); return s; }

auto minus(A,B)(A a, B b) { static struct S { A a; B b; int eval() { return 
a.eval() - b.eval(); }} auto s = S(a,b); return s; }

void main() {

     auto x = minus(val(5), plus(val(3), val(1)));

     writeln("val: ", x.eval());
}

relying on "parametric polymorphism". You could reduce the repetitive 
boilerplate by using a template mixin, but I just banged this out in a few 
minutes, and it illustrates the idea. Note that there is no heap allocation 
anywhere, nor even any testing/branching.

The compiler should inline this, but doesn't, but that's not a fault in D. The 
inliner could be improved.
Aug 10 2012
prev sibling parent Artur Skawina <art.08.09 gmail.com> writes:
On 08/10/12 14:32, bearophile wrote:

 (Repost from D.learn.)
 
 Through Reddit I've found a page that shows a small example of Rust code:
 
 http://www.reddit.com/r/programming/comments/xyfqg/playing_with_rust/
 https://gist.github.com/3299083
 
 The code:
 https://gist.github.com/3307450
 
 -----------------------------
 
 So I've tried to translate this first part of the Rust code to D (I have not
run it, but it looks correct):
 
 
 enum expr {
     val(int),
     plus(&expr, &expr),
     minus(&expr, &expr)
 }
 
 fn eval(e: &expr) -> int {
     alt *e {
       val(i) => i,
       plus(a, b) => eval(a) + eval(b),
       minus(a, b) => eval(a) - eval(b)
     }
 }
 
 fn main() {
     let x = eval(
         &minus(&val(5),
                &plus(&val(3), &val(1))));
 
     io::println(#fmt("val: %i", x));
 }
 


Ugh. Haven't really read that article, but how about this
D version:

   import std.stdio;

   template ALIAS(alias A) { alias A ALIAS; }

   static struct Expr(string EVAL, A...) {
      A a;
      static if (is(typeof(a[0].eval)))
          property a0() { return a[0].eval; }
      else
         alias ALIAS!(a[0]) a0;
      static if (is(typeof(a[1]))) {
         static if (is(typeof(a[1].eval)))
             property a1() { return a[1].eval; }
         else
            alias ALIAS!(a[1]) a1;
      }
       property auto eval() {
         static if (is(typeof(mixin(EVAL))))
            return mixin(EVAL);
         else
            mixin(EVAL);
      }
      //alias eval this; // Uncommenting this line will enable automatic
                         // evaluation -- which may not always be desirable.

      auto opBinary(string op, B)(B b) {
         return Expr!("a0" ~ op ~ "a1", Expr, B)(this, b);
      }
   }

   auto Val(V)(V v) { return Expr!("a0", V)(v); }

   void main() {
      auto r = Val(5) - (Val(3) + Val(1));
      writeln("r: ", r, " == ", r.eval);
      auto s = sqr(Val(5) * Val(2) ^^ Val(3));
      writeln("s: ", s, " == ", s.eval);
   }

   auto sqr(T)(T a) { return Expr!("a0*a0", T)(a); }

which is more readable while being much more powerful.

But still trivial enough that the compiler (GDC) evaluates it all
at compile time, even without being asked to do so.

artur
Aug 11 2012