www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Random D geekout

reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
I'm writing some code that does some very simplistic parsing, and I'm
just totally geeking out on how awesome D is for writing such code:

	import std.conv;
	import std.regex;
	import std.stdio;

	struct Data {
		string name;
		string phone;
		int age;
		... // a whole bunch of other stuff
	}

	void main() {
		Data d;
		foreach (line; stdin.byLine()) {
			auto m = match(line, "(\w+)\s+(\w+)");
			if (!m) continue;

			auto key = m.captures[1];
			auto value = m.captures[2];

			alias void delegate(string key, string value) attrDg;
			attrDg[string] dgs = [
				"name": delegate(string key, string value) {
					d.name = value;
				},
				"phone": delegate(string key, string value) {
					d.phone = value;
				},
				"age": delegate(string key, string value) {
					d.age = to!int(value);
				},
				...	// whole bunch of other stuff to
					// parse different attributes
			];
			attrDg errordg = delegate(string key, string value) {
				throw Exception("Invalid attribute '%s'"
					.format(key));
			};

			// This is pure awesomeness:
			dgs.get(key.idup, errordg)(key.idup, value.idup);
		}
		// ... do something with Data
	}

Basically, I use std.regex to extract keywords from the input, then use
an AA to map keywords to code that implement said keyword.  That AA of
delegates is just pure awesomeness. AA.get's default value parameter
lets you process keywords and handle errors with a single AA lookup.  I
mean, this is even better than Perl for this kind of text-processing
code!

The only complaint is that I couldn't write auto[string] dgs and have
the compiler auto-infer the delegate type. :-) Additionally, I wasn't
sure if I could omit the "delegate(string,string)" after each keyword;
if that's actually allowed, then this would make D totally pwn Perl!!

(I left out some stuff that makes this code even more of a joy to write:
using nested try/catch blocks, I can throw exceptions from deep-down
parsing code and have the loop that loops over input lines automatically
prefix error messages with the filename/line number where the error
occurred. This way, even errors thrown by to!int() will be formatted
nicely. With Perl, this gets extremely messy due to its pathological use
of $. for line numbers which can get overwritten in unexpected places if
you're processing more than one file at a time.)

Did I mention I'm totally in love with D?? Seriously. It can handle
system-level code and "high-level" text-processing code with total
impunity. What's there not to like?!


T

-- 
Without geometry, life would be pointless. -- VS
Apr 19 2012
next sibling parent "Nick Sabalausky" <SeeWebsiteToContactMe semitwist.com> writes:
"H. S. Teoh" <hsteoh quickfur.ath.cx> wrote in message 
news:mailman.1953.1334894800.4860.digitalmars-d puremagic.com...

 I'm writing some code that does some very simplistic parsing, and I'm
 just totally geeking out on how awesome D is for writing such code:



Heh, yup :)

I grew up on C/C++ (well, after outgrowing BASIC anyway), and one of the 
first things that blew me away about D was its string-processing.


 alias void delegate(string key, string value) attrDg;
 attrDg[string] dgs = [
 "name": delegate(string key, string value) {
 d.name = value;
 },
 "phone": delegate(string key, string value) {
 d.phone = value;
 },
 "age": delegate(string key, string value) {
 d.age = to!int(value);
 },
 ... // whole bunch of other stuff to
 // parse different attributes
 ];



Yea, I've done the same trick :) Fantastic stuff. But the one issue I have 
with it is that you can't do this:

void delegate()[string] dgs = [
    "name": delegate() {
        // do stuff
    },
    "phone": delegate() {
        // do stuff
        dgs["name"](); // ERR! (Shit!)
        // do stuff
    }
];

That limitation is kind of annoying sometimes. I think I filed a ticket for 
it...

http://d.puremagic.com/issues/show_bug.cgi?id=3995

Ahh, shit, it's been marked invalid :(


Did I mention I'm totally in love with D?? Seriously. It can handle
system-level code and "high-level" text-processing code with total
impunity. What's there not to like?!



Yup. Like, totally. :)
Apr 19 2012
prev sibling next sibling parent reply Denis Shelomovskij <verylonglogin.reg gmail.com> writes:
20.04.2012 8:06, H. S. Teoh написал:

 I'm writing some code that does some very simplistic parsing, and I'm
 just totally geeking out on how awesome D is for writing such code:

 	import std.conv;
 	import std.regex;
 	import std.stdio;

 	struct Data {
 		string name;
 		string phone;
 		int age;
 		... // a whole bunch of other stuff
 	}

 	void main() {
 		Data d;
 		foreach (line; stdin.byLine()) {
 			auto m = match(line, "(\w+)\s+(\w+)");



It's better not to create a regex every iteration. Use e.g.
---
auto regEx = regex(`(\w+)\s+(\w+)`);
---
before foreach. Of course, you are not claiming this as a 
high-performance program, but creating a regex every iteration is too 
common mistake to show such code to newbies.


 			if (!m) continue;

 			auto key = m.captures[1];



One `.idup` here will be better. (sorry, just like to nitpick)


 			auto value = m.captures[2];

 			alias void delegate(string key, string value) attrDg;
 			attrDg[string] dgs = [
 				"name": delegate(string key, string value) {
 					d.name = value;
 				},
 				"phone": delegate(string key, string value) {
 					d.phone = value;
 				},
 				"age": delegate(string key, string value) {
 					d.age = to!int(value);
 				},
 				...	// whole bunch of other stuff to
 					// parse different attributes
 			];
 			attrDg errordg = delegate(string key, string value) {
 				throw Exception("Invalid attribute '%s'"
 					.format(key));
 			};

 			// This is pure awesomeness:
 			dgs.get(key.idup, errordg)(key.idup, value.idup);
 		}
 		// ... do something with Data
 	}

 Basically, I use std.regex to extract keywords from the input, then use
 an AA to map keywords to code that implement said keyword.  That AA of
 delegates is just pure awesomeness. AA.get's default value parameter
 lets you process keywords and handle errors with a single AA lookup.  I
 mean, this is even better than Perl for this kind of text-processing
 code!

 The only complaint is that I couldn't write auto[string] dgs and have
 the compiler auto-infer the delegate type. :-) Additionally, I wasn't
 sure if I could omit the "delegate(string,string)" after each keyword;
 if that's actually allowed, then this would make D totally pwn Perl!!



A shorter variant:
---
void delegate(string, string)[string] dgs = [
	"name" : (key, value) { d.name = value; },
	"phone": (key, value) { d.phone = value; },
	"age"  : (key, value) { d.age = to!int(value); },
	...	// whole bunch of other stuff to
		// parse different attributes
];

// `delegate` is needed because otherwise `errordg` will be inferred
// as a `function`, not `delegate` and `dgs.get` will fail
auto errordg = delegate(string key, string value) {
	throw new Exception("Invalid attribute '%s'"
		.format(key));
};
---


 (I left out some stuff that makes this code even more of a joy to write:
 using nested try/catch blocks, I can throw exceptions from deep-down
 parsing code and have the loop that loops over input lines automatically
 prefix error messages with the filename/line number where the error
 occurred. This way, even errors thrown by to!int() will be formatted
 nicely. With Perl, this gets extremely messy due to its pathological use
 of $. for line numbers which can get overwritten in unexpected places if
 you're processing more than one file at a time.)

 Did I mention I'm totally in love with D?? Seriously. It can handle
 system-level code and "high-level" text-processing code with total
 impunity. What's there not to like?!


 T




-- 
Денис В. Шеломовский
Denis V. Shelomovskij
Apr 19 2012
next sibling parent Ary Manzana <ary esperanto.org.ar> writes:
On 4/20/12 1:09 PM, H. S. Teoh wrote:

 On Fri, Apr 20, 2012 at 08:44:06AM +0400, Denis Shelomovskij wrote:

 20.04.2012 8:06, H. S. Teoh написал:

 I'm writing some code that does some very simplistic parsing, and I'm
 just totally geeking out on how awesome D is for writing such code:

 	import std.conv;
 	import std.regex;
 	import std.stdio;

 	struct Data {
 		string name;
 		string phone;
 		int age;
 		... // a whole bunch of other stuff
 	}

 	void main() {
 		Data d;
 		foreach (line; stdin.byLine()) {
 			auto m = match(line, "(\w+)\s+(\w+)");



 It's better not to create a regex every iteration. Use e.g.
 ---
 auto regEx = regex(`(\w+)\s+(\w+)`);
 ---
 before foreach. Of course, you are not claiming this as a
 high-performance program, but creating a regex every iteration is
 too common mistake to show such code to newbies.



 You're right, it was unoptimized code. I ended up using ctRegex for
 them:

 	enum attrRx = ctRegex!`...`;
 	enum blockRx = ctRegex!`...`;

 	if (auto m = match(line, attrRx)) {
 		...
 	} else if (auto m = match(line, blockRx)) {
 		...
 	}

 The fact that D enums can be arbitrary types is just beyond awesome.



No, enum there means "manifest constant", it has nothing to do with an 
enumeration...
Apr 19 2012
prev sibling next sibling parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 20.04.2012 8:44, Denis Shelomovskij wrote:

 20.04.2012 8:06, H. S. Teoh написал:

 I'm writing some code that does some very simplistic parsing, and I'm
 just totally geeking out on how awesome D is for writing such code:

 import std.conv;
 import std.regex;
 import std.stdio;

 struct Data {
 string name;
 string phone;
 int age;
 ... // a whole bunch of other stuff
 }

 void main() {
 Data d;
 foreach (line; stdin.byLine()) {
 auto m = match(line, "(\w+)\s+(\w+)");



 It's better not to create a regex every iteration. Use e.g.
 ---
 auto regEx = regex(`(\w+)\s+(\w+)`);
 ---
 before foreach. Of course, you are not claiming this as a
 high-performance program, but creating a regex every iteration is too
 common mistake to show such code to newbies.



And that's why I pluged this hole - it happens too often. At least up to 
mm... 16 regexes are cached.



-- 
Dmitry Olshansky
Apr 19 2012
prev sibling parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 20.04.2012 9:19, F i L wrote:

 Denis Shelomovskij wrote:

 A shorter variant:
 ---
 void delegate(string, string)[string] dgs = [
 "name" : (key, value) { d.name = value; },
 "phone": (key, value) { d.phone = value; },
 "age" : (key, value) { d.age = to!int(value); },
 ... // whole bunch of other stuff to
 // parse different attributes
 ];






How about putting it on dlang?( [your code here] )

-- 
Dmitry Olshansky
Apr 19 2012
prev sibling next sibling parent "F i L" <witte2008 gmail.com> writes:
Denis Shelomovskij wrote:

 A shorter variant:
 ---
 void delegate(string, string)[string] dgs = [
 	"name" : (key, value) { d.name = value; },
 	"phone": (key, value) { d.phone = value; },
 	"age"  : (key, value) { d.age = to!int(value); },
 	...	// whole bunch of other stuff to
 		// parse different attributes
 ];



That's a pretty slick example of D's type inference. This example 
is worthy of a reference in the docs somewhere, IMO. Although, 
written to use UFCS of course:

     auto m = line.match("(\w+)\s+(\w+)");

     ...

     "age" : (key, value) { d.age = value.to!int(); }

:D gotta love UFCS!
Apr 19 2012
prev sibling next sibling parent "F i L" <witte2008 gmail.com> writes:
Denis Shelomovskij wrote:

 A shorter variant:
 ---
 void delegate(string, string)[string] dgs = [
 	"name" : (key, value) { d.name = value; },
 	"phone": (key, value) { d.phone = value; },
 	"age"  : (key, value) { d.age = to!int(value); },
 	...	// whole bunch of other stuff to
 		// parse different attributes
 ];



That's a pretty slick example of D's type inference. This example 
is worthy of a reference in the docs somewhere, IMO. Although, 
written to use UFCS of course:

     auto m = line.match("(\w+)\s+(\w+)");

     ...

     "age" : (key, value) { d.age = value.to!int(); }

:D gotta love UFCS!
Apr 19 2012
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-04-20 06:06, H. S. Teoh wrote:

 I'm writing some code that does some very simplistic parsing, and I'm
 just totally geeking out on how awesome D is for writing such code:

 	import std.conv;
 	import std.regex;
 	import std.stdio;

 	struct Data {
 		string name;
 		string phone;
 		int age;
 		... // a whole bunch of other stuff
 	}

 	void main() {
 		Data d;
 		foreach (line; stdin.byLine()) {
 			auto m = match(line, "(\w+)\s+(\w+)");
 			if (!m) continue;

 			auto key = m.captures[1];
 			auto value = m.captures[2];

 			alias void delegate(string key, string value) attrDg;
 			attrDg[string] dgs = [
 				"name": delegate(string key, string value) {
 					d.name = value;
 				},
 				"phone": delegate(string key, string value) {
 					d.phone = value;
 				},
 				"age": delegate(string key, string value) {
 					d.age = to!int(value);
 				},
 				...	// whole bunch of other stuff to
 					// parse different attributes
 			];
 			attrDg errordg = delegate(string key, string value) {
 				throw Exception("Invalid attribute '%s'"
 					.format(key));
 			};

 			// This is pure awesomeness:
 			dgs.get(key.idup, errordg)(key.idup, value.idup);
 		}
 		// ... do something with Data
 	}

 Basically, I use std.regex to extract keywords from the input, then use
 an AA to map keywords to code that implement said keyword.  That AA of
 delegates is just pure awesomeness. AA.get's default value parameter
 lets you process keywords and handle errors with a single AA lookup.  I
 mean, this is even better than Perl for this kind of text-processing
 code!

 The only complaint is that I couldn't write auto[string] dgs and have
 the compiler auto-infer the delegate type. :-) Additionally, I wasn't
 sure if I could omit the "delegate(string,string)" after each keyword;
 if that's actually allowed, then this would make D totally pwn Perl!!



I think you should be able to write:

"age": (key, value) {
     d.age = to!int(value);
}

Or perhaps even:

"age": (key, value) => d.age = to!int(value);

-- 
/Jacob Carlborg
Apr 20 2012
prev sibling next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 20 Apr 2012 00:06:41 -0400, H. S. Teoh <hsteoh quickfur.ath.cx>  
wrote:



 The only complaint is that I couldn't write auto[string] dgs and have
 the compiler auto-infer the delegate type. :-)



Does this not work?

auto dgs = ...

Also, it doesn't look like that needs to be in the inner loop.  Each time  
you specify an AA literal, it allocates a new one.  So you are allocating  
another AA literal per line.

-Steve
Apr 20 2012
next sibling parent =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 04/20/2012 11:50 AM, Arne wrote:

 On Friday, 20 April 2012 at 11:23:49 UTC, Steven Schveighoffer wrote:

 On Fri, 20 Apr 2012 00:06:41 -0400, H. S. Teoh
 <hsteoh quickfur.ath.cx> wrote:



 The only complaint is that I couldn't write auto[string] dgs and have
 the compiler auto-infer the delegate type. :-)



 Does this not work?

 auto dgs = ...

 Also, it doesn't look like that needs to be in the inner loop. Each
 time you specify an AA literal, it allocates a new one. So you are
 allocating another AA literal per line.

 -Steve



 auto dgs =
 [
 "name": (string value) {d.name = value; },
 "phone": (string value) => cast(void)(d.phone = value),
 "age": (string value) => cast(void)(d.age = value.to!int()),
 ];

 This works... is there a better way, to avoid cast?



The => syntax replaces:

- parentheses around the parameter if there is only one parameter
- curly brackets
- the return keyword
- the semicolon at the end of the return statement

   http://dlang.org/expression.html#Lambda

So => is most suitable when there is a single return statement.

Ali
Apr 20 2012
prev sibling next sibling parent "Nick Sabalausky" <SeeWebsiteToContactMe semitwist.com> writes:
"Arne" <arne linux.nu> wrote in message 
news:qmehxgyksrdxkabvcyiv forum.dlang.org...

 auto dgs =
 [
   "name":  (string value) {d.name = value; },
   "phone": (string value) => cast(void)(d.phone = value),
   "age":   (string value) => cast(void)(d.age   = value.to!int()),
 ];

 This works... is there a better way, to avoid cast?



Yes: Don't use the lambda syntax when it's not a lambda ;)
Apr 20 2012
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-04-20 20:50, Arne wrote:


 auto dgs =
 [
 "name": (string value) {d.name = value; },
 "phone": (string value) => cast(void)(d.phone = value),
 "age": (string value) => cast(void)(d.age = value.to!int()),
 ];

 This works... is there a better way, to avoid cast?



You could try explicitly declare the type of "dgs" and see if that helps.

-- 
/Jacob Carlborg
Apr 21 2012
prev sibling next sibling parent reply "ixid" <nuaccount gmail.com> writes:
As a D learner this thread is very interesting. It would be great 
to maintain it with a polished and error catching version that 
incorporates people's tweaks.
Apr 20 2012
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 21.04.2012 11:46, SomeDude wrote:

 I can't compile it. I get "Out of memory". Is it the regex.d module
 again ?:(
 This one really needs to be fixed ASAP, as the older working



Ah-ha-ha. OK, come on use it the source are out there in the open :)
It didn't even handle * properly.

  regexp is

 deprecated.



Just stop using ctRegex for now... it's experimental.

Or more to the point the problem is this. I've seen this one on bugzilla:

version(CtRgx) {
		enum Re = ctRegex!re;//auto is OK here BTW
	} else {//that's the problem. It's _parsed_ at compile-time
		static Re = regex(re);//switch static to auto
	}
}

And there is little I can do untill CTFE stops bleeding RAM.

-- 
Dmitry Olshansky
Apr 21 2012
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 21.04.2012 14:48, SomeDude wrote:

 On Saturday, 21 April 2012 at 10:21:49 UTC, Dmitry Olshansky wrote:

 Just stop using ctRegex for now... it's experimental.

 Or more to the point the problem is this. I've seen this one on bugzilla:

 version(CtRgx) {
 enum Re = ctRegex!re;//auto is OK here BTW
 } else {//that's the problem. It's _parsed_ at compile-time
 static Re = regex(re);//switch static to auto
 }
 }

 And there is little I can do untill CTFE stops bleeding RAM.



 Well, neither of those works:

 version(CtRgx) {
 auto Re = ctRegex!re;//auto is OK here BTW
 } else {//that's the problem. It's _parsed_ at compile-time
 auto Re = regex(re);//switch static to auto
 }



Probably there are other cases where compiler const folds stuff I dunno.

-- 
Dmitry Olshansky
Apr 21 2012
parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 21.04.2012 18:41, H. S. Teoh wrote:
[snip]

 How do I make it module-global without being compile-time evaluated??



No idea ;)

But as a workaround:

Global blah;

static this(){
	blah = ...;
}

-- 
Dmitry Olshansky
Apr 21 2012
prev sibling next sibling parent "Arne" <arne linux.nu> writes:
On Friday, 20 April 2012 at 11:23:49 UTC, Steven Schveighoffer 
wrote:

 On Fri, 20 Apr 2012 00:06:41 -0400, H. S. Teoh 
 <hsteoh quickfur.ath.cx> wrote:



 The only complaint is that I couldn't write auto[string] dgs 
 and have
 the compiler auto-infer the delegate type. :-)



 Does this not work?

 auto dgs = ...

 Also, it doesn't look like that needs to be in the inner loop.  
 Each time you specify an AA literal, it allocates a new one.  
 So you are allocating another AA literal per line.

 -Steve



auto dgs =
[
   "name":  (string value) {d.name = value; },
   "phone": (string value) => cast(void)(d.phone = value),
   "age":   (string value) => cast(void)(d.age   = value.to!int()),
];

This works... is there a better way, to avoid cast?
Apr 20 2012
prev sibling next sibling parent "Arne" <arne linux.nu> writes:
On Friday, 20 April 2012 at 19:00:29 UTC, Nick Sabalausky wrote:

 "Arne" <arne linux.nu> wrote in message
 news:qmehxgyksrdxkabvcyiv forum.dlang.org...

 auto dgs =
 [
   "name":  (string value) {d.name = value; },
   "phone": (string value) => cast(void)(d.phone = value),
   "age":   (string value) => cast(void)(d.age   = 
 value.to!int()),
 ];

 This works... is there a better way, to avoid cast?



 Yes: Don't use the lambda syntax when it's not a lambda ;)



But... but... that sounds entirely too reasonable! ;)
Apr 21 2012
prev sibling parent "nhk" <nhkcon googlemail.com> writes:
Please bear with my ignorance I'm new to D, but why is that any
better compared to a simple

switch(key){
default: throw Exception("Invalid attribute '%s'".format(key));
case "name": d.name = value;
                break;
...
...
}



On Friday, 20 April 2012 at 04:05:43 UTC, H. S. Teoh wrote:

 I'm writing some code that does some very simplistic parsing, 
 and I'm
 just totally geeking out on how awesome D is for writing such 
 code:

 	import std.conv;
 	import std.regex;
 	import std.stdio;

 	struct Data {
 		string name;
 		string phone;
 		int age;
 		... // a whole bunch of other stuff
 	}

 	void main() {
 		Data d;
 		foreach (line; stdin.byLine()) {
 			auto m = match(line, "(\w+)\s+(\w+)");
 			if (!m) continue;

 			auto key = m.captures[1];
 			auto value = m.captures[2];

 			alias void delegate(string key, string value) attrDg;
 			attrDg[string] dgs = [
 				"name": delegate(string key, string value) {
 					d.name = value;
 				},
 				"phone": delegate(string key, string value) {
 					d.phone = value;
 				},
 				"age": delegate(string key, string value) {
 					d.age = to!int(value);
 				},
 				...	// whole bunch of other stuff to
 					// parse different attributes
 			];
 			attrDg errordg = delegate(string key, string value) {
 				throw Exception("Invalid attribute '%s'"
 					.format(key));
 			};

 			// This is pure awesomeness:
 			dgs.get(key.idup, errordg)(key.idup, value.idup);
 		}
 		// ... do something with Data
 	}

 Basically, I use std.regex to extract keywords from the input, 
 then use
 an AA to map keywords to code that implement said keyword.  
 That AA of
 delegates is just pure awesomeness. AA.get's default value 
 parameter
 lets you process keywords and handle errors with a single AA 
 lookup.  I
 mean, this is even better than Perl for this kind of 
 text-processing
 code!

 The only complaint is that I couldn't write auto[string] dgs 
 and have
 the compiler auto-infer the delegate type. :-) Additionally, I 
 wasn't
 sure if I could omit the "delegate(string,string)" after each 
 keyword;
 if that's actually allowed, then this would make D totally pwn 
 Perl!!

 (I left out some stuff that makes this code even more of a joy 
 to write:
 using nested try/catch blocks, I can throw exceptions from 
 deep-down
 parsing code and have the loop that loops over input lines 
 automatically
 prefix error messages with the filename/line number where the 
 error
 occurred. This way, even errors thrown by to!int() will be 
 formatted
 nicely. With Perl, this gets extremely messy due to its 
 pathological use
 of $. for line numbers which can get overwritten in unexpected 
 places if
 you're processing more than one file at a time.)

 Did I mention I'm totally in love with D?? Seriously. It can 
 handle
 system-level code and "high-level" text-processing code with 
 total
 impunity. What's there not to like?!


 T
Apr 21 2012