www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - Silicon Valley D Meetup - January 26, 2017 - "High Performance Tools

reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
Our long time member and friend Jon Degenhardt has graciously accepted 
to present on very short notice.

He will give a preview of some performance benchmarks that he has been 
running of tools in C, Rust, and Go that overlap with what Jon writes in 
D at eBay. Jon has been observing that D versions are faster in nearly 
all cases, and not by a small margin. In a total surprise, Jon's version 
of ‘cut’ is faster than GNU cut on large files: On a 4.8GB, 7M lines 
test file, GNU cut takes 12.4 sec, while Jon's version takes 4.2 sec. 
(GNU cut is faster on small files.)

The big take-away is that this was achieved without a lot of low-level 
coding, using mostly high level D primitives and the standard library. 
There was some tuning and lessons learned, but nothing extensive.

 
https://www.meetup.com/D-Lang-Silicon-Valley/events/236421472/?eventId=236421472

I may post a link to Google Hangouts here at the time of the event (7pm 
Pacific time). Hopefully, the mic will not be muted. (True story! :p)

Ali
Jan 25 2017
next sibling parent Adil Baig via Digitalmars-d-announce writes:
Can't wait! Please ask Jon to write something up on it. For posterity

On Thu, Jan 26, 2017 at 1:23 PM, Ali =C3=87ehreli via Digitalmars-d-announc=
e <
digitalmars-d-announce puremagic.com> wrote:


 Our long time member and friend Jon Degenhardt has graciously accepted to
 present on very short notice.

 He will give a preview of some performance benchmarks that he has been
 running of tools in C, Rust, and Go that overlap with what Jon writes in =


D

 at eBay. Jon has been observing that D versions are faster in nearly all
 cases, and not by a small margin. In a total surprise, Jon's version of
 =E2=80=98cut=E2=80=99 is faster than GNU cut on large files: On a 4.8GB, =


7M lines test

 file, GNU cut takes 12.4 sec, while Jon's version takes 4.2 sec. (GNU cut
 is faster on small files.)

 The big take-away is that this was achieved without a lot of low-level
 coding, using mostly high level D primitives and the standard library.
 There was some tuning and lessons learned, but nothing extensive.


 https://www.meetup.com/D-Lang-Silicon-Valley/events/23642147
 2/?eventId=3D236421472

 I may post a link to Google Hangouts here at the time of the event (7pm
 Pacific time). Hopefully, the mic will not be muted. (True story! :p)

 Ali
Jan 26 2017
prev sibling next sibling parent reply Jack Stouffer <jack jackstouffer.com> writes:
On Thursday, 26 January 2017 at 07:53:22 UTC, Ali Çehreli wrote:

 I may post a link to Google Hangouts here at the time of the 
 event (7pm Pacific time). Hopefully, the mic will not be muted. 
 (True story! :p)

 Ali


If it's in hangouts, you can use Hangouts On Air to stream it to 
YouTube.

D could always use more content on YouTube, as Rust and Go swamp 
us on this point.
Jan 26 2017
parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 01/26/2017 08:59 AM, Jack Stouffer wrote:

 On Thursday, 26 January 2017 at 07:53:22 UTC, Ali Çehreli wrote:

 I may post a link to Google Hangouts here at the time of the event
 (7pm Pacific time). Hopefully, the mic will not be muted. (True story!
 :p)

 Ali


 If it's in hangouts, you can use Hangouts On Air to stream it to YouTube.

 D could always use more content on YouTube, as Rust and Go swamp us on
 this point.


Yes, On Air is the idea.

Ali
Jan 26 2017
prev sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
We're live now:

 
https://hangouts.google.com/hangouts/_/ytl/QZmU0_QJnXuKGdftK-0D7QT4QbVn09_YyOMFHXoNDt8=?hl=en_US&authuser=0

Ali

On 01/25/2017 11:53 PM, Ali Çehreli wrote:

 Our long time member and friend Jon Degenhardt has graciously accepted
 to present on very short notice.

 He will give a preview of some performance benchmarks that he has been
 running of tools in C, Rust, and Go that overlap with what Jon writes in
 D at eBay. Jon has been observing that D versions are faster in nearly
 all cases, and not by a small margin. In a total surprise, Jon's version
 of ‘cut’ is faster than GNU cut on large files: On a 4.8GB, 7M lines
 test file, GNU cut takes 12.4 sec, while Jon's version takes 4.2 sec.
 (GNU cut is faster on small files.)

 The big take-away is that this was achieved without a lot of low-level
 coding, using mostly high level D primitives and the standard library.
 There was some tuning and lessons learned, but nothing extensive.


 https://www.meetup.com/D-Lang-Silicon-Valley/events/236421472/?eventId=236421472


 I may post a link to Google Hangouts here at the time of the event (7pm
 Pacific time). Hopefully, the mic will not be muted. (True story! :p)

 Ali
Jan 26 2017
parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
And this:

   http://youtu.be/-DK4r5xewTY

Ali

On 01/26/2017 07:26 PM, Ali Çehreli wrote:

 We're live now:


 https://hangouts.google.com/hangouts/_/ytl/QZmU0_QJnXuKGdftK-0D7QT4QbVn09_YyOMFHXoNDt8=?hl=en_US&authuser=0


 Ali

 On 01/25/2017 11:53 PM, Ali Çehreli wrote:

 Our long time member and friend Jon Degenhardt has graciously accepted
 to present on very short notice.

 He will give a preview of some performance benchmarks that he has been
 running of tools in C, Rust, and Go that overlap with what Jon writes in
 D at eBay. Jon has been observing that D versions are faster in nearly
 all cases, and not by a small margin. In a total surprise, Jon's version
 of ‘cut’ is faster than GNU cut on large files: On a 4.8GB, 7M lines
 test file, GNU cut takes 12.4 sec, while Jon's version takes 4.2 sec.
 (GNU cut is faster on small files.)

 The big take-away is that this was achieved without a lot of low-level
 coding, using mostly high level D primitives and the standard library.
 There was some tuning and lessons learned, but nothing extensive.


 https://www.meetup.com/D-Lang-Silicon-Valley/events/236421472/?eventId=236421472



 I may post a link to Google Hangouts here at the time of the event (7pm
 Pacific time). Hopefully, the mic will not be muted. (True story! :p)

 Ali
Jan 26 2017
parent reply Jack Stouffer <jack jackstouffer.com> writes:
On Friday, 27 January 2017 at 03:58:26 UTC, Ali Çehreli wrote:

 And this:

   http://youtu.be/-DK4r5xewTY


Hey Jon, if you're in this thread, are you able to post any of 
the code that you use for tsv parsing?
Jan 27 2017
next sibling parent reply Jon Degenhardt <jond noreply.com> writes:
On Friday, 27 January 2017 at 16:21:51 UTC, Jack Stouffer wrote:

 On Friday, 27 January 2017 at 03:58:26 UTC, Ali Çehreli wrote:

 And this:

   http://youtu.be/-DK4r5xewTY


 Hey Jon, if you're in this thread, are you able to post any of 
 the code that you use for tsv parsing?


Code has been open-sourced: 
https://github.com/eBay/tsv-utils-dlang

The performance benchmarks showed in the talk are not in the 
repo, the benchmarks currently listed are from a year ago. I'm 
planning to update the repo in the next few weeks, probably after 
the next LDC release.

If there are questions about specific types of things perhaps a 
thread in General forum would work.

--Jon
Jan 27 2017
parent reply Joakim <dlang joakim.fea.st> writes:
On Friday, 27 January 2017 at 18:20:53 UTC, Jon Degenhardt wrote:

 On Friday, 27 January 2017 at 16:21:51 UTC, Jack Stouffer wrote:

 On Friday, 27 January 2017 at 03:58:26 UTC, Ali Çehreli wrote:

 And this:

   http://youtu.be/-DK4r5xewTY


 Hey Jon, if you're in this thread, are you able to post any of 
 the code that you use for tsv parsing?


 Code has been open-sourced: 
 https://github.com/eBay/tsv-utils-dlang

 The performance benchmarks showed in the talk are not in the 
 repo, the benchmarks currently listed are from a year ago. I'm 
 planning to update the repo in the next few weeks, probably 
 after the next LDC release.

 If there are questions about specific types of things perhaps a 
 thread in General forum would work.

 --Jon


Watched the video some time back, interesting results.  Any plans 
to blog about this?  It would be great if you could run them 
through a profiler too, see why D is so much faster.  Would be 
really worth writing this up, maybe on the D blog.
Feb 17 2017
parent reply Jon Degenhardt <jond noreply.com> writes:
On Saturday, 18 February 2017 at 07:50:02 UTC, Joakim wrote:

 On Friday, 27 January 2017 at 18:20:53 UTC, Jon Degenhardt 
 wrote:

 On Friday, 27 January 2017 at 16:21:51 UTC, Jack Stouffer 
 wrote:

 On Friday, 27 January 2017 at 03:58:26 UTC, Ali Çehreli wrote:

 And this:

   http://youtu.be/-DK4r5xewTY


 Hey Jon, if you're in this thread, are you able to post any 
 of the code that you use for tsv parsing?


 Code has been open-sourced: 
 https://github.com/eBay/tsv-utils-dlang

 The performance benchmarks showed in the talk are not in the 
 repo, the benchmarks currently listed are from a year ago. I'm 
 planning to update the repo in the next few weeks, probably 
 after the next LDC release.

 If there are questions about specific types of things perhaps 
 a thread in General forum would work.

 --Jon


 Watched the video some time back, interesting results.  Any 
 plans to blog about this?  It would be great if you could run 
 them through a profiler too, see why D is so much faster.  
 Would be really worth writing this up, maybe on the D blog.


Thanks for the feedback. I'm pretty close to publishing the 
benchmarks, they'll go in a doc file in the repository. They 
weren't quite complete when the meetup happened.

Regarding a blog post - I haven't talked to Mike Parker, if 
there's interest I'd be open to it.

As to why the tools compare so well - That's a really intriguing 
question, especially since the tools favor using high level 
constructs from D / Phobos rather than hand-built data structures 
or memory management. I have hypotheses, but no sure answers. 
Some of it likely involves design choices rather than language 
facilities per se, but even so, it's a good story for D.

--Jon
Feb 18 2017
parent Mike Parker <aldacron gmail.com> writes:
On Saturday, 18 February 2017 at 21:22:33 UTC, Jon Degenhardt 
wrote:


 Regarding a blog post - I haven't talked to Mike Parker, if 
 there's interest I'd be open to it.


Yes, there is interest. I was planning to contact you about this 
in my next round of queries. So I suppose now we can skip the 
query and get right to the details.
Feb 18 2017
prev sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 01/27/2017 08:21 AM, Jack Stouffer wrote:

 On Friday, 27 January 2017 at 03:58:26 UTC, Ali Çehreli wrote:

 And this:

   http://youtu.be/-DK4r5xewTY


 Hey Jon, if you're in this thread, are you able to post any of the code
 that you use for tsv parsing?


Yeah, the slide starting at 19'35 is the most interesting:

   https://www.youtube.com/watch?v=-DK4r5xewTY&feature=youtu.be&t=1175

Tools written in D (mostly with Phobos and with GC) are at least 3 times 
faster! Let's verify the results and then make some noise. :)

Ali
Jan 27 2017
parent Jon Degenhardt <jond noreply.com> writes:
On Friday, 27 January 2017 at 20:48:30 UTC, Ali Çehreli wrote:

 On 01/27/2017 08:21 AM, Jack Stouffer wrote:

 On Friday, 27 January 2017 at 03:58:26 UTC, Ali Çehreli wrote:

 And this:

   http://youtu.be/-DK4r5xewTY


 Hey Jon, if you're in this thread, are you able to post any of 
 the code
 that you use for tsv parsing?


 Yeah, the slide starting at 19'35 is the most interesting:

   
 https://www.youtube.com/watch?v=-DK4r5xewTY&feature=youtu.be&t=1175

 Tools written in D (mostly with Phobos and with GC) are at 
 least 3 times faster! Let's verify the results and then make 
 some noise. :)

 Ali


An independent verification of the results would be fantastic. 
Any time a single person does this type of benchmark, especially 
the author of the tool, there's real risk of an error. In this 
case I took every reasonable step I knew to be diligent about it, 
but still. And yes, the deltas are impressive. I was surprised.
Jan 27 2017