www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - Silicon Valley D Meetup - January 26, 2017 - "High Performance Tools

reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
Our long time member and friend Jon Degenhardt has graciously accepted 
to present on very short notice.

He will give a preview of some performance benchmarks that he has been 
running of tools in C, Rust, and Go that overlap with what Jon writes in 
D at eBay. Jon has been observing that D versions are faster in nearly 
all cases, and not by a small margin. In a total surprise, Jon's version 
of ‘cut’ is faster than GNU cut on large files: On a 4.8GB, 7M lines 
test file, GNU cut takes 12.4 sec, while Jon's version takes 4.2 sec. 
(GNU cut is faster on small files.)

The big take-away is that this was achieved without a lot of low-level 
coding, using mostly high level D primitives and the standard library. 
There was some tuning and lessons learned, but nothing extensive.

 
https://www.meetup.com/D-Lang-Silicon-Valley/events/236421472/?eventId=236421472

I may post a link to Google Hangouts here at the time of the event (7pm 
Pacific time). Hopefully, the mic will not be muted. (True story! :p)

Ali
Jan 25
next sibling parent Adil Baig via Digitalmars-d-announce writes:
Can't wait! Please ask Jon to write something up on it. For posterity

On Thu, Jan 26, 2017 at 1:23 PM, Ali =C3=87ehreli via Digitalmars-d-announc=
e <
digitalmars-d-announce puremagic.com> wrote:

 Our long time member and friend Jon Degenhardt has graciously accepted to
 present on very short notice.

 He will give a preview of some performance benchmarks that he has been
 running of tools in C, Rust, and Go that overlap with what Jon writes in =
D
 at eBay. Jon has been observing that D versions are faster in nearly all
 cases, and not by a small margin. In a total surprise, Jon's version of
 =E2=80=98cut=E2=80=99 is faster than GNU cut on large files: On a 4.8GB, =
7M lines test
 file, GNU cut takes 12.4 sec, while Jon's version takes 4.2 sec. (GNU cut
 is faster on small files.)

 The big take-away is that this was achieved without a lot of low-level
 coding, using mostly high level D primitives and the standard library.
 There was some tuning and lessons learned, but nothing extensive.


 https://www.meetup.com/D-Lang-Silicon-Valley/events/23642147
 2/?eventId=3D236421472

 I may post a link to Google Hangouts here at the time of the event (7pm
 Pacific time). Hopefully, the mic will not be muted. (True story! :p)

 Ali
Jan 26
prev sibling next sibling parent reply Jack Stouffer <jack jackstouffer.com> writes:
On Thursday, 26 January 2017 at 07:53:22 UTC, Ali Çehreli wrote:
 I may post a link to Google Hangouts here at the time of the 
 event (7pm Pacific time). Hopefully, the mic will not be muted. 
 (True story! :p)

 Ali
If it's in hangouts, you can use Hangouts On Air to stream it to YouTube. D could always use more content on YouTube, as Rust and Go swamp us on this point.
Jan 26
parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 01/26/2017 08:59 AM, Jack Stouffer wrote:
 On Thursday, 26 January 2017 at 07:53:22 UTC, Ali Çehreli wrote:
 I may post a link to Google Hangouts here at the time of the event
 (7pm Pacific time). Hopefully, the mic will not be muted. (True story!
 :p)

 Ali
If it's in hangouts, you can use Hangouts On Air to stream it to YouTube. D could always use more content on YouTube, as Rust and Go swamp us on this point.
Yes, On Air is the idea. Ali
Jan 26
prev sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
We're live now:

 
https://hangouts.google.com/hangouts/_/ytl/QZmU0_QJnXuKGdftK-0D7QT4QbVn09_YyOMFHXoNDt8=?hl=en_US&authuser=0

Ali

On 01/25/2017 11:53 PM, Ali Çehreli wrote:
 Our long time member and friend Jon Degenhardt has graciously accepted
 to present on very short notice.

 He will give a preview of some performance benchmarks that he has been
 running of tools in C, Rust, and Go that overlap with what Jon writes in
 D at eBay. Jon has been observing that D versions are faster in nearly
 all cases, and not by a small margin. In a total surprise, Jon's version
 of ‘cut’ is faster than GNU cut on large files: On a 4.8GB, 7M lines
 test file, GNU cut takes 12.4 sec, while Jon's version takes 4.2 sec.
 (GNU cut is faster on small files.)

 The big take-away is that this was achieved without a lot of low-level
 coding, using mostly high level D primitives and the standard library.
 There was some tuning and lessons learned, but nothing extensive.


 https://www.meetup.com/D-Lang-Silicon-Valley/events/236421472/?eventId=236421472


 I may post a link to Google Hangouts here at the time of the event (7pm
 Pacific time). Hopefully, the mic will not be muted. (True story! :p)

 Ali
Jan 26
parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
And this:

   http://youtu.be/-DK4r5xewTY

Ali

On 01/26/2017 07:26 PM, Ali Çehreli wrote:
 We're live now:


 https://hangouts.google.com/hangouts/_/ytl/QZmU0_QJnXuKGdftK-0D7QT4QbVn09_YyOMFHXoNDt8=?hl=en_US&authuser=0


 Ali

 On 01/25/2017 11:53 PM, Ali Çehreli wrote:
 Our long time member and friend Jon Degenhardt has graciously accepted
 to present on very short notice.

 He will give a preview of some performance benchmarks that he has been
 running of tools in C, Rust, and Go that overlap with what Jon writes in
 D at eBay. Jon has been observing that D versions are faster in nearly
 all cases, and not by a small margin. In a total surprise, Jon's version
 of ‘cut’ is faster than GNU cut on large files: On a 4.8GB, 7M lines
 test file, GNU cut takes 12.4 sec, while Jon's version takes 4.2 sec.
 (GNU cut is faster on small files.)

 The big take-away is that this was achieved without a lot of low-level
 coding, using mostly high level D primitives and the standard library.
 There was some tuning and lessons learned, but nothing extensive.


 https://www.meetup.com/D-Lang-Silicon-Valley/events/236421472/?eventId=236421472



 I may post a link to Google Hangouts here at the time of the event (7pm
 Pacific time). Hopefully, the mic will not be muted. (True story! :p)

 Ali
Jan 26
parent reply Jack Stouffer <jack jackstouffer.com> writes:
On Friday, 27 January 2017 at 03:58:26 UTC, Ali Çehreli wrote:
 And this:

   http://youtu.be/-DK4r5xewTY
Hey Jon, if you're in this thread, are you able to post any of the code that you use for tsv parsing?
Jan 27
next sibling parent reply Jon Degenhardt <jond noreply.com> writes:
On Friday, 27 January 2017 at 16:21:51 UTC, Jack Stouffer wrote:
 On Friday, 27 January 2017 at 03:58:26 UTC, Ali Çehreli wrote:
 And this:

   http://youtu.be/-DK4r5xewTY
Hey Jon, if you're in this thread, are you able to post any of the code that you use for tsv parsing?
Code has been open-sourced: https://github.com/eBay/tsv-utils-dlang The performance benchmarks showed in the talk are not in the repo, the benchmarks currently listed are from a year ago. I'm planning to update the repo in the next few weeks, probably after the next LDC release. If there are questions about specific types of things perhaps a thread in General forum would work. --Jon
Jan 27
parent reply Joakim <dlang joakim.fea.st> writes:
On Friday, 27 January 2017 at 18:20:53 UTC, Jon Degenhardt wrote:
 On Friday, 27 January 2017 at 16:21:51 UTC, Jack Stouffer wrote:
 On Friday, 27 January 2017 at 03:58:26 UTC, Ali Çehreli wrote:
 And this:

   http://youtu.be/-DK4r5xewTY
Hey Jon, if you're in this thread, are you able to post any of the code that you use for tsv parsing?
Code has been open-sourced: https://github.com/eBay/tsv-utils-dlang The performance benchmarks showed in the talk are not in the repo, the benchmarks currently listed are from a year ago. I'm planning to update the repo in the next few weeks, probably after the next LDC release. If there are questions about specific types of things perhaps a thread in General forum would work. --Jon
Watched the video some time back, interesting results. Any plans to blog about this? It would be great if you could run them through a profiler too, see why D is so much faster. Would be really worth writing this up, maybe on the D blog.
Feb 17
parent reply Jon Degenhardt <jond noreply.com> writes:
On Saturday, 18 February 2017 at 07:50:02 UTC, Joakim wrote:
 On Friday, 27 January 2017 at 18:20:53 UTC, Jon Degenhardt 
 wrote:
 On Friday, 27 January 2017 at 16:21:51 UTC, Jack Stouffer 
 wrote:
 On Friday, 27 January 2017 at 03:58:26 UTC, Ali Çehreli wrote:
 And this:

   http://youtu.be/-DK4r5xewTY
Hey Jon, if you're in this thread, are you able to post any of the code that you use for tsv parsing?
Code has been open-sourced: https://github.com/eBay/tsv-utils-dlang The performance benchmarks showed in the talk are not in the repo, the benchmarks currently listed are from a year ago. I'm planning to update the repo in the next few weeks, probably after the next LDC release. If there are questions about specific types of things perhaps a thread in General forum would work. --Jon
Watched the video some time back, interesting results. Any plans to blog about this? It would be great if you could run them through a profiler too, see why D is so much faster. Would be really worth writing this up, maybe on the D blog.
Thanks for the feedback. I'm pretty close to publishing the benchmarks, they'll go in a doc file in the repository. They weren't quite complete when the meetup happened. Regarding a blog post - I haven't talked to Mike Parker, if there's interest I'd be open to it. As to why the tools compare so well - That's a really intriguing question, especially since the tools favor using high level constructs from D / Phobos rather than hand-built data structures or memory management. I have hypotheses, but no sure answers. Some of it likely involves design choices rather than language facilities per se, but even so, it's a good story for D. --Jon
Feb 18
parent Mike Parker <aldacron gmail.com> writes:
On Saturday, 18 February 2017 at 21:22:33 UTC, Jon Degenhardt 
wrote:

 Regarding a blog post - I haven't talked to Mike Parker, if 
 there's interest I'd be open to it.
Yes, there is interest. I was planning to contact you about this in my next round of queries. So I suppose now we can skip the query and get right to the details.
Feb 18
prev sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 01/27/2017 08:21 AM, Jack Stouffer wrote:
 On Friday, 27 January 2017 at 03:58:26 UTC, Ali Çehreli wrote:
 And this:

   http://youtu.be/-DK4r5xewTY
Hey Jon, if you're in this thread, are you able to post any of the code that you use for tsv parsing?
Yeah, the slide starting at 19'35 is the most interesting: https://www.youtube.com/watch?v=-DK4r5xewTY&feature=youtu.be&t=1175 Tools written in D (mostly with Phobos and with GC) are at least 3 times faster! Let's verify the results and then make some noise. :) Ali
Jan 27
parent Jon Degenhardt <jond noreply.com> writes:
On Friday, 27 January 2017 at 20:48:30 UTC, Ali Çehreli wrote:
 On 01/27/2017 08:21 AM, Jack Stouffer wrote:
 On Friday, 27 January 2017 at 03:58:26 UTC, Ali Çehreli wrote:
 And this:

   http://youtu.be/-DK4r5xewTY
Hey Jon, if you're in this thread, are you able to post any of the code that you use for tsv parsing?
Yeah, the slide starting at 19'35 is the most interesting: https://www.youtube.com/watch?v=-DK4r5xewTY&feature=youtu.be&t=1175 Tools written in D (mostly with Phobos and with GC) are at least 3 times faster! Let's verify the results and then make some noise. :) Ali
An independent verification of the results would be fantastic. Any time a single person does this type of benchmark, especially the author of the tool, there's real risk of an error. In this case I took every reasonable step I knew to be diligent about it, but still. And yes, the deltas are impressive. I was surprised.
Jan 27