www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - MIT Technology Review: An Interview With Bjarne Stroustrup

reply Pragma <ericanderton yahoo.removeme.com> writes:
Something I ran into that the group might enjoy:

From: http://www.techreview.com/InfoTech/17831/

"Bjarne Stroustrup, the inventor of the C++ programming language, 
defends his legacy and examines what's wrong with most software code."

As always, Slashdot has some colorful coverage on this:

http://it.slashdot.org/it/06/12/05/0045234.shtml

"MIT's Technology Review has a Q&A with C++ inventor Bjarne Stroustrup. 
Highlights include Bjarne's answers on the trade-offs involved in the 
design of C++, and how they apply today, and his thoughts on the 
solution to the problems. From the interview: 'Software developers have 
become adept at the difficult art of building reasonably reliable 
systems out of unreliable parts. The snag is that often we do not know 
exactly how we did it.'"

-- 
- EricAnderton at yahoo
Dec 05 2006
next sibling parent reply "John Reimer" <terminal.node gmail.com> writes:
On Tue, 05 Dec 2006 08:55:09 -0800, Pragma  
<ericanderton yahoo.removeme.com> wrote:

 Something I ran into that the group might enjoy:

 From: http://www.techreview.com/InfoTech/17831/

Off-topic observation: I love the way the threading works in the comments section of this article! You just click on the comment and it immediatly shows the comment text. Clicking it again collapses it. You can do that for each comment. Now that's a great system! To bad the D newsgroup couldn't do something like that for its web interface. -JJR
Dec 05 2006
next sibling parent Sean Kelly <sean f4.ca> writes:
John Reimer wrote:
 On Tue, 05 Dec 2006 08:55:09 -0800, Pragma 
 <ericanderton yahoo.removeme.com> wrote:
 
 Something I ran into that the group might enjoy:

 From: http://www.techreview.com/InfoTech/17831/

Off-topic observation: I love the way the threading works in the comments section of this article! You just click on the comment and it immediatly shows the comment text. Clicking it again collapses it. You can do that for each comment. Now that's a great system! To bad the D newsgroup couldn't do something like that for its web interface.

The new Slashdot interface does the same thing. I love it. But I've yet to see a web-based newsreader that does this. I suppose it would involve the use of all sorts of acronyms I know very little about. Sean
Dec 05 2006
prev sibling next sibling parent reply Pragma <ericanderton yahoo.removeme.com> writes:
John Reimer wrote:
 On Tue, 05 Dec 2006 08:55:09 -0800, Pragma 
 <ericanderton yahoo.removeme.com> wrote:
 
 Something I ran into that the group might enjoy:

 From: http://www.techreview.com/InfoTech/17831/

Off-topic observation: I love the way the threading works in the comments section of this article! You just click on the comment and it immediatly shows the comment text. Clicking it again collapses it. You can do that for each comment. Now that's a great system! To bad the D newsgroup couldn't do something like that for its web interface. -JJR

It's a nice concept, but (IMO) it makes for slow reading if you want to digest an entire thread on-page. I'd much rather see the posts in full, but have the tree kept for expanding and collapsing threads as I go. Kind of like a compromise between a TreeView and Slashdot I suppose. Now, if you want to see something *really slick* for commenting, take a look at JackSlocum's site: http://www.jackslocum.com/blog/2006/10/09/my-wordpress-comments-system-built-with-yahoo-ui-and-yahooext/ It's pretty script-heavy, but it shows what can be done with AJAX and a solid widget set. Be sure to check out the "Documentation" section too. -- - EricAnderton at yahoo
Dec 05 2006
next sibling parent reply Brad Anderson <brad dsource.org> writes:
Pragma wrote:
 John Reimer wrote:
 On Tue, 05 Dec 2006 08:55:09 -0800, Pragma
 <ericanderton yahoo.removeme.com> wrote:

 Something I ran into that the group might enjoy:

 From: http://www.techreview.com/InfoTech/17831/

Off-topic observation: I love the way the threading works in the comments section of this article! You just click on the comment and it immediatly shows the comment text. Clicking it again collapses it. You can do that for each comment. Now that's a great system! To bad the D newsgroup couldn't do something like that for its web interface. -JJR

It's a nice concept, but (IMO) it makes for slow reading if you want to digest an entire thread on-page. I'd much rather see the posts in full, but have the tree kept for expanding and collapsing threads as I go. Kind of like a compromise between a TreeView and Slashdot I suppose. Now, if you want to see something *really slick* for commenting, take a look at JackSlocum's site: http://www.jackslocum.com/blog/2006/10/09/my-wordpress-comments-system-built-with-yahoo-ui-and-yahooext/ It's pretty script-heavy, but it shows what can be done with AJAX and a solid widget set. Be sure to check out the "Documentation" section too.

That site crushed my browser. I also got the FF2 "runaway script, debug or stop" dialog. A second visit once everything would theoretically be cached was no better. The google pageads were pretty slow, but things were crushed enough that even Firebug didn't respond very well. So I couldn't tell if it was a bunch of Ajax calls or what was running so slowly. BA
Dec 05 2006
next sibling parent Pragma <ericanderton yahoo.removeme.com> writes:
Brad Anderson wrote:
 
 That site crushed my browser.  I also got the FF2 "runaway script, debug or
 stop" dialog.
 
 A second visit once everything would theoretically be cached was no better.
 The google pageads were pretty slow, but things were crushed enough that even
 Firebug didn't respond very well.  So I couldn't tell if it was a bunch of
 Ajax calls or what was running so slowly.

Sorry about that. I've never had that problem on that site. :( But then again, the YUI plus Jack's extensions are nothing trivial to load and run. It's a pretty big library. -- - EricAnderton at yahoo
Dec 05 2006
prev sibling parent Lars Ivar Igesund <larsivar igesund.net> writes:
Brad Anderson wrote:

 Pragma wrote:
 John Reimer wrote:
 On Tue, 05 Dec 2006 08:55:09 -0800, Pragma
 <ericanderton yahoo.removeme.com> wrote:

 Something I ran into that the group might enjoy:

 From: http://www.techreview.com/InfoTech/17831/

Off-topic observation: I love the way the threading works in the comments section of this article! You just click on the comment and it immediatly shows the comment text. Clicking it again collapses it. You can do that for each comment. Now that's a great system! To bad the D newsgroup couldn't do something like that for its web interface. -JJR

It's a nice concept, but (IMO) it makes for slow reading if you want to digest an entire thread on-page. I'd much rather see the posts in full, but have the tree kept for expanding and collapsing threads as I go. Kind of like a compromise between a TreeView and Slashdot I suppose. Now, if you want to see something *really slick* for commenting, take a look at JackSlocum's site:


 
 
 It's pretty script-heavy, but it shows what can be done with AJAX and a
 solid widget set.  Be sure to check out the "Documentation" section too.
 

That site crushed my browser. I also got the FF2 "runaway script, debug or stop" dialog. A second visit once everything would theoretically be cached was no better. The google pageads were pretty slow, but things were crushed enough that even Firebug didn't respond very well. So I couldn't tell if it was a bunch of Ajax calls or what was running so slowly. BA

It works in mine FF2, but Konqueror only shows the background color, and says nothing about errors. Personally I would never bother to read that page, it was way too busy, I almost thought it was badly rendered at first. -- Lars Ivar Igesund blog at http://larsivi.net DSource & #D: larsivi
Dec 05 2006
prev sibling parent reply Hasan Aljudy <hasan.aljudy gmail.com> writes:
Pragma wrote:
 John Reimer wrote:
 On Tue, 05 Dec 2006 08:55:09 -0800, Pragma 
 <ericanderton yahoo.removeme.com> wrote:

 Something I ran into that the group might enjoy:

 From: http://www.techreview.com/InfoTech/17831/

Off-topic observation: I love the way the threading works in the comments section of this article! You just click on the comment and it immediatly shows the comment text. Clicking it again collapses it. You can do that for each comment. Now that's a great system! To bad the D newsgroup couldn't do something like that for its web interface. -JJR

It's a nice concept, but (IMO) it makes for slow reading if you want to digest an entire thread on-page. I'd much rather see the posts in full, but have the tree kept for expanding and collapsing threads as I go. Kind of like a compromise between a TreeView and Slashdot I suppose. Now, if you want to see something *really slick* for commenting, take a look at JackSlocum's site: http://www.jackslocum.com/blog/2006/10/09/my-wordpress-comments-system-built-with-ya oo-ui-and-yahooext/ It's pretty script-heavy, but it shows what can be done with AJAX and a solid widget set. Be sure to check out the "Documentation" section too.

That baby is too slow ... my FF2 had to suffer before opening it ..
Dec 05 2006
parent reply =?ISO-8859-15?Q?Julio_C=E9sar_Carrascal_Urquijo?= writes:
Hasan Aljudy wrote:
 Pragma wrote:
 John Reimer wrote:

 Now, if you want to see something *really slick* for commenting, take 
 a look at JackSlocum's site:

 http://www.jackslocum.com/blog/2006/10/09/my-wordpress-comments-system-built-with-ya
oo-ui-and-yahooext/ 


 It's pretty script-heavy, but it shows what can be done with AJAX and 
 a solid widget set.  Be sure to check out the "Documentation" section 
 too.

That baby is too slow ... my FF2 had to suffer before opening it ..

The Django book has the same commenting system but it's a lot faster: http://www.djangobook.com/en/beta/chapter14/
Dec 06 2006
next sibling parent Lars Ivar Igesund <larsivar igesund.net> writes:
Julio C├ęsar Carrascal Urquijo wrote:

 Hasan Aljudy wrote:
 Pragma wrote:
 John Reimer wrote:


 Now, if you want to see something *really slick* for commenting, take
 a look at JackSlocum's site:



 It's pretty script-heavy, but it shows what can be done with AJAX and
 a solid widget set.  Be sure to check out the "Documentation" section
 too.

That baby is too slow ... my FF2 had to suffer before opening it ..

The Django book has the same commenting system but it's a lot faster: http://www.djangobook.com/en/beta/chapter14/

This one also worked fine with Konqueror, that other link must do something weird then. -- Lars Ivar Igesund blog at http://larsivi.net DSource & #D: larsivi
Dec 06 2006
prev sibling next sibling parent reply Brad Anderson <brad dsource.org> writes:
Julio CÚsar Carrascal Urquijo wrote:
 Hasan Aljudy wrote:
 Pragma wrote:
 John Reimer wrote:

 Now, if you want to see something *really slick* for commenting, take
 a look at JackSlocum's site:

 http://www.jackslocum.com/blog/2006/10/09/my-wordpress-comments-system-built-with-yahoo-ui-and-yahooext/


 It's pretty script-heavy, but it shows what can be done with AJAX and
 a solid widget set.  Be sure to check out the "Documentation" section
 too.

That baby is too slow ... my FF2 had to suffer before opening it ..

The Django book has the same commenting system but it's a lot faster: http://www.djangobook.com/en/beta/chapter14/

Wow, that is *really slick*
Dec 06 2006
next sibling parent Bill Baxter <wbaxter gmail.com> writes:
Brad Anderson wrote:
 Julio CÚsar Carrascal Urquijo wrote:
 
Hasan Aljudy wrote:

Pragma wrote:

John Reimer wrote:

Now, if you want to see something *really slick* for commenting, take
a look at JackSlocum's site:

http://www.jackslocum.com/blog/2006/10/09/my-wordpress-comments-system-built-with-yahoo-ui-and-yahooext/


It's pretty script-heavy, but it shows what can be done with AJAX and
a solid widget set.  Be sure to check out the "Documentation" section
too.

That baby is too slow ... my FF2 had to suffer before opening it ..

The Django book has the same commenting system but it's a lot faster: http://www.djangobook.com/en/beta/chapter14/

Wow, that is *really slick*

That's exactly what's needed for the D spec pages. The "Comments" link is nice, but usually the comments one wants to make are like "the *foo in this code should be just foo". And many people don't even realize the Comments page is there as it is. Too easy to miss. --bb
Dec 06 2006
prev sibling parent Pragma <ericanderton yahoo.removeme.com> writes:
Brad Anderson wrote:
 Julio CÚsar Carrascal Urquijo wrote:
 Hasan Aljudy wrote:
 Pragma wrote:
 John Reimer wrote:

 Now, if you want to see something *really slick* for commenting, take
 a look at JackSlocum's site:

 http://www.jackslocum.com/blog/2006/10/09/my-wordpress-comments-system-built-with-yahoo-ui-and-yahooext/


 It's pretty script-heavy, but it shows what can be done with AJAX and
 a solid widget set.  Be sure to check out the "Documentation" section
 too.


http://www.djangobook.com/en/beta/chapter14/

Wow, that is *really slick*

Amazingly, it's using the same codebase: YUI-ext! - Eric -- - EricAnderton at yahoo
Dec 07 2006
prev sibling parent reply Hasan Aljudy <hasan.aljudy gmail.com> writes:
Julio CÚsar Carrascal Urquijo wrote:
 Hasan Aljudy wrote:
 Pragma wrote:
 John Reimer wrote:


 Now, if you want to see something *really slick* for commenting, take 
 a look at JackSlocum's site:

 http://www.jackslocum.com/blog/2006/10/09/my-wordpress-comments-system-built-with-ya
oo-ui-and-yahooext/ 


 It's pretty script-heavy, but it shows what can be done with AJAX and 
 a solid widget set.  Be sure to check out the "Documentation" section 
 too.

That baby is too slow ... my FF2 had to suffer before opening it ..

The Django book has the same commenting system but it's a lot faster: http://www.djangobook.com/en/beta/chapter14/

Wow! Amazing indeed! Commenting on a specific block! Such a simple idea yet very original and I can assume it can prove to be very useful and convenient.
Dec 06 2006
parent Bill Baxter <dnewsgroup billbaxter.com> writes:
Hasan Aljudy wrote:
 
 
 Julio CÚsar Carrascal Urquijo wrote:
 Hasan Aljudy wrote:
 Pragma wrote:
 John Reimer wrote:


 Now, if you want to see something *really slick* for commenting, 
 take a look at JackSlocum's site:

 http://www.jackslocum.com/blog/2006/10/09/my-wordpress-comments-system-built-with-ya
oo-ui-and-yahooext/ 


 It's pretty script-heavy, but it shows what can be done with AJAX 
 and a solid widget set.  Be sure to check out the "Documentation" 
 section too.

That baby is too slow ... my FF2 had to suffer before opening it ..

The Django book has the same commenting system but it's a lot faster: http://www.djangobook.com/en/beta/chapter14/

Wow! Amazing indeed! Commenting on a specific block! Such a simple idea yet very original and I can assume it can prove to be very useful and convenient.

But I wonder what happens when the maintainers of the documentation want to make changes that move, split, merge, delete, and reorganize paragraphs. --bb
Dec 06 2006
prev sibling parent Hasan Aljudy <hasan.aljudy gmail.com> writes:
John Reimer wrote:
 On Tue, 05 Dec 2006 08:55:09 -0800, Pragma 
 <ericanderton yahoo.removeme.com> wrote:
 
 Something I ran into that the group might enjoy:

 From: http://www.techreview.com/InfoTech/17831/

Off-topic observation: I love the way the threading works in the comments section of this article! You just click on the comment and it immediatly shows the comment text. Clicking it again collapses it. You can do that for each comment. Now that's a great system! To bad the D newsgroup couldn't do something like that for its web interface. -JJR

AJAX ruulz baby!!
Dec 05 2006
prev sibling next sibling parent reply "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
"Pragma" <ericanderton yahoo.removeme.com> wrote in message 
news:el489h$1f7a$1 digitaldaemon.com...
 Something I ran into that the group might enjoy:

 From: http://www.techreview.com/InfoTech/17831/

 "Bjarne Stroustrup, the inventor of the C++ programming language, defends 
 his legacy and examines what's wrong with most software code."

Bjarne: "There are just two kinds of languages: the ones everybody complains about and the ones nobody uses." And for C++ programmers, those are the same languages. ;) When will Bjarne see that C++ is a festering homonculus of a language from which pours forth a black bile of rot and putrescence? Okay, maybe I'm being a bit hyperbolistic, but..
Dec 05 2006
next sibling parent reply clayasaurus <clayasaurus gmail.com> writes:
Jarrett Billingsley wrote:
 "Pragma" <ericanderton yahoo.removeme.com> wrote in message 
 news:el489h$1f7a$1 digitaldaemon.com...
 Something I ran into that the group might enjoy:

 From: http://www.techreview.com/InfoTech/17831/

 "Bjarne Stroustrup, the inventor of the C++ programming language, defends 
 his legacy and examines what's wrong with most software code."

Bjarne: "There are just two kinds of languages: the ones everybody complains about and the ones nobody uses." And for C++ programmers, those are the same languages. ;) When will Bjarne see that C++ is a festering homonculus of a language from which pours forth a black bile of rot and putrescence? Okay, maybe I'm being a bit hyperbolistic, but..

I like how he talks about C++ being 'expert' friendly and having 'expert' level features. Is this just a nice way of saying that the language is difficult to use? :-P ~ Clay
Dec 05 2006
parent Pragma <ericanderton yahoo.removeme.com> writes:
clayasaurus wrote:
 Jarrett Billingsley wrote:
 
 I like how he talks about C++ being 'expert' friendly and having 
 'expert' level features. Is this just a nice way of saying that the 
 language is difficult to use? :-P

That's pretty much how I read that too. -- - EricAnderton at yahoo
Dec 05 2006
prev sibling parent reply Sean Kelly <sean f4.ca> writes:
Jarrett Billingsley wrote:
 "Pragma" <ericanderton yahoo.removeme.com> wrote in message 
 news:el489h$1f7a$1 digitaldaemon.com...
 Something I ran into that the group might enjoy:

 From: http://www.techreview.com/InfoTech/17831/

 "Bjarne Stroustrup, the inventor of the C++ programming language, defends 
 his legacy and examines what's wrong with most software code."

Bjarne: "There are just two kinds of languages: the ones everybody complains about and the ones nobody uses." And for C++ programmers, those are the same languages. ;) When will Bjarne see that C++ is a festering homonculus of a language from which pours forth a black bile of rot and putrescence? Okay, maybe I'm being a bit hyperbolistic, but..

In Bjarne's defense, I think it's worth noting two things: * The success of C++ was very much dependent on its acceptance by the C community, and C compatibility was crucial. Many of the language and syntax compromises were for very legitimate reasons at the time the language was designed. * Bjarne quite unapologetically designed C++ for expert programmers. It isn't his fault that the language has become so popular that it is now being used by people who can't tell the dull side of a pointer from, well, the pointy side. There is no debating the fact that C++ has been an incredible success, and it still has very little competition in many core markets. Personally, my only real problems with the language are that its popularity has driven it to be used in projects and by people where another language would be more suitable, and the language supports such a wide array of programming styles that it is extremely difficult to maintain any kind of design coherence in large team projects. Also, its age is such that many projects contain code written before templates even existed, and so a substantial codebase follows "old style C++" which is heavy with raw pointers, casts, etc. In fact, I'm continually amazed at how little new code I see even today that is written using STL components. IMO this is a strong argument for D's built-in dynamic array support and other features, as there is no doubt in my mind that one of the major problems with C++ is that the language was standardized and in use before the library was up to snuff. By integrating these features into the core language, Walter has neatly sidestepped this problem and provided a nice, clean syntax for some of the most commonly used programming constructs. Sean
Dec 05 2006
parent reply zz <zz zz.com> writes:
Sean Kelly wrote:

 There is no debating the fact that C++ has been an incredible success, 
 and it still has very little competition in many core markets. 
 Personally, my only real problems with the language are that its 
 popularity has driven it to be used in projects and by people where 
 another language would be more suitable, and the language supports such 
 a wide array of programming styles that it is extremely difficult to 
 maintain any kind of design coherence in large team projects.  Also, its 
 age is such that many projects contain code written before templates 
 even existed, and so a substantial codebase follows "old style C++" 
 which is heavy with raw pointers, casts, etc.  In fact, I'm continually 
 amazed at how little new code I see even today that is written using STL 
 components.  IMO this is a strong argument for D's built-in dynamic 
 array support and other features, as there is no doubt in my mind that 
 one of the major problems with C++ is that the language was standardized 
 and in use before the library was up to snuff.  By integrating these 
 features into the core language, Walter has neatly sidestepped this 
 problem and provided a nice, clean syntax for some of the most commonly 
 used programming constructs.

I agree with everything you mention above but there are cases where C++ will just be a better due to performance and nothing else. I did some work in D recently that is being used at a client's place and it took about 2 hours to design and write (it's was not something that was performance critical), one of my work mates got interested in D and for the sake of curiosity we ran some test comparing D's built in arrays with boost::ptr_vector which we use quite a lot and the results were as follows: VS2003 = ~3.75 secs VS2003 with NedMalloc = ~1.35 secs DMD = ~ 7 secs We could not get DMC to compile Nedmalloc so we dropped testing DMC with c++. While he liked that language and said the he might actually use it to prototype idea's, he will not use it in production code due to the performance. Conclusions: D is great, but DMD will have to do something about it's performance for some applications. Zz
 
 Sean

Dec 05 2006
next sibling parent reply Sean Kelly <sean f4.ca> writes:
zz wrote:
 
 I did some work in D recently that is being used at a client's place and 
 it took about 2 hours to design and write (it's was not something that 
 was performance critical), one of my work mates got interested in D and 
 for the sake of curiosity we ran some test comparing D's built in arrays 
 with boost::ptr_vector which we use quite a lot and the results were as 
 follows:
 VS2003 = ~3.75 secs
 VS2003 with NedMalloc = ~1.35 secs
 DMD = ~ 7 secs
 We could not get DMC to compile Nedmalloc so we dropped testing DMC with 
 c++.
 
 While he liked that language and said the he might actually use it to 
 prototype idea's, he will not use it in production code due to the 
 performance.

How long are these tests? Could they be put online somewhere? I'd be interested in trying these out for myself and see if I can close the gap a bit. Sean
Dec 05 2006
parent zz <zz zz.com> writes:
Sean Kelly wrote:
 zz wrote:
 
 How long are these tests?  Could they be put online somewhere?  I'd be 
 interested in trying these out for myself and see if I can close the gap 
 a bit.

The test were run about 20 times with the average taken. The test was very simplistic and I'll send them to you tomorrow (I'm out of the office until tomorrow). Generally it's a structure with three strings, and it's newed filled in and pushed back into the vector 1,000,000 times (we have higher). I also did the same test in C using talloc which is a hierarchical pool based memory allocator using VS2003 and DMC and if I recall correctly these are the numbers: VS2003 = 4.0 - 4.5 VS2003 with nedmalloc = 2.0 - 2.5 DMC = 6.5 - 7.5 Zz
 
 Sean

Dec 05 2006
prev sibling next sibling parent reply Pragma <ericanderton yahoo.removeme.com> writes:
zz wrote:
 
 Conclusions:
 D is great, but DMD will have to do something about it's performance for 
 some applications.

I tend to agree. D's array allocation algorithm, as well as it's GC behavior, are great for most cases. But they can fall flat in certain cases - of course that's true of any algorithm really. :) While I'm no STL guru, I have to ask: which allocator were you using with ptr_vector? I know that with D, pre-allocating data for arrays can make a big difference if you expect to perform a lot of concatenations - especially with atomic elements like pointers or references. -- - EricAnderton at yahoo
Dec 05 2006
next sibling parent reply Sean Kelly <sean f4.ca> writes:
Pragma wrote:
 zz wrote:
 Conclusions:
 D is great, but DMD will have to do something about it's performance 
 for some applications.

I tend to agree. D's array allocation algorithm, as well as it's GC behavior, are great for most cases. But they can fall flat in certain cases - of course that's true of any algorithm really. :) While I'm no STL guru, I have to ask: which allocator were you using with ptr_vector? I know that with D, pre-allocating data for arrays can make a big difference if you expect to perform a lot of concatenations - especially with atomic elements like pointers or references.

I still find this confusing. The GC uses "power of two" sized blocks up to page size where the grow strategy kicks in, so it should already be using "double my current size" allocation behind the scenes. But I suppose I should really spend some time testing this to see if it can be improved. One slightly crazy idea that's been kicked around is to allow the user to override the default grow strategy, but that only seems useful if the default one really isn't appropriate for most situations, and I'm hoping that testing plus perhaps some tweaks to the algorithm will obviate the need for this. Sean
Dec 05 2006
next sibling parent Pragma <ericanderton yahoo.removeme.com> writes:
Sean Kelly wrote:
 Pragma wrote:
 zz wrote:
 Conclusions:
 D is great, but DMD will have to do something about it's performance 
 for some applications.

I tend to agree. D's array allocation algorithm, as well as it's GC behavior, are great for most cases. But they can fall flat in certain cases - of course that's true of any algorithm really. :) While I'm no STL guru, I have to ask: which allocator were you using with ptr_vector? I know that with D, pre-allocating data for arrays can make a big difference if you expect to perform a lot of concatenations - especially with atomic elements like pointers or references.

I still find this confusing. The GC uses "power of two" sized blocks up to page size where the grow strategy kicks in, so it should already be using "double my current size" allocation behind the scenes. But I suppose I should really spend some time testing this to see if it can be improved. One slightly crazy idea that's been kicked around is to allow the user to override the default grow strategy, but that only seems useful if the default one really isn't appropriate for most situations, and I'm hoping that testing plus perhaps some tweaks to the algorithm will obviate the need for this.

Sean, I'm pretty sure you've seen this before. But just in case: http://svn.dsource.org/projects/ddl/trunk/ddl/ExpContainer.d It might give you (or others) a head-start along that path. -- - EricAnderton at yahoo
Dec 05 2006
prev sibling parent Dave <Dave_member pathlink.com> writes:
Sean Kelly wrote:
 Pragma wrote:
 zz wrote:
 Conclusions:
 D is great, but DMD will have to do something about it's performance 
 for some applications.

I tend to agree. D's array allocation algorithm, as well as it's GC behavior, are great for most cases. But they can fall flat in certain cases - of course that's true of any algorithm really. :) While I'm no STL guru, I have to ask: which allocator were you using with ptr_vector? I know that with D, pre-allocating data for arrays can make a big difference if you expect to perform a lot of concatenations - especially with atomic elements like pointers or references.

I still find this confusing. The GC uses "power of two" sized blocks up to page size where the grow strategy kicks in, so it should already be using "double my current size" allocation behind the scenes. But I suppose I should really spend some time testing this to see if it can be improved. One slightly crazy idea that's been kicked around is to allow the user to override the default grow strategy, but that only seems useful if the default one really isn't appropriate for most situations, and I'm hoping that testing plus perhaps some tweaks to the algorithm will obviate the need for this.

If concatenating to the vector is the bottleneck, they may have been concatenating to the D 'vector' with arr = arr ~ elem; instead of arr ~= elem;? In that case it wouldn't be preallocating, but creating a bunch of temporaries. Just a guess. If that's the case, wouldn't if be pretty 'easy' and safe to optimize the built-in?
 
 Sean

Dec 05 2006
prev sibling parent reply zz <zz zz.com> writes:
Pragma wrote:
 zz wrote:
 
 
 While I'm no STL guru, I have to ask: which allocator were you using 
 with ptr_vector?
 

With the other test we used nedmalloc as the allocator (just over load new and delete in C++ (it's a c library). http://www.nedprod.com/programs/portable/nedmalloc/ Zz
Dec 05 2006
next sibling parent Pragma <ericanderton yahoo.removeme.com> writes:
zz wrote:
 Pragma wrote:
 zz wrote:


 While I'm no STL guru, I have to ask: which allocator were you using 
 with ptr_vector?

With the other test we used nedmalloc as the allocator (just over load new and delete in C++ (it's a c library). http://www.nedprod.com/programs/portable/nedmalloc/

Wow. Well that explains it then. Sean, you should seriously check this thing out. Maybe this could be adapted to be used in Ares? -- - EricAnderton at yahoo
Dec 05 2006
prev sibling next sibling parent reply Pragma <ericanderton yahoo.removeme.com> writes:
zz wrote:
 Pragma wrote:
 zz wrote:


 While I'm no STL guru, I have to ask: which allocator were you using 
 with ptr_vector?

With the other test we used nedmalloc as the allocator (just over load new and delete in C++ (it's a c library). http://www.nedprod.com/programs/portable/nedmalloc/

Wow. Well that explains it then. Sean, you should seriously check this thing out. Maybe this could be adapted to be used in Ares? [Edit:] The freshmeat article I found for nedmalloc calls it an "alternative malloc implementation for multiple threads without lock contention based on dlmalloc 2.8.3." Suddenly, the speed increase is makes a lot more sense. -- - EricAnderton at yahoo
Dec 05 2006
parent reply Sean Kelly <sean f4.ca> writes:
Pragma wrote:
 zz wrote:
 Pragma wrote:
 zz wrote:


 While I'm no STL guru, I have to ask: which allocator were you using 
 with ptr_vector?

With the other test we used nedmalloc as the allocator (just over load new and delete in C++ (it's a c library). http://www.nedprod.com/programs/portable/nedmalloc/

Wow. Well that explains it then. Sean, you should seriously check this thing out. Maybe this could be adapted to be used in Ares?

It shouldn't be too hard to create a D wrapper for it, but converting it to a garbage collector would be non-trivial. It may be best to simple offer it as an add-on package for folks who really need that extra bit of performance. Sean
Dec 10 2006
parent reply zz <zz zz.com> writes:
Sean Kelly wrote:
 It shouldn't be too hard to create a D wrapper for it, but converting it 
 to a garbage collector would be non-trivial.  It may be best to simple 
 offer it as an add-on package for folks who really need that extra bit 
 of performance.
 

nedmalloc uses compiled and the results I got from some c code were like 1:5. I wonder what would happen if dlmalloc was used as the base allocator in Ares or Phobos, would there be any improvement? (I just messed thing up when playing with phobos but will look into it again as for Ares the dsource site seems to be having problems). Zz
Dec 10 2006
parent reply Frits van Bommel <fvbommel REMwOVExCAPSs.nl> writes:
zz wrote:
 I wonder what would happen if dlmalloc was used as the base allocator in 
 Ares or Phobos, would there be any improvement? (I just messed thing up 
 when playing with phobos but will look into it again as for Ares the 
 dsource site seems to be having problems).

For Phobos, I don't think it would make much of a difference. AFAICT it doesn't use malloc at all, except for COM objects. It requests pages directly from the OS instead. So unless you use a lot of COM objects the difference will be negligible. IIRC Ares uses essentially the same GC so the same would apply.
Dec 10 2006
parent zz <zz zz.com> writes:
Frits van Bommel wrote:
 zz wrote:
 
 I wonder what would happen if dlmalloc was used as the base allocator 
 in Ares or Phobos, would there be any improvement? (I just messed 
 thing up when playing with phobos but will look into it again as for 
 Ares the dsource site seems to be having problems).

For Phobos, I don't think it would make much of a difference. AFAICT it doesn't use malloc at all, except for COM objects. It requests pages directly from the OS instead. So unless you use a lot of COM objects the difference will be negligible. IIRC Ares uses essentially the same GC so the same would apply.

Thanks for the clarification, it's a gray area for me. I noticed a few calls to std.c.stdlib.malloc and family then assumed that something done at that level may make some difference. Zz
Dec 10 2006
prev sibling parent reply Bill Baxter <dnewsgroup billbaxter.com> writes:
zz wrote:
 Pragma wrote:
 zz wrote:


 While I'm no STL guru, I have to ask: which allocator were you using 
 with ptr_vector?

With the other test we used nedmalloc as the allocator (just over load new and delete in C++ (it's a c library). http://www.nedprod.com/programs/portable/nedmalloc/ Zz

I was going to ask why he didn't compare his algorithm with dlmalloc, but I see it actually is based on dlmalloc. So I wonder what exactly the differences is between this and dlmalloc. Incidentally I know this guy from his posts on the FOX mailing lists. He's been working on this thing called Tn[1] for about 10 years. He's either insane or brilliant I can't really tell. :-) Read the description of "What is Tn" to see what I mean. [1] http://www.nedprod.com/Tn/index.html --bb
Dec 05 2006
next sibling parent zz <zz zz.com> writes:
Bill Baxter wrote:
 I was going to ask why he didn't compare his algorithm with dlmalloc, 
 but I see it actually is based on dlmalloc.  So I wonder what exactly 
 the differences is between this and dlmalloc.

nedmalloc actually uses dlmalloc underneath.
 Incidentally I know this guy from his posts on the FOX mailing lists. 
 He's been working on this thing called Tn[1] for about 10 years.  He's 
 either insane or brilliant I can't really tell.  :-) Read the 
 description of "What is Tn" to see what I mean.

It's a very thin line between the two, but we use nedmalloc internally and at least there I would have to claim that he is brilliant. Zz
 [1] http://www.nedprod.com/Tn/index.html
 
 --bb

Dec 05 2006
prev sibling next sibling parent Pragma <ericanderton yahoo.removeme.com> writes:
Bill Baxter wrote:
 zz wrote:
 Pragma wrote:
 zz wrote:


 While I'm no STL guru, I have to ask: which allocator were you using 
 with ptr_vector?

With the other test we used nedmalloc as the allocator (just over load new and delete in C++ (it's a c library). http://www.nedprod.com/programs/portable/nedmalloc/ Zz

I was going to ask why he didn't compare his algorithm with dlmalloc, but I see it actually is based on dlmalloc. So I wonder what exactly the differences is between this and dlmalloc. Incidentally I know this guy from his posts on the FOX mailing lists. He's been working on this thing called Tn[1] for about 10 years. He's either insane or brilliant I can't really tell. :-) Read the description of "What is Tn" to see what I mean. [1] http://www.nedprod.com/Tn/index.html --bb

Wow, that's pretty neat. I'm leaning more towards brilliance than insanity on this one. For a solid baseline of insanity, please visit the Time Cube: http://www.timecube.com/ (Warning: you'll go schizophrenic just *trying* to make sense that page.) -- - EricAnderton at yahoo
Dec 07 2006
prev sibling parent reply Pragma <ericanderton yahoo.removeme.com> writes:
Bill Baxter wrote:
 zz wrote:
 Pragma wrote:
 zz wrote:


 While I'm no STL guru, I have to ask: which allocator were you using 
 with ptr_vector?

With the other test we used nedmalloc as the allocator (just over load new and delete in C++ (it's a c library). http://www.nedprod.com/programs/portable/nedmalloc/ Zz

I was going to ask why he didn't compare his algorithm with dlmalloc, but I see it actually is based on dlmalloc. So I wonder what exactly the differences is between this and dlmalloc. Incidentally I know this guy from his posts on the FOX mailing lists. He's been working on this thing called Tn[1] for about 10 years. He's either insane or brilliant I can't really tell. :-) Read the description of "What is Tn" to see what I mean. [1] http://www.nedprod.com/Tn/index.html --bb

Wow, that's pretty neat. I'm leaning more towards brilliance than insanity on this one. For a solid baseline of insanity, please visit the Time Cube: http://www.timecube.com/ (Warning: you'll go schizophrenic just trying to make sense of that page.) -- - EricAnderton at yahoo
Dec 07 2006
next sibling parent =?ISO-8859-1?Q?Julio_C=E9sar_Carrascal_Urquijo?= writes:
Pragma wrote:
 For a solid baseline of insanity, please visit the Time Cube:
 
 http://www.timecube.com/
 
 (Warning: you'll go schizophrenic just trying to make sense of that page.)
 

Mmmmm. It looks like the output of a Markov chain generator.
Dec 07 2006
prev sibling parent reply "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
"Pragma" <ericanderton yahoo.removeme.com> wrote in message 
news:el9b6l$khl$3 digitaldaemon.com...

 For a solid baseline of insanity, please visit the Time Cube:

 http://www.timecube.com/

 (Warning: you'll go schizophrenic just trying to make sense of that page.)

Wow... This _has_ to have been the influence for this joke page: http://www.somethingawful.com/learning_triangle/index.htm Unless, of course, there are more pages like it, which certainly wouldn't surprise me.
Dec 07 2006
parent Sean Kelly <sean f4.ca> writes:
Jarrett Billingsley wrote:
 "Pragma" <ericanderton yahoo.removeme.com> wrote in message 
 news:el9b6l$khl$3 digitaldaemon.com...
 
 For a solid baseline of insanity, please visit the Time Cube:

 http://www.timecube.com/

 (Warning: you'll go schizophrenic just trying to make sense of that page.)

Wow... This _has_ to have been the influence for this joke page: http://www.somethingawful.com/learning_triangle/index.htm Unless, of course, there are more pages like it, which certainly wouldn't surprise me.

More than you can count. Here is a page with links to some of the classics: http://www.f4.ca/crackpot/ Sean
Dec 07 2006
prev sibling parent reply Walter Bright <newshound digitalmars.com> writes:
zz wrote:
 I agree with everything you mention above but there are cases where C++ 
 will just be a better due to performance and nothing else.
 
 I did some work in D recently that is being used at a client's place and 
 it took about 2 hours to design and write (it's was not something that 
 was performance critical), one of my work mates got interested in D and 
 for the sake of curiosity we ran some test comparing D's built in arrays 
 with boost::ptr_vector which we use quite a lot and the results were as 
 follows:
 VS2003 = ~3.75 secs
 VS2003 with NedMalloc = ~1.35 secs
 DMD = ~ 7 secs
 We could not get DMC to compile Nedmalloc so we dropped testing DMC with 
 c++.
 
 While he liked that language and said the he might actually use it to 
 prototype idea's, he will not use it in production code due to the 
 performance.
 
 Conclusions:
 D is great, but DMD will have to do something about it's performance for 
 some applications.

There are many possible explanations for this, and not all of them are because C++ has better performance. We need to see the code and the compiler switches used. Here's a case where D is substantially faster than C++: http://www.digitalmars.com/d/cppbench.html
Dec 05 2006
next sibling parent reply BCS <BCS pathilink.com> writes:
 
 Here's a case where D is substantially faster than C++: 
 http://www.digitalmars.com/d/cppbench.html

bad link
Dec 05 2006
parent zz <zz zz.com> writes:
BCS wrote:
 Here's a case where D is substantially faster than C++: 
 http://www.digitalmars.com/d/cppbench.html

bad link

Zz
Dec 05 2006
prev sibling parent reply zz <zz zz.com> writes:
Walter Bright wrote:
 
 There are many possible explanations for this, and not all of them are 
 because C++ has better performance. We need to see the code and the 
 compiler switches used.

Walter, it's not a C++ vs D issue (I preffer D) it's a compiler/library issue. I don't claim C++ has better performance, but that VS2003 does a better job than DMC/DMD when there lot of memory allocations and even better when combined with nedmalloc.
 Here's a case where D is substantially faster than C++: 
 http://www.digitalmars.com/d/cppbench.html

I'll have to run it under VS2003 or VS2005. I'll post the trivial test we did tommorow, but in another test in C using VS2003 default allocator, the same code was much faster in VS2003 under release then in DMC -o. In the test we are doing new is called 1,000,000 times and the result put into a boost::ptr_vector in c++. Zz
Dec 05 2006
parent reply "Craig Black" <cblack ara.com> writes:
I don't see why D doesn't use nedmalloc for its non-GC malloc 
implementation.  This would allow expert developers that avoid GC in certain 
situations glean the best performance possible.

-Craig

"zz" <zz zz.com> wrote in message news:el51rm$2cth$1 digitaldaemon.com...
 Walter Bright wrote:
 There are many possible explanations for this, and not all of them are 
 because C++ has better performance. We need to see the code and the 
 compiler switches used.

Walter, it's not a C++ vs D issue (I preffer D) it's a compiler/library issue. I don't claim C++ has better performance, but that VS2003 does a better job than DMC/DMD when there lot of memory allocations and even better when combined with nedmalloc.
 Here's a case where D is substantially faster than C++: 
 http://www.digitalmars.com/d/cppbench.html

I'll have to run it under VS2003 or VS2005. I'll post the trivial test we did tommorow, but in another test in C using VS2003 default allocator, the same code was much faster in VS2003 under release then in DMC -o. In the test we are doing new is called 1,000,000 times and the result put into a boost::ptr_vector in c++. Zz

Dec 06 2006
parent reply Sean Kelly <sean f4.ca> writes:
Craig Black wrote:
 I don't see why D doesn't use nedmalloc for its non-GC malloc 
 implementation.  This would allow expert developers that avoid GC in certain 
 situations glean the best performance possible.

The user could do this quite easily himself. Just create a D header module for the malloc calls and use them instead of std.c.stdlib.malloc, etc. Sean
Dec 06 2006
parent reply "Craig Black" <cblack ara.com> writes:
Perhaps.  But why not do this by default and spare everyone the headache?

-Craig

"Sean Kelly" <sean f4.ca> wrote in message 
news:el75b4$1ho3$2 digitaldaemon.com...
 Craig Black wrote:
 I don't see why D doesn't use nedmalloc for its non-GC malloc 
 implementation.  This would allow expert developers that avoid GC in 
 certain situations glean the best performance possible.

The user could do this quite easily himself. Just create a D header module for the malloc calls and use them instead of std.c.stdlib.malloc, etc. Sean

Dec 06 2006
parent reply zz <zz zz.com> writes:
I modified the C++ that was written to reflect the D code more closely:
And now DMD is slightly faster than VS2003 without nedmalloc.

Has anyone managed to compile nedmalloc under DMC?

I don't believe this is a good test, but anyway here it is.

And as I said before D is great and I'll continue using it everyday.

Zz

Results:
----------------------------------------------
VS2003:
Element count: 1000000

ContextSwitches - 122127
First level fills = 0
Second level fills = 0

ETime(   0:00:05.921 ) UTime(   0:00:05.687 ) KTime(   0:00:00.171 )
ITime(   0:00:05.296 )

----------------------------------------------
VS2003 With nedmalloc:
Element count: 1000000

ContextSwitches - 52979
First level fills = 0
Second level fills = 0

ETime(   0:00:02.593 ) UTime(   0:00:02.562 ) KTime(   0:00:00.062 )
ITime(   0:00:02.328 )

----------------------------------------------
DMD:
Element count: 1000000

ContextSwitches - 118191
First level fills = 0
Second level fills = 0

ETime(   0:00:05.281 ) UTime(   0:00:05.203 ) KTime(   0:00:00.031 )
ITime(   0:00:04.875 )

----------------------------------------------
C++ Code:
#include <iostream>
#include <string>
#include <memory>
#include <boost/ptr_container/ptr_vector.hpp>
#include "nedmalloc.h"

void* operator new(size_t sz)
{
	void* m = nedmalloc(sz);
	if(!m) puts("out of memory");
	return m;
}

void operator delete(void* m)
{
	nedfree(m);
}

class cpp_element
{
public:
	std::string Creator;
	std::string CreationDate;
	std::string Label;
};

class cpp_root
{
public:
	boost::ptr_vector<cpp_element> elements;
};


int main(int argc, char* argv[])
{
	cpp_root* _root = new cpp_root();
	std::auto_ptr<cpp_root> root(_root);

	std::string cd1 = "AAAAAAAAAAAAAAAAAAAAAAAAA";
	std::string cd2 = "BBBBBBBBBBBBBBBBBBBBBBBBB";
	std::string cd3 = "CCCCCCCCCCCCCCCCCCCCCCCCC";

	for(int i = 0; i < 1000000; i++)
	{
		cpp_element* element = new cpp_element();

		element->CreationDate = cd1 + "cd1";
		element->Creator      = cd2 + "cd2";
		element->Label        = cd3 + "cd3";

		root->elements.push_back(element);
	}

	printf("Element count: %d\n", root->elements.size());

	return 0;
}


----------------------------------------------
D Code:
module test;
import std.stdio;

class d_element
{
public:
	char[] Creator;
	char[] CreationDate;
	char[] Label;
}

class d_root
{
public:
	d_element[] elements;
}

int main()
{
	char[] cd1 = "AAAAAAAAAAAAAAAAAAAAAAAAA";
	char[] cd2 = "BBBBBBBBBBBBBBBBBBBBBBBBB";
	char[] cd3 = "CCCCCCCCCCCCCCCCCCCCCCCCC";
	
	d_root root = new d_root();
	
	for(int i = 0; i < 1000000; i++)
	{
		d_element element = new d_element();
		element.CreationDate = cd1 ~= "cd1";
		element.Creator      = cd2 ~= "cd2";
		element.Label        = cd3 ~= "cd3";
		root.elements ~= element;
	}

	printf("Element count: %d\n", root.elements.length);	
	return 0;
}
Dec 06 2006
parent reply Frits van Bommel <fvbommel REMwOVExCAPSs.nl> writes:
zz wrote:
         cpp_element* element = new cpp_element();
 
         element->CreationDate = cd1 + "cd1";
         element->Creator      = cd2 + "cd2";
         element->Label        = cd3 + "cd3";

         d_element element = new d_element();
         element.CreationDate = cd1 ~= "cd1";
         element.Creator      = cd2 ~= "cd2";
         element.Label        = cd3 ~= "cd3";

I don't think those lines are really equivalent. Hint: write the last element in the list to output, in both versions. You might notice a difference :).
Dec 06 2006
parent reply zz <zz zz.com> writes:
Frits van Bommel wrote:
 zz wrote:
 
         cpp_element* element = new cpp_element();

         element->CreationDate = cd1 + "cd1";
         element->Creator      = cd2 + "cd2";
         element->Label        = cd3 + "cd3";

<snip>
         d_element element = new d_element();
         element.CreationDate = cd1 ~= "cd1";
         element.Creator      = cd2 ~= "cd2";
         element.Label        = cd3 ~= "cd3";

I don't think those lines are really equivalent. Hint: write the last element in the list to output, in both versions. You might notice a difference :).

Ouch. That was really silly of me (Didn't notice it). It should have been element.CreationDate = cd ~ "cd1" and yes it would make a difference. What I had yesterday was using cd1.dup, didn't notice the silly mistake when making the change. After fixing this problem here are my results on a different machine. From my point of view these tests are not really nessesary on my side since I still continue using D and I belive that someday the memory stuff will be optimized. Zz ------------------------------------------------------- VS2003 Element count: 1000000 ContextSwitches - 24787 First level fills = 0 Second level fills = 0 ETime( 0:00:05.328 ) UTime( 0:00:05.046 ) KTime( 0:00:00.187 ) ITime( 0:00:04.828 ) ------------------------------------------------------- VS2003 with nedmalloc Element count: 1000000 ContextSwitches - 8854 First level fills = 0 Second level fills = 0 ETime( 0:00:02.390 ) UTime( 0:00:02.218 ) KTime( 0:00:00.078 ) ITime( 0:00:02.015 ) ------------------------------------------------------- DMC Element count: 1000000 ContextSwitches - 35008 First level fills = 0 Second level fills = 0 ETime( 0:00:07.312 ) UTime( 0:00:07.000 ) KTime( 0:00:00.031 ) ITime( 0:00:06.656 ) D change: for(int i = 0; i < 1000000; i++) { d_element element = new d_element(); element.CreationDate = cd1 ~ "cd1"; element.Creator = cd2 ~ "cd2"; element.Label = cd3 ~ "cd3"; root.elements ~= element; }
Dec 06 2006
next sibling parent Sean Kelly <sean f4.ca> writes:
zz wrote:
 
  From my point of view these tests are not really nessesary on my side 
 since I still continue using D and I belive that someday the memory 
 stuff will be optimized.

It will, but the changes will happen sooner if particular use cases are brought up as problematic. Particularly if tweaks can be made here and there to improve performance without writing an entirely new garbage collector. That said, I have considered writing a GC more targeted at concurrent programming than the current implementation (basically an improved version of Hoard with mark/sweep collection added), but doing so would take time I don't have at the moment. Maybe sometime after 1.0. Sean
Dec 06 2006
prev sibling next sibling parent Dave <Dave_member pathlink.com> writes:
zz wrote:
 
 After fixing this problem here are my results on a different machine.
 
  From my point of view these tests are not really nessesary on my side 
 since I still continue using D and I belive that someday the memory 
 stuff will be optimized.
 
 Zz
 
 -------------------------------------------------------
 VS2003
 Element count: 1000000
 
 ContextSwitches - 24787
 First level fills = 0
 Second level fills = 0
 
 ETime(   0:00:05.328 ) UTime(   0:00:05.046 ) KTime(   0:00:00.187 )
 ITime(   0:00:04.828 )
 
 -------------------------------------------------------
 VS2003 with nedmalloc
 Element count: 1000000
 
 ContextSwitches - 8854
 First level fills = 0
 Second level fills = 0
 
 ETime(   0:00:02.390 ) UTime(   0:00:02.218 ) KTime(   0:00:00.078 )
 ITime(   0:00:02.015 )
 
 -------------------------------------------------------
 DMC
 Element count: 1000000
 
 ContextSwitches - 35008
 First level fills = 0
 Second level fills = 0
 
 ETime(   0:00:07.312 ) UTime(   0:00:07.000 ) KTime(   0:00:00.031 )
 ITime(   0:00:06.656 )
 
 D change:
     for(int i = 0; i < 1000000; i++)
     {
         d_element element = new d_element();
         element.CreationDate = cd1 ~ "cd1";
         element.Creator      = cd2 ~ "cd2";
         element.Label        = cd3 ~ "cd3";
         root.elements ~= element;
     }
 
 

If the code is changed for both D and C++ to not cat the small strings: for(int i = 0; i < 1000000; i++) { d_element element = new d_element(); element.CreationDate = cd1;// ~ "cd1"; element.Creator = cd2;// ~ "cd2"; element.Label = cd3;// ~ "cd3"; root.elements ~= element; } VC++: 2.609 (using NedAlloc) DMD: 0.937 Pretty interesting. The opCatAssign seems very fast. opCat not. Zz, I don't know how closely your test mimics the bottlenecks in your production code, but maybe if you could move things like that opCat out of loops in some cases, you'd get faster code than with ptr_vector<>?
Dec 06 2006
prev sibling parent reply Walter Bright <newshound digitalmars.com> writes:
zz wrote:
  From my point of view these tests are not really nessesary on my side 
 since I still continue using D and I belive that someday the memory 
 stuff will be optimized.

One thing that is happening is that the C++ code allocates memory, it never frees it. The same with the D code. What happens with a garbage collector is it gets new chunks of memory from the operating system as it needs it. But before it does, it runs a collection cycle. So in the D version, it will be running probably several collection cycles (accomplishing nothing), while the C++ version does not. This will make the gc version slower. To get better numbers, one can add calls to gc.disable() and gc.enable() to tell the gc that there is no point to running a collection cycle for this section of code.
Dec 07 2006
next sibling parent reply zz <zz zz.com> writes:
Walter Bright wrote:
 zz wrote:
 
  From my point of view these tests are not really nessesary on my side 
 since I still continue using D and I belive that someday the memory 
 stuff will be optimized.

One thing that is happening is that the C++ code allocates memory, it never frees it. The same with the D code.

Not until it's out of scope (in this case main).
 What happens with a garbage collector is it gets new chunks of memory 
 from the operating system as it needs it. But before it does, it runs a 
 collection cycle.

Would something like dlmalloc by Doug Lea make a difference in D, someone suggested trying it out so I tested it using DMC on some C code and there was a big difference the two compiled versions (Numbers are below and I was supprised).
 So in the D version, it will be running probably several collection 
 cycles (accomplishing nothing), while the C++ version does not. This 
 will make the gc version slower.
 
 To get better numbers, one can add calls to gc.disable() and gc.enable() 
 to tell the gc that there is no point to running a collection cycle for 
 this section of code.

The above is something that, i'll not do (In DMC i can get better numbers by just changing my allocator how can I go about it in D). Zz ----------------------------------------------- - Default memory allocator dmc -o+all test.c Capacity C: 1048576 Count: 1000000 ContextSwitches - 19954 First level fills = 0 Second level fills = 0 ETime( 0:00:06.562 ) UTime( 0:00:06.015 ) KTime( 0:00:00.375 ) ITime( 0:00:03.828 ) ----------------------------------------------- dmc -DREPLACE_SYSTEM_ALLOCATOR -o+all test.c Capacity C: 1048576 Count: 1000000 ContextSwitches - 3092 First level fills = 0 Second level fills = 0 ETime( 0:00:01.250 ) UTime( 0:00:00.968 ) KTime( 0:00:00.250 ) ITime( 0:00:00.796 ) Note: I used dlmalloc Version pre-2.8.4 Wed Aug 2 14:13:56 2006 which comes with nedmalloc. The current version is 2.8.3 and ptmalloc3 comes with an earlier version of the one used above.
Dec 07 2006
next sibling parent reply Dave <Dave_member pathlink.com> writes:
zz wrote:
 Walter Bright wrote:
 zz wrote:

  From my point of view these tests are not really nessesary on my 
 side since I still continue using D and I belive that someday the 
 memory stuff will be optimized.

One thing that is happening is that the C++ code allocates memory, it never frees it. The same with the D code.

Not until it's out of scope (in this case main).

With that in mind, I wrapped your original code in a loop for 10 iterations and decreased the inner loop to 100000. C++ (NedAlloc): Total Element count: 1000000 5.438000 D: Total Element count: 1000000 1.360000 Because it's not doing all of it's allocation in one tight loop, maybe this better 'emulates' a 'typical' service or client application? Point is, have you and your coworker from the OP actually tried D in a couple of real-world application examples? Quote: "While he liked that language and said the he might actually use it to prototype idea's, he will not use it in production code due to the performance." That sounds like me two years ago (and D hasn't gotten all that much faster since then) <g> Don't take this as some sort of attack -- you bring up some very good points. And I agree that the GC performance should be looked at (and that great performance is critical), but I'm wondering if D is getting a fair shake in your shop? Even if you had to spend 10% development time to hoist some allocations out of loops with D, if you save 20% developing the first-cut you'll end up better off ;)
 What happens with a garbage collector is it gets new chunks of memory 
 from the operating system as it needs it. But before it does, it runs 
 a collection cycle.

Would something like dlmalloc by Doug Lea make a difference in D, someone suggested trying it out so I tested it using DMC on some C code and there was a big difference the two compiled versions (Numbers are below and I was supprised).
 So in the D version, it will be running probably several collection 
 cycles (accomplishing nothing), while the C++ version does not. This 
 will make the gc version slower.

 To get better numbers, one can add calls to gc.disable() and 
 gc.enable() to tell the gc that there is no point to running a 
 collection cycle for this section of code.

The above is something that, i'll not do (In DMC i can get better numbers by just changing my allocator how can I go about it in D). Zz ----------------------------------------------- - Default memory allocator dmc -o+all test.c Capacity C: 1048576 Count: 1000000 ContextSwitches - 19954 First level fills = 0 Second level fills = 0 ETime( 0:00:06.562 ) UTime( 0:00:06.015 ) KTime( 0:00:00.375 ) ITime( 0:00:03.828 ) ----------------------------------------------- dmc -DREPLACE_SYSTEM_ALLOCATOR -o+all test.c Capacity C: 1048576 Count: 1000000 ContextSwitches - 3092 First level fills = 0 Second level fills = 0 ETime( 0:00:01.250 ) UTime( 0:00:00.968 ) KTime( 0:00:00.250 ) ITime( 0:00:00.796 ) Note: I used dlmalloc Version pre-2.8.4 Wed Aug 2 14:13:56 2006 which comes with nedmalloc. The current version is 2.8.3 and ptmalloc3 comes with an earlier version of the one used above.

Dec 07 2006
parent reply zz <zz zz.com> writes:
Dave wrote:
 zz wrote:
 
 Walter Bright wrote:

 zz wrote:

  From my point of view these tests are not really nessesary on my 
 side since I still continue using D and I belive that someday the 
 memory stuff will be optimized.

One thing that is happening is that the C++ code allocates memory, it never frees it. The same with the D code.

Not until it's out of scope (in this case main).

With that in mind, I wrapped your original code in a loop for 10 iterations and decreased the inner loop to 100000. C++ (NedAlloc): Total Element count: 1000000 5.438000 D: Total Element count: 1000000 1.360000

In this case D is faster I get: C++ NedAlloc = 2.859 D = 1.421 But if you leave the original count and put in the outer loop for 10, D's performance becomes really bad. c++ (NedMalloc) = 00:26.796 D = 02:23.375
 Quote: "While he liked that language and said the he might actually use 
 it to prototype idea's, he will not use it in production code due to the 
 performance."
 
 That sounds like me two years ago (and D hasn't gotten all that much 
 faster since then) <g>

Sill uses D to prototype idea's, but this was someone else who is new.
 Don't take this as some sort of attack -- you bring up some very good 
 points. And I agree that the GC performance should be looked at (and 
 that great performance is critical), but I'm wondering if D is getting a 
 fair shake in your shop?

On my side I've used it in small stuff that is running at clients sites, We handle a lot of data that comes from mainframes and AS/400 and it you need to process the text reports from those platforms a lot of string processing needs to be done, in one case I recall we needed to convert the data (over 1 gig of raw data) into XML and I was asked by my boss to see how we can get it done and who should write it, I one sitting I did a prototype in D and that was what was sent to the client (I don't think it could have been done faster), for this case and some others we never looked at performance since: a) the client was happy just to get something. b) most but not all large jobs involving processing are run overnight. While it's only 2 of us who use D from time to time, there is a lot of respect for D at work but sometimes people put you to the test with 1 to 1 examples and from there they might make their desicions on whether to use D or not.
 Even if you had to spend 10% development time to hoist some allocations 
 out of loops with D, if you save 20% developing the first-cut you'll end 
 up better off ;)

Yes, but this was a 1 to 1 test more on the line my gun is bigger than yours. Zz
Dec 08 2006
parent reply Sean Kelly <sean f4.ca> writes:
zz wrote:
 Dave wrote:
 zz wrote:

 Walter Bright wrote:

 zz wrote:

  From my point of view these tests are not really nessesary on my 
 side since I still continue using D and I belive that someday the 
 memory stuff will be optimized.

One thing that is happening is that the C++ code allocates memory, it never frees it. The same with the D code.

Not until it's out of scope (in this case main).

With that in mind, I wrapped your original code in a loop for 10 iterations and decreased the inner loop to 100000. C++ (NedAlloc): Total Element count: 1000000 5.438000 D: Total Element count: 1000000 1.360000

In this case D is faster I get: C++ NedAlloc = 2.859 D = 1.421 But if you leave the original count and put in the outer loop for 10, D's performance becomes really bad. c++ (NedMalloc) = 00:26.796 D = 02:23.375

Try explicitly calling _gc.fullCollect() between iterations of the inner loop, and disabling the GC explicitly there. Manipulation of the GC for performance-critical areas is an important and intentional feature. Sean
Dec 08 2006
parent reply Dave <Dave_member pathlink.com> writes:
Sean Kelly wrote:
 zz wrote:
 Dave wrote:
 zz wrote:

 Walter Bright wrote:

 zz wrote:

  From my point of view these tests are not really nessesary on my 
 side since I still continue using D and I belive that someday the 
 memory stuff will be optimized.

One thing that is happening is that the C++ code allocates memory, it never frees it. The same with the D code.

Not until it's out of scope (in this case main).

With that in mind, I wrapped your original code in a loop for 10 iterations and decreased the inner loop to 100000. C++ (NedAlloc): Total Element count: 1000000 5.438000 D: Total Element count: 1000000 1.360000

In this case D is faster I get: C++ NedAlloc = 2.859 D = 1.421 But if you leave the original count and put in the outer loop for 10, D's performance becomes really bad. c++ (NedMalloc) = 00:26.796 D = 02:23.375

Try explicitly calling _gc.fullCollect() between iterations of the inner loop, and disabling the GC explicitly there. Manipulation of the GC for performance-critical areas is an important and intentional feature.

With that in mind... What if some of the GC API were built-in to the language? This could be one of those areas that would set D apart as a lower-level, performance orientated language. If a GC implementation didn't support something, it would be stubbed for portability. Likewise anything not covered explicitly by a built-in could be covered by an import. Specifically: gcFullCollect() gcGenCollect() gcDisable() gcEnable() or some such. ?
 
 Sean

Dec 08 2006
parent reply Sean Kelly <sean f4.ca> writes:
Dave wrote:
 Sean Kelly wrote:
 zz wrote:
 Dave wrote:
 zz wrote:

 Walter Bright wrote:

 zz wrote:

  From my point of view these tests are not really nessesary on my 
 side since I still continue using D and I belive that someday the 
 memory stuff will be optimized.

One thing that is happening is that the C++ code allocates memory, it never frees it. The same with the D code.

Not until it's out of scope (in this case main).

With that in mind, I wrapped your original code in a loop for 10 iterations and decreased the inner loop to 100000. C++ (NedAlloc): Total Element count: 1000000 5.438000 D: Total Element count: 1000000 1.360000

In this case D is faster I get: C++ NedAlloc = 2.859 D = 1.421 But if you leave the original count and put in the outer loop for 10, D's performance becomes really bad. c++ (NedMalloc) = 00:26.796 D = 02:23.375

Try explicitly calling _gc.fullCollect() between iterations of the inner loop, and disabling the GC explicitly there. Manipulation of the GC for performance-critical areas is an important and intentional feature.

With that in mind... What if some of the GC API were built-in to the language? This could be one of those areas that would set D apart as a lower-level, performance orientated language. If a GC implementation didn't support something, it would be stubbed for portability. Likewise anything not covered explicitly by a built-in could be covered by an import. Specifically: gcFullCollect() gcGenCollect() gcDisable() gcEnable() or some such.

Phobos already has std.gc for exactly this purpose. Or are you suggesting some of these features should be added as keywords, or perhaps automatically available, similar to the declaration of Object and Exception? Sean
Dec 08 2006
parent Dave <Dave_member pathlink.com> writes:
Sean Kelly wrote:
 Dave wrote:
 Sean Kelly wrote:
 Try explicitly calling _gc.fullCollect() between iterations of the 
 inner loop, and disabling the GC explicitly there.  Manipulation of 
 the GC for performance-critical areas is an important and intentional 
 feature.

With that in mind... What if some of the GC API were built-in to the language? This could be one of those areas that would set D apart as a lower-level, performance orientated language. If a GC implementation didn't support something, it would be stubbed for portability. Likewise anything not covered explicitly by a built-in could be covered by an import. Specifically: gcFullCollect() gcGenCollect() gcDisable() gcEnable() or some such.

Phobos already has std.gc for exactly this purpose. Or are you suggesting some of these features should be added as keywords, or perhaps automatically available, similar to the declaration of Object and Exception?

Yes, that way those optimization 'hints' would be codified into the language itself (like register in C or inline in C++). The goal would be to lower the barrier as much as possible ('encourage the use of') and give D something codified that most other languages don't have. You know, taking this one step further for convenience and safety could give us: Replace:
 gcDisable()
 gcEnable()


with a built-in for: std.gc.disable; scope(exit) std.gc.enable; usage: void foo() { scopeGcDisable; for(...){ ... } }
 
 Sean

Dec 08 2006
prev sibling parent Walter Bright <newshound digitalmars.com> writes:
zz wrote:
 Walter Bright wrote:
 zz wrote:

  From my point of view these tests are not really nessesary on my 
 side since I still continue using D and I belive that someday the 
 memory stuff will be optimized.

One thing that is happening is that the C++ code allocates memory, it never frees it. The same with the D code.

Not until it's out of scope (in this case main).

True, but it doesn't free any memory inside the loop.
 What happens with a garbage collector is it gets new chunks of memory 
 from the operating system as it needs it. But before it does, it runs 
 a collection cycle.

Would something like dlmalloc by Doug Lea make a difference in D, someone suggested trying it out so I tested it using DMC on some C code and there was a big difference the two compiled versions (Numbers are below and I was supprised).

You can call any C function from D, so if you want to explicitly manage memory using dlmalloc, that is certainly possible. D allows overriding new/delete on a per class/per struct basis, but such won't be garbage collected if you do so.
 So in the D version, it will be running probably several collection 
 cycles (accomplishing nothing), while the C++ version does not. This 
 will make the gc version slower.

 To get better numbers, one can add calls to gc.disable() and 
 gc.enable() to tell the gc that there is no point to running a 
 collection cycle for this section of code.

The above is something that, i'll not do (In DMC i can get better numbers by just changing my allocator how can I go about it in D).

I'm not sure why you wouldn't want to do it, it is much more localized in effect than swapping out global operators new/delete. The gc in D is pluggable, but nobody has written a different one to plug in yet.
Dec 07 2006
prev sibling parent Reiner Pope <xxxxxx xxx.xx> writes:
Walter Bright wrote:
 zz wrote:
  From my point of view these tests are not really nessesary on my side 
 since I still continue using D and I belive that someday the memory 
 stuff will be optimized.

One thing that is happening is that the C++ code allocates memory, it never frees it. The same with the D code. What happens with a garbage collector is it gets new chunks of memory from the operating system as it needs it. But before it does, it runs a collection cycle. So in the D version, it will be running probably several collection cycles (accomplishing nothing), while the C++ version does not. This will make the gc version slower. To get better numbers, one can add calls to gc.disable() and gc.enable() to tell the gc that there is no point to running a collection cycle for this section of code.

Has anyone looked into analysis of D code to automatically insert frees/deletes in pertinent places, so that GC is required less often? The good thing about allowing this sort of analysis is that no harm is done if you don't find anything (since GC will still guarantee it is eventually collected), but it could yield benefits in certain places where deterministic deletion is possible. Cheers, Reiner
Dec 07 2006
prev sibling parent reply clayasaurus <clayasaurus gmail.com> writes:
Pragma wrote:
 Something I ran into that the group might enjoy:
 
 From: http://www.techreview.com/InfoTech/17831/
 
 "Bjarne Stroustrup, the inventor of the C++ programming language, 
 defends his legacy and examines what's wrong with most software code."
 
 As always, Slashdot has some colorful coverage on this:
 
 http://it.slashdot.org/it/06/12/05/0045234.shtml
 
 "MIT's Technology Review has a Q&A with C++ inventor Bjarne Stroustrup. 
 Highlights include Bjarne's answers on the trade-offs involved in the 
 design of C++, and how they apply today, and his thoughts on the 
 solution to the problems. From the interview: 'Software developers have 
 become adept at the difficult art of building reasonably reliable 
 systems out of unreliable parts. The snag is that often we do not know 
 exactly how we did it.'"
 

Here is the follow up interview http://www.techreview.com/InfoTech/17868/page1/ http://developers.slashdot.org/article.pl?sid=06/12/09/220218
Dec 09 2006
parent Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
clayasaurus wrote:
 Pragma wrote:
 Something I ran into that the group might enjoy:

 From: http://www.techreview.com/InfoTech/17831/

 "Bjarne Stroustrup, the inventor of the C++ programming language, 
 defends his legacy and examines what's wrong with most software code."

 As always, Slashdot has some colorful coverage on this:

 http://it.slashdot.org/it/06/12/05/0045234.shtml

 "MIT's Technology Review has a Q&A with C++ inventor Bjarne 
 Stroustrup. Highlights include Bjarne's answers on the trade-offs 
 involved in the design of C++, and how they apply today, and his 
 thoughts on the solution to the problems. From the interview: 
 'Software developers have become adept at the difficult art of 
 building reasonably reliable systems out of unreliable parts. The snag 
 is that often we do not know exactly how we did it.'"

Here is the follow up interview http://www.techreview.com/InfoTech/17868/page1/ http://developers.slashdot.org/article.pl?sid=06/12/09/220218

Reading that, it has just sprung in my mind that Bjarne's initials spell BS... ;P -- Bruno Medeiros - MSc in CS/E student http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D
Dec 11 2006