www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - TDD is BS?

reply "Szymon Gatner" <noemail gmail.com> writes:
This is not strictly D related but I am very curious about D's 
community opinion on the points made by non other than Jim 
Coplien here:

http://www.tele-task.de/archive/video/flash/16130/

D is the only language (that I am aware of) that has first class 
unit testing support. What do you think? Do we really just 
"mentally masturbate"?

Article about the myths of TDD referenced in the talk:

http://digitalcommons.calpoly.edu/cgi/viewcontent.cgi?article=1027&context=csse_fac
Jun 19 2013
next sibling parent reply "Paulo Pinto" <pjmlp progtools.org> writes:
On Wednesday, 19 June 2013 at 11:01:05 UTC, Szymon Gatner wrote:
 This is not strictly D related but I am very curious about D's 
 community opinion on the points made by non other than Jim 
 Coplien here:

 http://www.tele-task.de/archive/video/flash/16130/

 D is the only language (that I am aware of) that has first 
 class unit testing support. What do you think? Do we really 
 just "mentally masturbate"?

 Article about the myths of TDD referenced in the talk:

 http://digitalcommons.calpoly.edu/cgi/viewcontent.cgi?article=1027&context=csse_fac

I do believe in testing and while I seldom use D, the JVM and .NET projects I take part on, have usually a focus on unit testing. However, testing is done in parallel with coding or on the final week of each sprint. I don't believe in TDD without design, coding without design is cowboy coding where the system architecture ends up full with broken design. TDD, mocking and so on, is nice product to sell in conferences presented with dynamic languages where the presenter has full control of all libraries. This is surely not the case in the enterprise with strong typed languages, where most of the third party libraries are available in binary form only. Additionally I am yet to find any form of unit testing that is possible to apply to UI code, in a way that the ROI really pays off. Finally, 100% test coverage does not mean anything if the tests are not written in a way to really test the product. If you want to see a junk pile unit tests just ask for 100% coverage, you will get tests that exercise all code paths but without any real meaning. -- Paulo
Jun 19 2013
parent reply Jacob Carlborg <doob me.com> writes:
On 2013-06-19 14:08, Paulo Pinto wrote:

 Additionally I am yet to find any form of unit testing that is possible
 to apply to UI code, in a way that the ROI really pays off.

At work we use RSpec together with Capybara to test the UI, or rather the full stack. That is, for our web site. Capybara is used, via different drivers, to control the web browser. When writing the test you can use a driver that opens the web browser and you can actually see it clicking on buttons, filling in forms and so on. When running the tests in a CI server we use a faster headless web browser. -- /Jacob Carlborg
Jun 19 2013
parent reply Paulo Pinto <pjmlp progtools.org> writes:
Am 19.06.2013 17:54, schrieb Jacob Carlborg:
 On 2013-06-19 14:08, Paulo Pinto wrote:

 Additionally I am yet to find any form of unit testing that is possible
 to apply to UI code, in a way that the ROI really pays off.

At work we use RSpec together with Capybara to test the UI, or rather the full stack. That is, for our web site. Capybara is used, via different drivers, to control the web browser. When writing the test you can use a driver that opens the web browser and you can actually see it clicking on buttons, filling in forms and so on. When running the tests in a CI server we use a faster headless web browser.

With browser UI testing is a bit easier, but what do you do when doing native applications? Or to come back to the browser area, you need to test the UI in all target browsers, across all supported OS. Not all tools do this, even Selenium, which I have in high regard, has issues. -- Paulo
Jun 19 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-06-19 19:47, Paulo Pinto wrote:

 With browser UI testing is a bit easier, but what do you do when doing
 native applications?

I haven't done any testing of the UI in native applications. It was quite a while since I worked on a native application with a GUI. For CLI based tools I use similar tools, Cucumber (I could use RSpec instead) and Aruba.
 Or to come back to the browser area, you need to test the UI in all
 target browsers, across all supported OS. Not all tools do this, even
 Selenium, which I have in high regard, has issues.

Yes, absolutely. We currently don't do that. But it at least better to test in one browser than no one. There's a testing service, sauce labs, that supports many web browser. I haven't used it myself. https://saucelabs.com/ -- /Jacob Carlborg
Jun 19 2013
prev sibling next sibling parent "qznc" <qznc web.de> writes:
On Wednesday, 19 June 2013 at 11:01:05 UTC, Szymon Gatner wrote:
 D is the only language (that I am aware of) that has first 
 class unit testing support. What do you think? Do we really 
 just "mentally masturbate"?

Let us distinguish between "writing tests" and "test-driven development". I am for all for "writing tests" where it makes sense. I do not subscribe to 100% test coverage. Sometimes other things have higher priority (time-to-market, fun, return-of-investment, politics, etc). I consider 100% an ideal, which can be thought about from time to time. It is not a good recommendation for rookies or best practice. Test-driven development is an interesting beast. If you listen to the inventors, it makes a lot of sense in the scenarios they are talking about: Consultant creating or modifying a component in an enterprise environment. Continuous refactoring tries to evenly distribute the technical debt, so developers can be moved in and out of the project without friction. Without continuous refactoring someone can get the fame, while leaving the technical debt to somebody else. Test coverage enables continuous refactoring because you can check the component in isolation, if your refactoring did not break anything. The test-first principle ensures good test coverage. Considering this reasoning, TDD can be seen as an approach for managing technical debt.
Jun 19 2013
prev sibling next sibling parent "Regan Heath" <regan netmail.co.nz> writes:
On Wed, 19 Jun 2013 12:01:02 +0100, Szymon Gatner <noemail gmail.com>  
wrote:
 D is the only language (that I am aware of) that has first class unit  
 testing support. What do you think? Do we really just "mentally  
 masturbate"?

I'm more interested in whether DCI is doable/natural in D.. that seems like an interesting idea. R -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Jun 19 2013
prev sibling next sibling parent reply "irritate" <irritate gmail.com> writes:
On Wednesday, 19 June 2013 at 11:01:05 UTC, Szymon Gatner wrote:
 D is the only language (that I am aware of) that has first 
 class unit testing support. What do you think? Do we really 
 just "mentally masturbate"?

My feelings about TDD changed when I saw that talk explaining TDD in the context of double-entry bookkeeping in accounting (maybe linked off of here: http://unhandled-exceptions.com/blog/index.php/2009/02/15/uncle-bob-tdd-as-double-entry-bookkeeping/). Writing your tests and code separately is actually an important part of it. If you are copy-pasting your code into your test after you write it, then that's not really giving you the guarantees you want. And for anyone who's never tried the loop of: 1. Write a failing test. 2. Write the code that makes the test pass. 3. Make all the lights go green, then goto 1. You might want to give it a try. I was surprised by the feeling of accomplishment I would get from making the failing tests pass. irritate
Jun 19 2013
next sibling parent reply Paulo Pinto <pjmlp progtools.org> writes:
Am 19.06.2013 17:32, schrieb Szymon Gatner:
 On Wednesday, 19 June 2013 at 14:44:26 UTC, irritate wrote:
 On Wednesday, 19 June 2013 at 11:01:05 UTC, Szymon Gatner wrote:
 D is the only language (that I am aware of) that has first class unit
 testing support. What do you think? Do we really just "mentally
 masturbate"?

My feelings about TDD changed when I saw that talk explaining TDD in the context of double-entry bookkeeping in accounting (maybe linked off of here: http://unhandled-exceptions.com/blog/index.php/2009/02/15/uncle-bob-tdd-as-double-entry-bookkeeping/). Writing your tests and code separately is actually an important part of it. If you are copy-pasting your code into your test after you write it, then that's not really giving you the guarantees you want. And for anyone who's never tried the loop of: 1. Write a failing test. 2. Write the code that makes the test pass. 3. Make all the lights go green, then goto 1. You might want to give it a try. I was surprised by the feeling of accomplishment I would get from making the failing tests pass. irritate

That is actually the way I do TDD too. In fact, doing it any other way is actually against 3 laws of TDD: http://butunclebob.com/ArticleS.UncleBob.TheThreeRulesOfTdd and is plain wrong. Writing tests for working (legacy) code will not in any way shape it's structure and often results in useless tests. TDD is a design technique not a testing technique. I am pretty sure that many people will agree that TDD works, I was actually curious on opinion on the presentation which claims that TDD does not work at all and is in fact total BS.

How do you TDD: - native application UI code - embedded native code - distributed algorithms - applications done in native languages with third party libraries only available in binary form - GPGPU code Any of those cases full new greenfield applications. -- Paulo
Jun 19 2013
parent reply Jacob Carlborg <doob me.com> writes:
On 2013-06-19 19:52, Paulo Pinto wrote:

 - applications done in native languages with third party libraries only

When I write a test for my own code that uses third party code or services I assume that they work 100% and will never fail. That's because I'm testing MY code not the third party code. For services it's also a good idea to mock them. -- /Jacob Carlborg
Jun 19 2013
parent reply Paulo Pinto <pjmlp progtools.org> writes:
Am 19.06.2013 23:11, schrieb Jacob Carlborg:
 On 2013-06-19 19:52, Paulo Pinto wrote:

 - applications done in native languages with third party libraries only

When I write a test for my own code that uses third party code or services I assume that they work 100% and will never fail. That's because I'm testing MY code not the third party code. For services it's also a good idea to mock them.

The issue is not to test third party libraries, far from it. The problem is that you cannot mock them, specially if you rely a lot on non virtual methods or pure function calls. Or on framework code that calls your code back, after certain events happened in the system. To do that properly, you end up wrapping third party code into code that can be replaced for unit tests execution, thus making the test effort an herculean task. Good luck convincing any project manager on the enterprise world that wrapping third party code to ease unit testing is worthwhile. -- Paulo
Jun 19 2013
parent reply Jacob Carlborg <doob me.com> writes:
On 2013-06-20 00:13, Paulo Pinto wrote:

 The issue is not to test third party libraries, far from it.

 The problem is that you cannot mock them, specially if you rely a lot on
 non virtual methods or pure function calls. Or on framework code that
 calls your code back, after certain events happened in the system.

I would only mock an external service that needs to be available online. I don't mock third party code. That would be insane. There's no limit. Should I mock the standard library?
 To do that properly, you end up wrapping third party code into code that
 can be replaced for unit tests execution, thus making the test effort an
 herculean task.

I have only had a few cases where I needed to mock. I have only done that in our Ruby on Rails project. I guess that's quite a lot easier since you can very easy replace any method at runtime. -- /Jacob Carlborg
Jun 20 2013
parent reply Paulo Pinto <pjmlp progtools.org> writes:
Am 20.06.2013 11:26, schrieb Jacob Carlborg:
 On 2013-06-20 00:13, Paulo Pinto wrote:

 The issue is not to test third party libraries, far from it.

 The problem is that you cannot mock them, specially if you rely a lot on
 non virtual methods or pure function calls. Or on framework code that
 calls your code back, after certain events happened in the system.

I would only mock an external service that needs to be available online. I don't mock third party code. That would be insane. There's no limit. Should I mock the standard library?

No, but if your code calls something that ends up calling again other part of your code, you might need to mock it somehow. Specially if it is a library that does some IO (network, disk) in the process and is full of static methods. -- Paulo
Jun 23 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-06-23 21:50, Paulo Pinto wrote:

 No, but if your code calls something that ends up calling again other
 part of your code, you might need to mock it somehow.

I'm using same approach there. If I'm testing method "a" and it also calls method "b". I assume method "b" works and I don't mock it. Then I make sure to have a test for method "b" as well. A given test only tests one method.
 Specially if it is a library that does some IO (network, disk) in the
 process and is full of static methods.

Especially network IO is a good idea to mock. -- /Jacob Carlborg
Jun 23 2013
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-06-19 19:29, H. S. Teoh wrote:

 Producing "minimal
 code" that satisfies the test simply gets you stuck in local minima
 without ever getting any nearer an actual solution for the problem.

Then your test doesn't cover enough cases.
 Before methodologies like TDD could even begin to work, one has to
 *solve* the problem at hand first -- analyse the problem, explore its
 structure, invent an algorithm, then one can verify the correctness of
 one's implementation with unittests. You have to already have an idea
 about how things are going to work, before TDD can help you.

I agree. I haven't watch the video(s) but I think TDD is for testing an implementation not a design. Sometimes you can also test the design, or rather find wholes in the design. -- /Jacob Carlborg
Jun 19 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-06-19 23:19, H. S. Teoh wrote:

 How do you write a test that covers "enough" cases for a sudoku solver?

I don't know. I you're about to create a sum function and use TDD. 1. Create the test: auto result = sum(1, 2); assert(result == 3); 2. Implement: int sum (int a, int b) { return 3; } In the above case you obviously haven't enough cases in your test to assert that your implementation does what it supposed to do. This example is very simple compared to sudoku. -- /Jacob Carlborg
Jun 20 2013
prev sibling next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
irritate:

 My feelings about TDD changed when I saw that talk explaining 
 TDD in the context of double-entry bookkeeping in accounting

I think the usefulness of the TDD method is greatly different for different kinds of code to write. This is another data point for the discussion: http://devgrind.com/2007/04/25/how-to-not-solve-a-sudoku/ Regarding testing, I also like QuickCheck: http://www.haskell.org/haskellwiki/Introduction_to_QuickCheck Bye, bearophile
Jun 19 2013
prev sibling next sibling parent "Szymon Gatner" <noemail gmail.com> writes:
On Wednesday, 19 June 2013 at 14:44:26 UTC, irritate wrote:
 On Wednesday, 19 June 2013 at 11:01:05 UTC, Szymon Gatner wrote:
 D is the only language (that I am aware of) that has first 
 class unit testing support. What do you think? Do we really 
 just "mentally masturbate"?

My feelings about TDD changed when I saw that talk explaining TDD in the context of double-entry bookkeeping in accounting (maybe linked off of here: http://unhandled-exceptions.com/blog/index.php/2009/02/15/uncle-bob-tdd-as-double-entry-bookkeeping/). Writing your tests and code separately is actually an important part of it. If you are copy-pasting your code into your test after you write it, then that's not really giving you the guarantees you want. And for anyone who's never tried the loop of: 1. Write a failing test. 2. Write the code that makes the test pass. 3. Make all the lights go green, then goto 1. You might want to give it a try. I was surprised by the feeling of accomplishment I would get from making the failing tests pass. irritate

That is actually the way I do TDD too. In fact, doing it any other way is actually against 3 laws of TDD: http://butunclebob.com/ArticleS.UncleBob.TheThreeRulesOfTdd and is plain wrong. Writing tests for working (legacy) code will not in any way shape it's structure and often results in useless tests. TDD is a design technique not a testing technique. I am pretty sure that many people will agree that TDD works, I was actually curious on opinion on the presentation which claims that TDD does not work at all and is in fact total BS.
Jun 19 2013
prev sibling next sibling parent "Szymon Gatner" <noemail gmail.com> writes:
On Wednesday, 19 June 2013 at 14:04:35 UTC, Regan Heath wrote:
 On Wed, 19 Jun 2013 12:01:02 +0100, Szymon Gatner 
 <noemail gmail.com> wrote:
 D is the only language (that I am aware of) that has first 
 class unit testing support. What do you think? Do we really 
 just "mentally masturbate"?

I'm more interested in whether DCI is doable/natural in D.. that seems like an interesting idea. R

I actually learned about DCI from that very presentation and am wondering the same thing.
Jun 19 2013
prev sibling next sibling parent "irritate" <irritate gmail.com> writes:
On Wednesday, 19 June 2013 at 15:32:41 UTC, Szymon Gatner wrote:
 I am pretty sure that many people will agree that TDD works, I
 was actually curious on opinion on the presentation which claims
 that TDD does not work at all and is in fact total BS.

I am having trouble getting the video to play at the moment (it freezes every few seconds), so I can't address his actual claims right now. However, while searching for an alternate site for it, I found this video which others in this thread might find interesting: http://www.youtube.com/watch?v=KtHQGs3zFAM. irritate
Jun 19 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 19 June 2013 at 11:01:05 UTC, Szymon Gatner wrote:
 This is not strictly D related but I am very curious about D's 
 community opinion on the points made by non other than Jim 
 Coplien here:

 http://www.tele-task.de/archive/video/flash/16130/

 D is the only language (that I am aware of) that has first 
 class unit testing support. What do you think? Do we really 
 just "mentally masturbate"?

 Article about the myths of TDD referenced in the talk:

 http://digitalcommons.calpoly.edu/cgi/viewcontent.cgi?article=1027&context=csse_fac

From experience, unitests are various return depending on the code. Some code is easily testable and the ROI is high, some code is just hard to automatically test. On the code that is unittest compliant, I notice a very low number of bugs (actually close to 0 defect) when I do proper unittesting. Sadly, all code isn't like that.
Jun 19 2013
prev sibling next sibling parent "Regan Heath" <regan netmail.co.nz> writes:
On Wed, 19 Jun 2013 16:34:03 +0100, Szymon Gatner <noemail gmail.com>  
wrote:

 On Wednesday, 19 June 2013 at 14:04:35 UTC, Regan Heath wrote:
 On Wed, 19 Jun 2013 12:01:02 +0100, Szymon Gatner <noemail gmail.com>  
 wrote:
 D is the only language (that I am aware of) that has first class unit  
 testing support. What do you think? Do we really just "mentally  
 masturbate"?

I'm more interested in whether DCI is doable/natural in D.. that seems like an interesting idea. R

I actually learned about DCI from that very presentation and am wondering the same thing.

So.. to me it seems the basic idea is that you have your object i.e. BankAccount which is a dumb/data object with few methods (perhaps just properties). Then, depending on the use-case it participates in, it takes on a role (at run time) which carries with it some methods. So.. something like? import std.stdio; // Dumb/data object class BankAccount { public: string name; int balance; this(string _name, int _balance) { name = _name; balance = _balance; } } // Roles.. class SourceAccount(T) { private: T account; public: alias account this; this(T _account) { account = _account; } void TransferTo(TargetAccount!BankAccount target, int amount) { target.balance += amount; balance -= amount; } } class TargetAccount(T) { private: T account; public: alias account this; this(T _account) { account = _account; } } // Use case.. void TransferFunds(BankAccount sa, BankAccount ta, int amount) { auto source = new SourceAccount!BankAccount(sa); auto target = new TargetAccount!BankAccount(ta); source.TransferTo(target, amount); } // Logic.. void main() { auto savings = new BankAccount("savings", 10000); auto current = new BankAccount("current", 250); writefln("Current balances: "); writefln(" %s: %d", savings.name, savings.balance); writefln(" %s: %d", current.name, current.balance); writefln(""); writefln("Transfer funds.."); TransferFunds(savings, current, 500); writefln(""); writefln("New balances: "); writefln(" %s: %d", savings.name, savings.balance); writefln(" %s: %d", current.name, current.balance); } R -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Jun 19 2013
prev sibling next sibling parent "Jesse Phillips" <Jesse.K.Phillips+D gmail.com> writes:
On Wednesday, 19 June 2013 at 11:01:05 UTC, Szymon Gatner wrote:
 This is not strictly D related but I am very curious about D's 
 community opinion on the points made by non other than Jim 
 Coplien here:

 http://www.tele-task.de/archive/video/flash/16130/

 D is the only language (that I am aware of) that has first 
 class unit testing support. What do you think? Do we really 
 just "mentally masturbate"?

 Article about the myths of TDD referenced in the talk:

 http://digitalcommons.calpoly.edu/cgi/viewcontent.cgi?article=1027&context=csse_fac

If you could point to the time where he talks about TDD that would be good. Based on what I've be hearing is his biggest complaint is about classes and class hierarchy design. From this I'm pretty sure this is where he takes issue with TDD, it doesn't help build good class design. At about 1:09 he goes into talking about roles, specifically, write a script then assign out the role to a class to perform that role. And in this way, it sounds very much like test driven design. Script the action, find/build/train those to fulfill those actions, he just left out the testing part. For my hobby projects I find it very hard to find a use for Objects and their hierarchy, I find structures and functions are much easier to deal with and test (not that I do the best on testing).
Jun 19 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Jun 19, 2013 at 04:52:22PM +0200, bearophile wrote:
 irritate:
 
My feelings about TDD changed when I saw that talk explaining TDD
in the context of double-entry bookkeeping in accounting

I think the usefulness of the TDD method is greatly different for different kinds of code to write. This is another data point for the discussion: http://devgrind.com/2007/04/25/how-to-not-solve-a-sudoku/

As someone else has pointed out, we need to distinguish between unit-testing and TDD. I love D's built-in unittests. They have singlehandedly improved the quality of my code by leaps and bounds, just because they're so convenient to use, and help catch careless mistakes early. However, I am skeptical about TDD. Ideally, what unittests *really* should be used for, is to set expectations about what results a piece of code should give, but *how* that code achieves said result is a major question that's never adequately addressed by TDD. Using the above sudoku example (thanks, bearophile!), one can write a unittest that verifies a sudoku solution, but that in itself says *nothing* about how the code should go about deriving such a solution. Producing "minimal code" that satisfies the test simply gets you stuck in local minima without ever getting any nearer an actual solution for the problem. Before methodologies like TDD could even begin to work, one has to *solve* the problem at hand first -- analyse the problem, explore its structure, invent an algorithm, then one can verify the correctness of one's implementation with unittests. You have to already have an idea about how things are going to work, before TDD can help you. Of course, for trivial everyday problems like sorting and manipulating lists of stuff, moving numbers around, etc., TDD can certainly work well -- because the underlying algorithms are so trivial that there's not much need to actually do any algorithm design. In such cases, "minimal code" is pretty close to "actual solution", so TDD converges on it pretty quickly. For algorithmically hard problems, however, like solving sudoku, or computing n-dimensional convex hulls, or writing a compiler optimizer, TDD simply falls flat on its face because it doesn't even begin to address fundamental issues of how one achieves the desired results from the inputs in the first place. T -- Без труда не выловишь и рыбку из пруда.
Jun 19 2013
prev sibling next sibling parent "Jesse Phillips" <Jesse.K.Phillips+D gmail.com> writes:
Start at about 23 min. Key phrase at about 29 min.

I completely agree with, "we right tests because we don't 
understand our code." We don't want to understand all the code, 
we also couldn't possibly achieve perfect understanding anyway, 
so let us make the burden of transferring that knowledge easier 
by having tests which catch mistakes.

I think his school analogy is wrong. TDD is more like teachers 
writing a test to define what a successful teaching would be, 
then developing a course which produces those results. Then you 
administer the test to make sure your teaching fulfilled the 
requirement. Oh that is exactly what we do.
Jun 19 2013
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, June 19, 2013 10:29:49 H. S. Teoh wrote:
 On Wed, Jun 19, 2013 at 04:52:22PM +0200, bearophile wrote:
 irritate:
My feelings about TDD changed when I saw that talk explaining TDD
in the context of double-entry bookkeeping in accounting

I think the usefulness of the TDD method is greatly different for different kinds of code to write. This is another data point for the discussion: http://devgrind.com/2007/04/25/how-to-not-solve-a-sudoku/

[...] As someone else has pointed out, we need to distinguish between unit-testing and TDD. I love D's built-in unittests. They have singlehandedly improved the quality of my code by leaps and bounds, just because they're so convenient to use, and help catch careless mistakes early. However, I am skeptical about TDD.

Exactly. With TDD, you write the tests and then write the code, and I personally don't like that approach at all. I write the code; then I write the tests; and then I work on both until the code works with a reasonable amount of testing (which in most cases, I think is 100% with lots of corner cases being tested, but there's plenty of code where you can't reasonably do that). And while that approach involves heavy unit testing, it's not "test-driven" at all. But if you want to do TDD, D certainly allows it. The unit testing facilities in the language enable unit testing, but how you go about writing them is completely up to you. - Jonathan M Davis
Jun 19 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Jun 19, 2013 at 11:06:26PM +0200, Jacob Carlborg wrote:
 On 2013-06-19 19:29, H. S. Teoh wrote:
 
Producing "minimal code" that satisfies the test simply gets you
stuck in local minima without ever getting any nearer an actual
solution for the problem.

Then your test doesn't cover enough cases.

How do you write a test that covers "enough" cases for a sudoku solver?
Before methodologies like TDD could even begin to work, one has to
*solve* the problem at hand first -- analyse the problem, explore its
structure, invent an algorithm, then one can verify the correctness
of one's implementation with unittests. You have to already have an
idea about how things are going to work, before TDD can help you.

I agree. I haven't watch the video(s) but I think TDD is for testing an implementation not a design.

Doesn't TDD stand for Test-Driven *Design*? T -- Those who've learned LaTeX swear by it. Those who are learning LaTeX swear at it. -- Pete Bleackley
Jun 19 2013
prev sibling next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 06/19/2013 08:58 PM, Jonathan M Davis wrote:
 Exactly. With TDD, you write the tests and then write the code

Isn't this what is highlighted in the second article linked to by the OP, as being precisely one of the misinterpretations of TDD? I'm not sure if I've misinterpreted you here, because one needs to distinguish between "write all tests first, then code" (which is not what TDD means) and the rather tighter iteration/co-evolution of test and code writing which is supposed to characterize TDD. If you meant the latter then I apologize.
Jun 19 2013
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/19/2013 4:01 AM, Szymon Gatner wrote:
 This is not strictly D related but I am very curious about D's community
opinion
 on the points made by non other than Jim Coplien here:

TDD strikes me as an ad-hoc way of constructing code, and is a poor substitute for thinking about the problem as a whole. For example, I don't really see how getting a square root function to pass its test cases is going to lead one to implementing one of the classic algorithms for computing square roots. I don't see how TDD for a compiler will lead one to rediscover what is known about the best way to organize the logic of a compiler. TDD is "curve fitting" which is what one does when one has no understanding. D's support of unit testing is not an endorsement of TDD.
Jun 19 2013
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/19/2013 3:23 PM, Szymon Gatner wrote:
 Point of TDD for a square root function would be to create a good API for
 getting a square root of a number. Implementation is just a detail.

I just can't accept that. For one thing, implementation details often must drive the interface. Just writing specs without any knowledge of how it would be implemented will not produce an efficient design. For the square root, there's a definite tradeoff between accuracy and speed. With no knowledge of those tradeoffs, and just coming up with a spec, how can you make the right decisions?
Jun 19 2013
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-06-20 01:44, Walter Bright wrote:

 I just can't accept that. For one thing, implementation details often
 must drive the interface. Just writing specs without any knowledge of
 how it would be implemented will not produce an efficient design.

 For the square root, there's a definite tradeoff between accuracy and
 speed. With no knowledge of those tradeoffs, and just coming up with a
 spec, how can you make the right decisions?

First, just because you have written a test doesn't mean you can't change it. Second, if you need on beforehand that solving this particular problem need to use a given algorithm which will require this certain API, then write your tests to match that. Otherwise, I would say, that write an API that is as straightforward and easy to use as possible. Then benchmark and profile your code and see where the implementation need to be change to satisfy your needs. If changing the implementation requires you to change the API, then do that and change the tests as well. And in the end, don't be so religious about these things and use your common sense :) -- /Jacob Carlborg
Jun 20 2013
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-06-20 20:14, Sean Kelly wrote:

 And this is why the waterfall model (which TDD seems to encourage) tends to be
problematic.

You don't write all the test at once and then all the implementations at once. Write tests for one function first, then implement it. Then move to the next function. Also, tests are not written in stone. They should evolve and change just as much as the rest of the code. -- /Jacob Carlborg
Jun 20 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-06-21 00:37, Nick Sabalausky wrote:

 I'm not saying that it's necessarily bad to write a function's tests
 before the function itself, I just don't see what it really matters to
 code the tests first. The important thing is that the tests get
 written, ideally before you move on to something else. If you have the
 discipline to write the tests before the function, then you have the
 discipline to write the tests after the function.

I agree. But the point is also, as other has mentioned, that you should look from the client's point of view when designing your API. So if you write the test firs you're writing the client code first. The you think about how the API should look like from the client's point of view. If the implementation cannot work or fit in that API you have to change the API. -- /Jacob Carlborg
Jun 21 2013
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-06-21 00:18, Nick Sabalausky wrote:

 Right. It's like creating a Vacuum Cleaner or a car or a kitchen
 appliance by designing the case, housing and controls first, and then
 trying to design working internals to fit that mold. Form needs to
 follow function.

Ever heard of Apple :) -- /Jacob Carlborg
Jun 21 2013
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-06-19 23:54, Walter Bright wrote:

 TDD strikes me as an ad-hoc way of constructing code, and is a poor
 substitute for thinking about the problem as a whole. For example, I
 don't really see how getting a square root function to pass its test
 cases is going to lead one to implementing one of the classic algorithms
 for computing square roots. I don't see how TDD for a compiler will lead
 one to rediscover what is known about the best way to organize the logic
 of a compiler.

 TDD is "curve fitting" which is what one does when one has no
 understanding.

I have not watch the video(s) but I think you completely failed to understand what TDD is about. This is how I think about TDD: 1. You have a problem to solve 2. You think about how to solve the problem and how a design can look like 3. You come up with a design 4. Write a test according to the design 5. Write the implementation 6. Run the test to see that the implementation matches your design 7. Repeat 2-6 until satisfied Example, write a compiler: Problem: Lex string literals Test: auto code = `"asd"`; auto token = lex(code); assert(token.lexeme == code); assert(token.type == Type.stringLiteral); Implementation: Write the implementation to pass the above test. Token lex (string code) { return Token(code, Type.stringLiteral); } This is obviously a dummy implementation but it's enough to pass the test. The you need write more tests that force you to implement the function correctly. Like below. Test again: auto code = `"foo""bar"`; auto token = lex(code); assert(token.lexeme == `"foo"`); assert(token.type == Type.stringLiteral); Fix the implementation to pass the above test. Test again: auto code = `"foo`; auto token = lex(code); assert(token.type == Type.error); // unterminated string literal Fix the implementation to pass the above test. Usually I write the test first and immediately write the "correct" implementation instead of these dummy implementation. Then I add some more tests which possibly will fail because I found an edge case I forgot to think about when I implemented the code. BTW, when you don't know how to implement something and you're trying your way forward. You can write proper tests instead of using printf or similar. -- /Jacob Carlborg
Jun 20 2013
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/20/13 5:49 AM, Jacob Carlborg wrote:
 1. You have a problem to solve
 2. You think about how to solve the problem and how a design can look like
 3. You come up with a design
 4. Write a test according to the design
 5. Write the implementation
 6. Run the test to see that the implementation matches your design
 7. Repeat 2-6 until satisfied

This is reasonable.
 Example, write a compiler:

 Problem: Lex string literals

 Test:

 auto code = `"asd"`;
 auto token = lex(code);
 assert(token.lexeme == code);
 assert(token.type == Type.stringLiteral);

 Implementation:

 Write the implementation to pass the above test.

 Token lex (string code)
 {
 return Token(code, Type.stringLiteral);
 }

 This is obviously a dummy implementation but it's enough to pass the
 test.

I dislike this. I've seen conference talks and such where the speakers present the inevitable throwaway implementation that is patently wrong but makes the first unittest pass. That mindset doesn't sit well with me at all. I don't see why I need to waste time writing tongue-in-cheek code that is nonsensical and will obviously be deleted next. Andrei
Jun 20 2013
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-06-20 16:46, Andrei Alexandrescu wrote:

 I dislike this. I've seen conference talks and such where the speakers
 present the inevitable throwaway implementation that is patently wrong
 but makes the first unittest pass. That mindset doesn't sit well with me
 at all. I don't see why I need to waste time writing tongue-in-cheek
 code that is nonsensical and will obviously be deleted next.

Yes, I agree with you. I skip that test and write the correct implementation from the beginning. That's what I wrote in the last paragraph. -- /Jacob Carlborg
Jun 20 2013
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-06-21 05:32, Nick Sabalausky wrote:

 [Snip]

 Bottom line is, schools are absolute garbage, and academic
 achievement is *at best* completely meaningless.

I couldn't agree more with your post. The problem though is that most hiring process uses the HR department, as you mentioned in an other post, and they only look at the degree. I only took a degree to have something "to show" for these HR departments, most of what I learned I learned on my own time. -- /Jacob Carlborg
Jun 21 2013
parent reply Jacob Carlborg <doob me.com> writes:
On 2013-06-21 16:56, H. S. Teoh wrote:

 +1, me too! I can say that 85-90% of what I do at work today, I learned
 from my personal coding projects, not from the CS courses I took in
 university. (That's why I like to joke about CS grads knowing more about
 uncomputable problems than computable ones...)

It feels like there's something wrong with the world here :) -- /Jacob Carlborg
Jun 21 2013
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-06-22 00:12, Nick Sabalausky wrote:

 - The nasty little details like pointer/memory problems, linker errors,
    etc that real people have to deal with are neatly glossed over and
    sidestepped.

They never teach you about that.
 - There was one CS101 teacher (I had to tutor her unfortunate students)
    who constantly bragged about being from a real-world software
    company...but she was a Java-addict (circa v1.2-v1.4) who kept trying
    to teach OO *before* basic flow-of-execution. Consequently, none of
    her unfortunate students had the slightest clue what was going on.

 - Many of the professors are terrible programmers themselves. For
    example, I had one who openly admitted the only language he knew was
    C, and yet at one point it became painfully obvious that he had
    almost no comprehension of null-terminated strings.

What annoys me is that sometimes they teach flat out lies to the students. I watch a couple of videos from a course from Standford, I think it was algorithms and data structures or similar. The teacher introduce the Map data structure by showing an interface for it and how it works. She tells the student that you cannot have any other type for the keys than strings. I think she even says it's impossible. I can understand that they don't want to make it too complicated when it's introduced for the first time. But she at least could have said something like, "for now, lets imagine only string types are possible". -- /Jacob Carlborg
Jun 22 2013
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/22/13 3:18 AM, Jacob Carlborg wrote:
 I watch a couple of videos from a course from Standford, I think it was
 algorithms and data structures or similar. The teacher introduce the Map
 data structure by showing an interface for it and how it works. She
 tells the student that you cannot have any other type for the keys than
 strings.

I'm very surprised to hear that, Stanford is a good university. You'd certainly need good evidence to make us believe that :o). Andrei
Jun 22 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-06-22 18:27, Andrei Alexandrescu wrote:

 I'm very surprised to hear that, Stanford is a good university. You'd
 certainly need good evidence to make us believe that :o).

I was surprised when I watch the video. But here it is: https://itunes.apple.com/se/podcast/6.-programming-abstractions/id384232917?i=85092416&mt=2 Around 14:50 in the video. She actually didn't say it was impossible. -- /Jacob Carlborg
Jun 22 2013
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-06-22 00:12, Nick Sabalausky wrote:

 - There was one CS101 teacher (I had to tutor her unfortunate students)
    who constantly bragged about being from a real-world software
    company...but she was a Java-addict (circa v1.2-v1.4) who kept trying
    to teach OO *before* basic flow-of-execution. Consequently, none of
    her unfortunate students had the slightest clue what was going on.

We learned OO before any programming. First we had a course about analyze and design of object oriented systems, or something like that. They thought the basic concepts of OO, UML diagrams and other stuff. After that course we learned programming and the practical side of OO, that is Java.
 - Many of the teachers don't even teach, they just collect the
    thousands of dollars in tuition and give you a book recommendation
    (really more of a "demand" than a recommendation). Now, I'm a strong
    believer in being self-taught and learning from books, but all I need
    for that is a library card, not a $100k debt and four years of elitist
    attitudes from people who clearly don't know what they're doing
    anyway.

During my years at the university I bought three books. One of the was really good, the rest wasn't worth it. -- /Jacob Carlborg
Jun 22 2013
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-06-22 17:24, QAston wrote:

 I'm not saying that you can't get all of this on your own, yet it's
 easier that way. I was a self taught programmer, but I feel that my
 knowledge is more complete now (and some of the knowledge is very
 important for a systems programmer). It's obviously the same other way
 around - you can pass all the exams and get nothing from that. It's up
 to you what you're going to do with free time. Important note: public
 unis are free in my country, I probably wouldn't study otherwise.

I would say if you want to learn about programming, read these newsgroup. I have learned more when reading these newsgroup then I ever could at the university. So I just want to say, thank you all. -- /Jacob Carlborg
Jun 22 2013
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-06-21 08:33, Andrej Mitrovic wrote:

 Seems like nowadays it's not too far-fetched to ask for a
 github/bitbucket/etc username and see the work they've done, the way
 they write code, contribute, etc. It should give a better "feel" than
 any interview.

I agree. Most interviews I have been on lately is due to my github project. -- /Jacob Carlborg
Jun 21 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-06-21 12:20, Andrej Mitrovic wrote:

 Oh? That's cool. Which [rpkect?

I don't know if it was a particular project but at least one company was interested in several projects, DStep (convert C/Objective-C headers to D modules), Jazz (D frontend) and Orbit (D package manager). https://github.com/jacob-carlborg/dstep https://github.com/jacob-carlborg/jazz https://github.com/jacob-carlborg/orbit -- /Jacob Carlborg
Jun 21 2013
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-06-21 07:57, H. S. Teoh wrote:

 we got to ask HR to first
 administer a technical test before any interviews are arranged; test
 results are reviewed before deciding to interview the candidate

I done tests like that, they all suck. This is how it usually works: You get a problem to solve. They say: "solve it anyway you want, using any language you want". You solve it farily quickly and straight forward. Then they say, "you are not allowed to use that function". Basically they're saying it's cheating. Then you remove that function and change the implementation accordingly. Then they say, "you are not allowed to use that other function". You change the code accordingly and this dance continues. Then you're thinking to yourself, "Am I supposed to reimplement the standard library?". I would say, most times, it's almost irresponsible to _not_ use the standard library. A bunch of people, a lot smarter than I, have had a good 20 years or so to perfect and fine tune the standard library. Not a chance in hell that I'm going to beat that, on an interview, with code written on a whiteboard. A side note. You're of curse not allowed to look at any documentation at all and you're not allowed to use a text editor. You write the code on a whiteboard, yes a _whiteboard_. That's just so stupid. That's not how programming works in the real world. Why the hell do they think IDE's have built in, easy to read/find, documentation. -- /Jacob Carlborg
Jun 21 2013
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-06-21 17:14, H. S. Teoh wrote:

 Heh, at my job, we're looking for skill in specific languages, so
 candidates are explicitly asked to write C code. And this is done not in
 an interview setting (which puts too much pressure on the candidate --
 can *you* write code in front of interviewers without panicking at least
 a bit?), but in a classroom setting. Though, granted, when *I* had to go
 through the procedure, it felt like I was taking my CS final exams all
 over again. Not exactly a pleasant experience. :-P

I had four, one hour each, interviews in one afternoon. Afterwards my brain was basically fried.
 That's the sign of a poorly-chosen test question. The kind of question I
 like to use doesn't focus on nitpicky details like that, but on actual
 algorithms or domain-specific knowledge, like showing that the candidate
 understands bit operations, pointer arithmetic, and stuff like that. On
 that note, it's rather shocking how few candidates know how to use
 malloc() and free() correctly...

I guess I managed to avoid what they actually wanted to test.
 That's not to say the tests my employer gives have only "good"
 questions, though; there are some that basically do what you describe:
 "please re-implement strstr(), please concatenate strings without using
 strcat()", and the like. When I was involved in hiring, I usually ignore
 these questions when evaluating the candidate, or cut them a lot of
 slack.

I can also understand that it not that easy to create meaningful tests like these.
 But I do have a slightly evil streak when it comes to technical tests...
 once I wrote a Javascript question that requires the candidate to
 evaluate what a piece of code does, that involves several nested
 closures and variable shadowing. We hired one of the (very) few
 candidates who actually got that question right, and she turned out to
 be an excellent coder.

Hehe.
 Well, I don't think the point of the test is to prove that you can write
 code that's *superior* to the standard library, but that you *can* write
 non-trivial functionality from ground up.

Yeah, I guess it's hard to find meaningful tests.
 Well, that's a new one. :-P  Our tests are administered as pen-and-paper
 tests. Not unlike CS final exams, unfortunately. :-/

I hate pen-and-paper tests too. I remember those from the university. I understand that they don't want you to have access to a computer with access to the internet or a compiler. But come on, can I at least get a simple text editor. Half of the time I spend on erasing code because I cannot insert new code in the middle of existing code. Then also writing those annoying curly braces (the actual symbols) which I never get right.
 Actually, we instruct HR to provide the standard C library manuals if
 candidates ask for it.

That's generous.
 There *are* some unfair questions that expect you
 to know, e.g., the exact order of arguments to select() and stuff like
 that, but like I said, it's not HR who evaluates the tests, but the
 hiring managers, so it's up to our discretion how much weight we put on
 questions like that. We usually disregard minor mistakes like using the
 wrong order of arguments to qsort(), if the intent is clear. We don't
 (or at least I don't) just blindly look for the highest-scoring test and
 hire those candidates; I usually take the time to look at the code and
 ask, "why did they write it that way?  what does that show about their
 coding abilities?".

I guess we have different experiences. -- /Jacob Carlborg
Jun 21 2013
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/21/13 5:59 AM, Jacob Carlborg wrote:
 On 2013-06-21 07:57, H. S. Teoh wrote:

 we got to ask HR to first
 administer a technical test before any interviews are arranged; test
 results are reviewed before deciding to interview the candidate

I done tests like that, they all suck. This is how it usually works: You get a problem to solve. They say: "solve it anyway you want, using any language you want". You solve it farily quickly and straight forward. Then they say, "you are not allowed to use that function". Basically they're saying it's cheating. Then you remove that function and change the implementation accordingly. Then they say, "you are not allowed to use that other function". You change the code accordingly and this dance continues. Then you're thinking to yourself, "Am I supposed to reimplement the standard library?". I would say, most times, it's almost irresponsible to _not_ use the standard library. A bunch of people, a lot smarter than I, have had a good 20 years or so to perfect and fine tune the standard library. Not a chance in hell that I'm going to beat that, on an interview, with code written on a whiteboard. A side note. You're of curse not allowed to look at any documentation at all and you're not allowed to use a text editor. You write the code on a whiteboard, yes a _whiteboard_. That's just so stupid. That's not how programming works in the real world. Why the hell do they think IDE's have built in, easy to read/find, documentation.

If there's any need to reach for documentation, the interviewer has failed. When interviewing we (at Facebook) ask problems that are likely to appear in a normal day's work, but for which the typical libraries don't help. (E.g. many libraries don't can't help with implementing unstable remove (see std.algorithm).) Also it's fair to ask about implementing a stdlib function itself if the interview concerns some systems-level work; e.g. brute-force strstr() is fair game and I think any engineer should be able to lift it off the ground quickly (to my dismay, only a fraction can). Paradoxically use of stdlib functions may actually hurt; I've seen people who e.g. call strlen() in a loop in order to implement strstr()! Andrei
Jun 21 2013
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/21/13 2:50 PM, Adam D. Ruppe wrote:
 On Friday, 21 June 2013 at 21:33:43 UTC, Andrei Alexandrescu wrote:
 brute-force strstr() is fair game and I think any engineer should be
 able to lift it off the ground quickly (to my dismay, only a fraction
 can).

But, should the return value be const or not? :P I think I'd write in D just cuz of inout().

That would be a nice touch and bring bonus points, but in most of the cases I tell the candidate to gloss over that detail. Andrei
Jun 21 2013
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/21/13 3:22 PM, Adam D. Ruppe wrote:
 Just for laughs I just slapped together a strstr

Post it and I'll destroy it. Andrei
Jun 21 2013
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/21/2013 3:35 PM, Andrei Alexandrescu wrote:
 On 6/21/13 3:22 PM, Adam D. Ruppe wrote:
 Just for laughs I just slapped together a strstr

Post it and I'll destroy it.

Can I play, too? Mine from the Digital Mars C library. Haven't looked at it since 2001. ================================================== /*_ strstr.c */ /* Copyright (C) 1985-2001 by Digital Mars */ /* All Rights Reserved */ /* www.digitalmars.com */ /* Written by Walter Bright */ #include <stdio.h> #include <ctype.h> #include <stddef.h> #include <string.h> #include <stdlib.h> #if 0 /* Smaller but slower under many circumstances. */ char *strstr(const char *s1,const char *s2) { size_t len2; size_t len1; char c2 = *s2; len1 = strlen(s1); len2 = strlen(s2); if (!len2) return (char *) s1; while (len2 <= len1) { if (c2 == *s1) if (memcmp(s2,s1,len2) == 0) return (char *) s1; s1++; len1--; } return NULL; } #else #include <limits.h> /* defines UCHAR_MAX */ /**************************************************************************** See "Algorithms" Second Edition by Robert Sedgewick. Boyer-Moore string search routine. ****************************************************************************/ char *strstr(const char *text, const char * pattern) { const size_t M = strlen(pattern); const unsigned long N = strlen(text); const char *p_p = pattern; const char *t_p; size_t skip[UCHAR_MAX + 1]; size_t i, j; if(M == 0) return (char *)text; if(M > N) /* If pattern is longer than the text string. */ return 0; #if __INTSIZE == 4 _memintset((int *)skip, M, UCHAR_MAX + 1); #else { size_t *s_p = skip + UCHAR_MAX; do { *s_p = M; }while(s_p-- > skip); } #endif p_p = pattern; do { skip[*(const unsigned char *)p_p] = M - 1 - (p_p - pattern); } while(*++p_p); p_p = pattern + M - 1; t_p = text + M - 1; while(1) { char c; c = *t_p; if(c == *p_p) { if(p_p - pattern == 0) return (char *)t_p; t_p--; p_p--; } else { size_t step = M - (p_p - pattern); if (step < skip[(unsigned char)c]) step = skip[(unsigned char)c]; /* If we have run out of text to search in. */ /* Need cast for case of large strings with 16 bit size_t... */ if((unsigned long)(t_p - text) + step >= N) return 0; t_p += step; p_p = pattern + M - 1; } } } #endif /* #if 0 */
Jun 21 2013
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-06-22 01:58, H. S. Teoh wrote:

 And where's the unittest block? ;-)  (OK OK, I know this wasn't written
 in D. But I had to ask. :-P)

DMC does support contracts, as an extension. That is, in and out contracts. I don't know about unit test blocks. -- /Jacob Carlborg
Jun 22 2013
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/22/2013 4:01 AM, Andrej Mitrovic wrote:
 On 6/22/13, Walter Bright <newshound2 digitalmars.com> wrote:
 Can I play, too? Mine from the Digital Mars C library. Haven't looked at it

 #if 0 /* Smaller but slower under many circumstances. */
 char *strstr(const char *s1,const char *s2)
 {   size_t len2;
       size_t len1;
       char c2 = *s2;

       len1 = strlen(s1);
       len2 = strlen(s2);
       if (!len2)
 	return (char *) s1;
       while (len2 <= len1)
       {
 	if (c2 == *s1)
 	    if (memcmp(s2,s1,len2) == 0)
 		return (char *) s1;
 	s1++;
 	len1--;
       }
       return NULL;
 }

Some things i've noticed about this implementation:

It's wrapped in #if 0 ... #endif ? :-)
 - c2 is assigned by pointer dereference, but then strlen is called on
 s2 and you may get an early exit, meaning c2 was not needed at this
 point. I'd move c2 below the strlen call.

Yup.
 - The memcp doesn't have to compare len2 amount of chars if you've
 already compared the first character, it can call memcmp(s2+1, s1+1,
 len2-1). Although I'm not sure whether it's faster to do it this way
 (maybe it is, if those variables are actually registers..).

Yup.
 And notes about both implementations:

 - You're casting a const char * to a char * in the return, you should
 instead provide an overload of strstr taking char* and returning
 char*.

The function bodies would be identical - why have two of them?
 - You seem to have missed the most basic and most important check. You
 didn't attempt to compare the memory address of both pointers against
 each other before doing any other work. If they both point to the same
 memory, the strings are guaranteed to be equal.

Most important? I've never even run across this case.
Jun 22 2013
parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/22/2013 12:12 PM, Andrej Mitrovic wrote:
 Most important? I've never even run across this case.

Ok, maybe not that important. It's likely a rare case. Btw, does anyone know if the typical memcmp implementation tries to compare the pointers' addresses first to see if they match?

I don't know about typical, but I've never seen one that did.
Jun 22 2013
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/21/13 4:02 PM, Walter Bright wrote:
 On 6/21/2013 3:35 PM, Andrei Alexandrescu wrote:
 On 6/21/13 3:22 PM, Adam D. Ruppe wrote:
 Just for laughs I just slapped together a strstr

Post it and I'll destroy it.

Can I play, too? Mine from the Digital Mars C library. Haven't looked at it since 2001. ================================================== /*_ strstr.c */ /* Copyright (C) 1985-2001 by Digital Mars */ /* All Rights Reserved */ /* www.digitalmars.com */ /* Written by Walter Bright */ #include <stdio.h> #include <ctype.h> #include <stddef.h> #include <string.h> #include <stdlib.h> #if 0 /* Smaller but slower under many circumstances. */ char *strstr(const char *s1,const char *s2) { size_t len2; size_t len1; char c2 = *s2; len1 = strlen(s1); len2 = strlen(s2); if (!len2) return (char *) s1; while (len2 <= len1) { if (c2 == *s1) if (memcmp(s2,s1,len2) == 0) return (char *) s1; s1++; len1--; } return NULL; } #else

I won't comment on the B-M implementation. This brute force implementation has the problem of calling strlen on both strings upfront, which is arguably unnecessary. Although it doesn't affect complexity, it is an inefficiency hard to recover from. (The tradeoff is that memcmp will help with that.) Andrei
Jun 23 2013
prev sibling parent reply =?UTF-8?B?IkrDqXLDtG1lIE0uIEJlcmdlciI=?= <jeberger free.fr> writes:
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Adam D. Ruppe wrote:
 On Friday, 21 June 2013 at 22:35:55 UTC, Andrei Alexandrescu=20
 wrote:
 Post it and I'll destroy it.

=20 inout(char)* mystrstr(inout(char)* haystack, const(char*) needle)=20 { assert(haystack !is null); =20 if(needle is null) return haystack; =20 const(char)* where =3D needle; inout(char)* gotit; =20 while(*haystack) { if(*haystack =3D=3D *where) { if(gotit is null) gotit =3D haystack; // store where the match started where++; haystack++; if(*where =3D=3D 0) return gotit; } else { // if we were in the middle of a match, we'll want to // check the current character again so only advance=20 if // we're at the beginning if(gotit is null) haystack++; else { // partial match, but not complete so no good // start over, including the current *haystack where =3D needle; gotit =3D null; } } } =20 return null; } =20

"ababc" in "abababc"... Jerome --=20 mailto:jeberger free.fr http://jeberger.free.fr Jabber: jeberger jabber.fr
Jun 22 2013
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/22/13 7:10 AM, Adam D. Ruppe wrote:
 On Saturday, 22 June 2013 at 13:55:26 UTC, Jérôme M. Berger wrote:
 I haven't tried running it, but this looks to me like it won't find
 "ababc" in "abababc"...

You're right. I should have went backwards all the way, not just in the one case. This passes all the tests:

It's still buggy. Overly complicated, too - at 21 lines, it's about twice as large as the canonical implementation. Andrei
Jun 22 2013
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/22/13 12:28 PM, Adam D. Ruppe wrote:
 On Saturday, 22 June 2013 at 16:38:31 UTC, Andrei Alexandrescu wrote:
 Huh, even the shortest impl I can think of is about the same length:

 inout(char)* mystrstr(inout(char)* haystack, const(char*) needle) {
 assert(haystack !is null);

 if(needle is null)
 return haystack;

 while(*haystack) {
 auto h = haystack;
 const(char)* n = needle;
 while(*n == *h && *n && *h) {
 h++;
 n++;
 }
 if(*n == 0)
 return haystack;
 haystack++;
 }

 return null;
 }


 I like this a lot better because it is more straightforward.

Still buggy. The empty string must be a prefix of any string including the empty string. I describe the canonical implementation of substring brute for string search as far as I see it http://www.serversidemagazine.com/news/10-questions-with-facebook-research-engineer-andrei-alexandrescu/. Andrei
Jun 23 2013
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/23/13 9:21 AM, Adam D. Ruppe wrote:
 http://arsdnet.net/dcode/dhp.d

 It works by just reading the file and translating everything outside the
 <?d?> into a giant writeln(string literal), pasting in the D code, then
 compile+running it, inserting a bunch of imports so it works.

 dom.d now supports those kinds of tags, it could probably be even nicer,
 especially with a D lexer so it doesn't think "?>" is the same as ?>.
 But as I'm sure I said then, this is a bad idea anyway since mixing code
 and html data leads to ugliness, whether the code is php or D.

Nice! (BTW: foreach(ln; stdin.byLine) { string line = ln.idup ~ "\n"; is simpler written as foreach(ln; stdin.byLine(KeepTerminator.yes)) { string line = ln.idup; .) Would be awesome if an Apache extension would make it trivial to write Web pages in D. Andrei
Jun 23 2013
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/23/13 10:34 AM, Adam D. Ruppe wrote:
 Just use cgi or fastcgi, both are really easy to configure on apache
 (often needing nothing more than copying your executable into /cgi-bin/,
 or adding three lines to ,htaccess), and being separate processes, if
 you crash them it is no big deal. No need to do set up a reverse proxy
 or get your system administrator to load strange new Apache modules.

We should do what php does, it was very successful. I assume it's a dynamic library.
 Or if you are just playing, you can skip apache altogether and use a D
 http server, like my cgi.d and vibe.d both offer (vibe.d's scales way
 better but mine is simpler).


 But I kinda want to play with this now with shared libraries just for
 something to do so maybe I will.

Awesome! The more I think of it the more I get to the conclusion I should migrate my own website (www.erdani.com) to a 100% D toolchain. Andrei
Jun 23 2013
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/23/13 11:04 AM, Dicebot wrote:
 On Sunday, 23 June 2013 at 17:54:01 UTC, Andrei Alexandrescu wrote:
 On 6/23/13 10:34 AM, Adam D. Ruppe wrote:
 We should do what php does, it was very successful. I assume it's a
 dynamic library.

And what reason behind this other than "millions of lemmings can't be wrong"? This approach is disaster when it comes to high load.

Would separate processes work better under high load? Educate me. Andrei
Jun 23 2013
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/23/13 11:51 AM, Adam D. Ruppe wrote:
 I think it is just an accident of history that mod_php ever got used.
 Classic cgi implementations were still slow enough (especially with an
 interpreted language) that people wanted to try something else, but the
 other world of options hadn't taken root yet either (I think mod_php
 even slightly predates fastcgi's introduction), and continues to exist
 just out of inertia.

OK so what's the way to go now? One process per request? Seems heavy to me seeing as most requests last very little. Andrei
Jun 23 2013
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-06-21 23:33, Andrei Alexandrescu wrote:

 If there's any need to reach for documentation, the interviewer has
 failed. When interviewing we (at Facebook) ask problems that are likely
 to appear in a normal day's work, but for which the typical libraries
 don't help. (E.g. many libraries don't can't help with implementing
 unstable remove (see std.algorithm).)

 Also it's fair to ask about implementing a stdlib function itself if the
 interview concerns some systems-level work; e.g. brute-force strstr() is
 fair game and I think any engineer should be able to lift it off the
 ground quickly (to my dismay, only a fraction can). Paradoxically use of
 stdlib functions may actually hurt; I've seen people who e.g. call
 strlen() in a loop in order to implement strstr()!

Yes, if they tell you that from the beginning. Instead they said: "solve it anyway you like", then they started to add restrictions. -- /Jacob Carlborg
Jun 22 2013
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/20/2013 8:50 PM, H. S. Teoh wrote:
 One of my previous supervisors told me that when he gets resumés, as
 soon as he sees "Ph.D" he chucks it straight into the trash.

He'd have missed out on Andrei, then.
Jun 21 2013
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/21/2013 12:49 PM, John Colvin wrote:
 If you want a normal programming job, you need to show more real world
 experience than a PhD, but just throwing out people who have proved their
 originality and in depth understanding of a topic through a PhD is nothing
short
 of absurd.

You need a mix of people - young and old, newbies and old hands, academics and mechanics, detail guys and big picture guys, to have a healthy dev organization.
Jun 21 2013
parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/21/2013 1:34 PM, H. S. Teoh wrote:
 But what actually happened was, nobody wanted to work on mundane
 projects -- maintaining existing systems, keeping the servers running,
 doing routine chores necessary for everyday operations, etc.. There was
 a lot of infighting on who gets to work on the fun stuff, and the people
 who didn't get it began to quit one by one. Eventually, none of the
 mundane things were adequately taken care of, and most of the employees
 left 'cos they didn't get to do what they wanted, and the company
 collapsed.

Yup. Similar things happen if you only hire people fresh out of college. Things are find for a while, and then everyone wants to move up into management at the same time.
Jun 21 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Jun 21, 2013 at 05:33:45PM -0400, Andrei Alexandrescu wrote:
[...]
 Also it's fair to ask about implementing a stdlib function itself if
 the interview concerns some systems-level work; e.g. brute-force
 strstr() is fair game and I think any engineer should be able to
 lift it off the ground quickly (to my dismay, only a fraction can).

Yeah, I was rather dismayed at how many people lacked basic understanding of the programming language they purport to be skilled in. Like not understanding variable shadowing in C, or simple closures in Javascript (not the evilly-nested convolution I tested for in a different test question :-P).
 Paradoxically use of stdlib functions may actually hurt; I've seen
 people who e.g. call strlen() in a loop in order to implement
 strstr()!

My favorite example of poor strlen() usage is writing "strlen()==0" in a loop condition. More typical causes of disqualification, though, are what one of my university professors affectionately called "squirrelly code": code that *probably* does what it's supposed to, but does so by roundabout, non-obvious ways, running around here and there doing little things that don't really relate directly to the problem at hand (like squirrels running around hunting for their forgotten stash of nuts). Code like this typically demonstrate a lack of understanding of programming in general, or lack of understanding of the problem at hand, or inability to identify and address the salient points of said problem. Typical symptoms include declaring and using useless variables, calling printf inside a function that shouldn't be doing I/O, doing unnecessary type conversions to perform trivial computations (converting int to string and attempting to do arithmetic on the digits in the string), odd usage of loops and conditionals in unexpected places, using comments to urge the computer to do what one wishes, etc.. The software analogue of Rube Goldberg machines, if you will. :-P T -- Век живи - век учись. А дураком помрёшь.
Jun 21 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Jun 21, 2013 at 06:12:57PM -0400, Nick Sabalausky wrote:
 On Fri, 21 Jun 2013 09:47:46 -0700
 "H. S. Teoh" <hsteoh quickfur.ath.cx> wrote:
 
 universities that actually *teach* real programming are more
 interested in finding solutions to uncomputable problems than
 teaching students how to solve computable ones
 

That doesn't match my experience (also in the US here).

Actually, I'm up in the Great White North, so my perception of "North America" may be a bit biased on the first word. :-P
 Granted, all my info is from ten years ago, but what I saw was mainly
 a bunch of what The "Joel on Software" guy called "Java Schools"
 <http://www.joelonsoftware.com/articles/ThePerilsofJavaSchools.html>.

LOL... that article made me laugh. Thanks for the great read!
 The following includes what I've seen at BGSU and OSU (party schools,
 not that I personally attended OSU, but I did have a friend that went
 through OSU's "RESOLVE/C++" stuff) and also JCU (a private univ that,
 at least around Cleveland, is highly-regarded by everyone except me).
 
 What I've seen at these places, and apparently many others from what I
 understand, is that while they *do* recognize the importance of
 creating programmers, the problems are:
 
 - The theory is minimal to make sure they get all those high-paying
   students through the revolving door.

Huh. I had the opposite experience. We were pounded with so much theory from day one, and then summarily executed on our Computability Theory finals (the infamous exam that was so hard, you were allowed to bring *anything* -- any textbook, reference book, your desktop PC if you were so inclined -- and none of it would help), that I came out with a rather strong distaste for anything that mentions the word "computability", lasting for quite a while before it wore off.
 - Tests and exams do *not* teach people. An yet, that's where the
   emphasis is, instead of on instruction.
 
 - The really *big* issue: You just simply CANNOT expect people to go
   from beginner to competent programmer when they spend *at most*
   one-third of their credits, and about 3 hours a week, over a mere 4
   years on actual programming material instead of irrelevant liberal
   arts garbage that *belongs* in high school, not a college so-called
   "major".

That, I have to agree. I wanted to double-major in CS and Chemistry, but gave up on the first day in 2nd year when I looked at my course / lab / programming project schedule, and said, "that's not gonna work". Without adequate time actually spent programming, it's impossible to get good at it.
 - The nasty little details like pointer/memory problems, linker errors,
   etc that real people have to deal with are neatly glossed over and
   sidestepped.

OK, that's one of my pet peeves. Back in the day, I was taught pointers (though not linker errors -- I learnt that on my own) and recursion. Nowadays, people hear the word "pointer" and go "ewwww", and when you say "recursion" their eyes glaze over. What on earth did they do to the CS programs?!
 - There was one CS101 teacher (I had to tutor her unfortunate students)
   who constantly bragged about being from a real-world software
   company...but she was a Java-addict (circa v1.2-v1.4) who kept trying
   to teach OO *before* basic flow-of-execution. Consequently, none of
   her unfortunate students had the slightest clue what was going on.

Ugh. Talk about putting the cart before the horse...
 - Many of the professors are terrible programmers themselves. For
   example, I had one who openly admitted the only language he knew was
   C, and yet at one point it became painfully obvious that he had
   almost no comprehension of null-terminated strings.

That's the beef I have with over-emphasis on CS theory. There's nothing wrong with theory in and of itself -- in fact it's foundational and very much indispensible -- but when you become so detached from reality that you think in terms of pure idealizations -- when you can *only* think in terms of pure idealizations -- that you can't even write a single line of real code without some external help, then something has clearly gone very, very wrong.
 - Many of the teachers don't even teach, they just collect the
   thousands of dollars in tuition and give you a book recommendation
   (really more of a "demand" than a recommendation).

That's because the incentives are all wrong. Professors aren't paid to teach; they're paid to produce research. Publish or perish, so goes the saying in those circles. To them, teaching is an additional burden imposed upon them that they'd rather get over with ASAP and get back to their research, whatever it takes. Turn away the students showing up at your office hours. Bore them to death in class so they wouldn't know *what* to ask even if they wanted to. Read from a photocopy of the textbook word-for-word to pass lecture time with zero effort (I've actually been in a class where this was done). Anything, to get it over with and get back to the work that pays. Some of my best teachers in university were part-time lecturers, one of whom won several teaching awards and accolades for 3 years straight. I don't know if there's even one faculty member that ever got *one* teaching award. (On the contrary, a certain faculty member was so arrogant of his tenurehood that he'd show up on evaluation day and tell students to their face that they can write whatever they want and it wouldn't affect him in the least, 'cos his tenure meant he can never be fired. And of course, he consistently gets rock-bottom reviews from students, and incoming students are consistently warned by said students to avoid his courses at all costs.)
   Now, I'm a strong believer in being self-taught and learning from
   books, but all I need for that is a library card, not a $100k debt
   and four years of elitist attitudes from people who clearly don't
   know what they're doing anyway.

Heh. Nearly all of my programming skills are self-taught (well, and learned from experience now that I have some number of years in the industry), but I'm no reader either. I was doing online learning long before the 'Net became cool, and every now and then I still browse around learning new algorithms and stuff, while everybody else is clicking their lives away on FB and twitter (no offense, Andrei, but I do think FB is an evil waste of time, at least when it comes to the way most people use it). T -- If it tastes good, it's probably bad for you.
Jun 21 2013
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/21/2013 4:10 AM, Paulo Pinto wrote:
 Except not everyone has the authorization to place their work code in such
 public places nor the availability or desire to code after work, just to please
 job interviewers.

True, but your odds of being 'discovered' go up enormously if you make such an effort to make yourself findable with a professional online persona. It reminds me of some music superstars who started out singing at shopping malls, dive bars, weddings, anyplace they could get anyone to listen. They'd do this grind for a couple years until someone discovered them. The Beatles is one of those that comes to mind who did this. Anyhow, I recommend that anyone who wants to promote their 'brand' and increase their prospects do: 1. get a domain name that is "yourname.com" and put on it your resume, links to significant things you've done, etc. 2. use your own name as your handle when posting professionally 3. submit presentation proposals about your work to conferences 4. blog or otherwise write articles about your work or some neat tech thing you figured out 5. get a twitter account & github account in your own name - even if you don't use it, reserve it for when you do want to etc. Essentially, you want to own your online persona. Read any of Donald Trump's books on how he operates, it's the same thing. Gene Simmons is another one who excels at promoting himself.
Jun 27 2013
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/20/2013 2:49 AM, Jacob Carlborg wrote:
 On 2013-06-19 23:54, Walter Bright wrote:

 TDD strikes me as an ad-hoc way of constructing code, and is a poor
 substitute for thinking about the problem as a whole. For example, I
 don't really see how getting a square root function to pass its test
 cases is going to lead one to implementing one of the classic algorithms
 for computing square roots. I don't see how TDD for a compiler will lead
 one to rediscover what is known about the best way to organize the logic
 of a compiler.

 TDD is "curve fitting" which is what one does when one has no
 understanding.

I have not watch the video(s) but I think you completely failed to understand what TDD is about. This is how I think about TDD: 1. You have a problem to solve 2. You think about how to solve the problem and how a design can look like 3. You come up with a design 4. Write a test according to the design 5. Write the implementation 6. Run the test to see that the implementation matches your design 7. Repeat 2-6 until satisfied

Uncle Bob's 3 laws of TDD: 1. You are not allowed to write any production code unless it is to make a failing unit test pass. 2. You are not allowed to write any more of a unit test than is sufficient to fail; and compilation failures are failures. 3. You are not allowed to write any more production code than is sufficient to pass the one failing unit test. http://butunclebob.com/ArticleS.UncleBob.TheThreeRulesOfTdd Your procedure is much more reasonable, but I suspect the latter is more "principled" TDD, although it is fairly absurd as it implies a complete lack of understanding or design, and was what I was referring to. And, btw, your lexer example of TDD is exactly what I think is wrong with TDD.
Jun 20 2013
parent reply Jacob Carlborg <doob me.com> writes:
On 2013-06-20 20:44, Walter Bright wrote:

 Uncle Bob's 3 laws of TDD:

 1. You are not allowed to write any production code unless it is to make
 a failing unit test pass.

 2. You are not allowed to write any more of a unit test than is
 sufficient to fail; and compilation failures are failures.

 3. You are not allowed to write any more production code than is
 sufficient to pass the one failing unit test.

 http://butunclebob.com/ArticleS.UncleBob.TheThreeRulesOfTdd

 Your procedure is much more reasonable, but I suspect the latter is more
 "principled" TDD, although it is fairly absurd as it implies a complete
 lack of understanding or design, and was what I was referring to.

I don't care about Uncle Bob. I say use your common sense.
 And, btw, your lexer example of TDD is exactly what I think is wrong
 with TDD.

I never write those dummy implementation, I really don't see the point. -- /Jacob Carlborg
Jun 20 2013
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/20/2013 12:51 PM, Jacob Carlborg wrote:
 I say use your common sense.

Exactly. TDD strikes me as one of those endless attempts to substitute a rigid set of rules for thought and common sense.
 I never write those dummy implementation, I really don't see the point.

Well, it was your example :-)
Jun 20 2013
parent reply Jacob Carlborg <doob me.com> writes:
On 2013-06-20 22:22, Walter Bright wrote:

 Well, it was your example :-)

You got me there :) That's how you're supposed to write TDD according most strong believers. But I do some researcher about TDD and similar techniques, use my common sense, pick some pieces from here and there and use what I think works. But I also do think that it can be a good idea to at least once or twice exactly follow these religious believers and see how it works. Then make my own decisions based on that. -- /Jacob Carlborg
Jun 21 2013
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/21/2013 3:04 AM, Jacob Carlborg wrote:
 But I do some researcher about TDD and similar techniques, use
 my common sense, pick some pieces from here and there and use what I think
works.

I've been around long enough to have seen an endless parade of magic new techniques du jour, most of which purport to remove the necessity of thought about your programming problem. In the end they wind up contributing one or two pieces to the collective wisdom, and fade away in the rearview mirror. But there is one technique that stands head and shoulders above the others in improving code quality, sometimes dramatically so. That's unit testing coupled with a coverage analyzer. I can't even count the number of times I thought "this is so simple, I couldn't have gotten it wrong, I doan need no steeenkin unit tests." But I write a couple anyway, and you can guess the rest. Whether you write the unit tests before, during, or after the module is written is irrelevant.
Jun 21 2013
parent reply Jacob Carlborg <doob me.com> writes:
On 2013-06-21 21:11, Walter Bright wrote:

 I've been around long enough to have seen an endless parade of magic new
 techniques du jour, most of which purport to remove the necessity of
 thought about your programming problem.

One needs to be opened minded and be able to try new things.
 But there is one technique that stands head and shoulders above the
 others in improving code quality, sometimes dramatically so. That's unit
 testing coupled with a coverage analyzer.

I can imagine that these techniques have been new at some point in time as well. It's possible that a new technique comes a long that help you as much as these techniques do. -- /Jacob Carlborg
Jun 22 2013
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/22/13 5:54 AM, Jacob Carlborg wrote:
 On 2013-06-21 21:11, Walter Bright wrote:

 I've been around long enough to have seen an endless parade of magic new
 techniques du jour, most of which purport to remove the necessity of
 thought about your programming problem.

One needs to be opened minded and be able to try new things.
 But there is one technique that stands head and shoulders above the
 others in improving code quality, sometimes dramatically so. That's unit
 testing coupled with a coverage analyzer.

I can imagine that these techniques have been new at some point in time as well. It's possible that a new technique comes a long that help you as much as these techniques do.

Was new in 1962: http://www.dtic.mil/dtic/tr/fulltext/u2/282767.pdf. Andrei
Jun 22 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Jun 21, 2013 at 11:59:12AM +0200, Jacob Carlborg wrote:
 On 2013-06-21 07:57, H. S. Teoh wrote:
 
we got to ask HR to first administer a technical test before any
interviews are arranged; test results are reviewed before deciding to
interview the candidate

I done tests like that, they all suck. This is how it usually works: You get a problem to solve. They say: "solve it anyway you want, using any language you want".

Heh, at my job, we're looking for skill in specific languages, so candidates are explicitly asked to write C code. And this is done not in an interview setting (which puts too much pressure on the candidate -- can *you* write code in front of interviewers without panicking at least a bit?), but in a classroom setting. Though, granted, when *I* had to go through the procedure, it felt like I was taking my CS final exams all over again. Not exactly a pleasant experience. :-P
 You solve it farily quickly and straight forward. Then they say, "you
 are not allowed to use that function". Basically they're saying it's
 cheating. Then you remove that function and change the implementation
 accordingly. Then they say, "you are not allowed to use that other
 function". You change the code accordingly and this dance continues.
 Then you're thinking to yourself, "Am I supposed to reimplement the
 standard library?".

That's the sign of a poorly-chosen test question. The kind of question I like to use doesn't focus on nitpicky details like that, but on actual algorithms or domain-specific knowledge, like showing that the candidate understands bit operations, pointer arithmetic, and stuff like that. On that note, it's rather shocking how few candidates know how to use malloc() and free() correctly... That's not to say the tests my employer gives have only "good" questions, though; there are some that basically do what you describe: "please re-implement strstr(), please concatenate strings without using strcat()", and the like. When I was involved in hiring, I usually ignore these questions when evaluating the candidate, or cut them a lot of slack. But I do have a slightly evil streak when it comes to technical tests... once I wrote a Javascript question that requires the candidate to evaluate what a piece of code does, that involves several nested closures and variable shadowing. We hired one of the (very) few candidates who actually got that question right, and she turned out to be an excellent coder.
 I would say, most times, it's almost irresponsible to _not_ use the
 standard library. A bunch of people, a lot smarter than I, have had
 a good 20 years or so to perfect and fine tune the standard library.
 Not a chance in hell that I'm going to beat that, on an interview,
 with code written on a whiteboard.

Well, I don't think the point of the test is to prove that you can write code that's *superior* to the standard library, but that you *can* write non-trivial functionality from ground up.
 A side note. You're of curse not allowed to look at any documentation
 at all and you're not allowed to use a text editor.  You write the
 code on a whiteboard, yes a _whiteboard_.

Well, that's a new one. :-P Our tests are administered as pen-and-paper tests. Not unlike CS final exams, unfortunately. :-/
 That's just so stupid. That's not how programming works in the real
 world. Why the hell do they think IDE's have built in, easy to
 read/find, documentation.

Actually, we instruct HR to provide the standard C library manuals if candidates ask for it. There *are* some unfair questions that expect you to know, e.g., the exact order of arguments to select() and stuff like that, but like I said, it's not HR who evaluates the tests, but the hiring managers, so it's up to our discretion how much weight we put on questions like that. We usually disregard minor mistakes like using the wrong order of arguments to qsort(), if the intent is clear. We don't (or at least I don't) just blindly look for the highest-scoring test and hire those candidates; I usually take the time to look at the code and ask, "why did they write it that way? what does that show about their coding abilities?". T -- I see that you JS got Bach.
Jun 21 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Jun 21, 2013 at 01:11:44PM -0700, Walter Bright wrote:
 On 6/21/2013 12:49 PM, John Colvin wrote:
If you want a normal programming job, you need to show more real
world experience than a PhD, but just throwing out people who have
proved their originality and in depth understanding of a topic
through a PhD is nothing short of absurd.

You need a mix of people - young and old, newbies and old hands, academics and mechanics, detail guys and big picture guys, to have a healthy dev organization.

This reminds me of a HR seminar I attended once, where an example was given of a particular (real) company who only ever hired top-notch programmers. They set their standards very high, and have a review system that would basically fire anyone whose skills are sub-par. The guys in charge thought this was going to skyrocket the company into success. But what actually happened was, nobody wanted to work on mundane projects -- maintaining existing systems, keeping the servers running, doing routine chores necessary for everyday operations, etc.. There was a lot of infighting on who gets to work on the fun stuff, and the people who didn't get it began to quit one by one. Eventually, none of the mundane things were adequately taken care of, and most of the employees left 'cos they didn't get to do what they wanted, and the company collapsed. T -- Many open minds should be closed for repairs. -- K5 user
Jun 21 2013
prev sibling next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Fri, 21 Jun 2013 16:03:57 -0700
"H. S. Teoh" <hsteoh quickfur.ath.cx> wrote:

 On Fri, Jun 21, 2013 at 06:12:57PM -0400, Nick Sabalausky wrote:
 - Many of the professors are terrible programmers themselves. For
   example, I had one who openly admitted the only language he knew
 was C, and yet at one point it became painfully obvious that he had
   almost no comprehension of null-terminated strings.

That's the beef I have with over-emphasis on CS theory. There's nothing wrong with theory in and of itself -- in fact it's foundational and very much indispensible -- but when you become so detached from reality that you think in terms of pure idealizations -- when you can *only* think in terms of pure idealizations -- that you can't even write a single line of real code without some external help, then something has clearly gone very, very wrong.

Yea. I'm convinced that the theory and the practical are both critically important. Either one is only minimally useful without the other.
 
 - Many of the teachers don't even teach, they just collect the
   thousands of dollars in tuition and give you a book recommendation
   (really more of a "demand" than a recommendation).

That's because the incentives are all wrong. Professors aren't paid to teach; they're paid to produce research. Publish or perish, so goes the saying in those circles. To them, teaching is an additional burden imposed upon them that they'd rather get over with ASAP and get back to their research, whatever it takes. Turn away the students showing up at your office hours. Bore them to death in class so they wouldn't know *what* to ask even if they wanted to. Read from a photocopy of the textbook word-for-word to pass lecture time with zero effort (I've actually been in a class where this was done). Anything, to get it over with and get back to the work that pays.

That's a good point. Although it doesn't help that (from what I've seen) the students are given basically no indication ever that the profs are there for anything other than teaching. As far as the "school" ever indicates to the students, it's a "school" rather than an R&D lab. And that's a natural conclusion for the student to make when they're forking out $100k for an alleged "education".
 Some of my best teachers in university were part-time lecturers, one
 of whom won several teaching awards and accolades for 3 years
 straight. I don't know if there's even one faculty member that ever
 got *one* teaching award.

You know, I honestly noticed the same thing. I always thought it was...interesting...in college that the best teachers I had were almost exclusively the part-timers, and vice versa.
 (On the contrary, a certain faculty member
 was so arrogant of his tenurehood that he'd show up on evaluation day
 and tell students to their face that they can write whatever they
 want and it wouldn't affect him in the least, 'cos his tenure meant
 he can never be fired. And of course, he consistently gets
 rock-bottom reviews from students, and incoming students are
 consistently warned by said students to avoid his courses at all
 costs.)
 

Tenure is a concept I've *always* despised. May as well just hand out Royalty status to people. It ends up the pretty much the same way.
 
   Now, I'm a strong believer in being self-taught and learning from
   books, but all I need for that is a library card, not a $100k debt
   and four years of elitist attitudes from people who clearly don't
   know what they're doing anyway.

Heh. Nearly all of my programming skills are self-taught (well, and learned from experience now that I have some number of years in the industry), but I'm no reader either. I was doing online learning long before the 'Net became cool,

Actually, my first introduction to programming was the interactive tutorial disks that came with the Apple IIc. I sometimes find it kind of depressing that instruction isn't even *that* far along anymore, let alone any further advanced.
 and every now and then I still browse
 around learning new algorithms and stuff, while everybody else is
 clicking their lives away on FB and twitter (no offense, Andrei, but I
 do think FB is an evil waste of time, at least when it comes to the
 way most people use it).
 

My biggest issues with FB/Twit are that: A. They degenerate the internet into proprietary walled-gardens (it's internet w/ orwellian training wheels, or a Dvorak put it: It's "New AOL") B. They've managed to turn every product and advertisement, ever, into their *own* advertisement with the "f" and "t" plastered everywhere you look. I even have a granola bar sitting here with FB's and Twit's advertising plastered on the wrapper, for fuck's sake. I wouldn't mind the twitfaces so much if I were able to opt-out of their constant bombardment from every fucking facet of society. But yea, the fact that, for the most part, they ultimately amount to LiveJournal anyway (itself an abomination of pointless self-indulgent minutia and drivel) doesn't help matters.
Jun 26 2013
prev sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Wed, 26 Jun 2013 16:24:27 -0400
Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> wrote:

 On Fri, 21 Jun 2013 16:03:57 -0700
 "H. S. Teoh" <hsteoh quickfur.ath.cx> wrote:
 and every now and then I still browse
 around learning new algorithms and stuff, while everybody else is
 clicking their lives away on FB and twitter (no offense, Andrei,
 but I do think FB is an evil waste of time, at least when it comes
 to the way most people use it).
 

My biggest issues with FB/Twit are that: A. They degenerate the internet into proprietary walled-gardens (it's internet w/ orwellian training wheels, or a Dvorak put it: It's "New AOL") B. They've managed to turn every product and advertisement, ever, into their *own* advertisement with the "f" and "t" plastered everywhere you look. I even have a granola bar sitting here with FB's and Twit's advertising plastered on the wrapper, for fuck's sake. I wouldn't mind the twitfaces so much if I were able to opt-out of their constant bombardment from every fucking facet of society. But yea, the fact that, for the most part, they ultimately amount to LiveJournal anyway (itself an abomination of pointless self-indulgent minutia and drivel) doesn't help matters.

Again though, like you said, nothing against Andrei. I completely understand the difference between a person and their employer. Hell, I used to work for a [car] insurance company!
Jun 26 2013
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-06-20 00:23, Szymon Gatner wrote:

 TDD claims to help with this by forcing you to code client code first
 (the test) which arguably leads to better decoupling of components. You
 don't have to split components for reuse later because they are already
 being used in unrelated contexts in tests.

This is a very good point. -- /Jacob Carlborg
Jun 20 2013
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 06/19/2013 11:54 PM, Walter Bright wrote:
 On 6/19/2013 4:01 AM, Szymon Gatner wrote:
 This is not strictly D related but I am very curious about D's
 community opinion
 on the points made by non other than Jim Coplien here:

TDD strikes me as an ad-hoc way of constructing code, and is a poor substitute for thinking about the problem as a whole. For example, I don't really see how getting a square root function to pass its test cases is going to lead one to implementing one of the classic algorithms for computing square roots. I don't see how TDD for a compiler will lead one to rediscover what is known about the best way to organize the logic of a compiler. TDD is "curve fitting" which is what one does when one has no understanding.

Concrete example in DMD: 'forward reference error'.
 D's support of unit testing is not an endorsement of TDD.

Jun 20 2013
prev sibling next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, June 19, 2013 22:27:57 Joseph Rushton Wakeling wrote:
 On 06/19/2013 08:58 PM, Jonathan M Davis wrote:
 Exactly. With TDD, you write the tests and then write the code

Isn't this what is highlighted in the second article linked to by the OP, as being precisely one of the misinterpretations of TDD?

I haven't read the article yet, so I don't know.
 I'm not sure if I've misinterpreted you here, because one needs to
 distinguish between "write all tests first, then code" (which is not what
 TDD means) and the rather tighter iteration/co-evolution of test and code
 writing which is supposed to characterize TDD. If you meant the latter
 then I apologize.

It's not necessarily the case that you write _all_ of the tests and then all of the code, but as I understand it, with TDD, you write a test for a function before you write that function (though both could be written well before other functions and whatever tests go with them). And I completely disagree with the idea of writing a test for a piece of code that doesn't exist yet, even if you're going to write it immediately after writing the test. I _do_ agree with writing the tests fora function as soon as the function is done, in which case, you're likely going to have to do more work on the function, since it'll probably fail the test, and you'll probably improve the tests some more at that point as well. But I completely disagree with writing the test before the code, which is one of the key features of TDD as its always been explained to me. - Jonathan M Davis
Jun 19 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-06-20 00:34, Jonathan M Davis wrote:

 So, I think that it's clear that I will never agree with TDD. If someone else
 wants to code that way, and it works for them, great, but it's not how I want
 to function.

No, but that's how your functions should function :) -- /Jacob Carlborg
Jun 20 2013
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, June 19, 2013 14:19:28 H. S. Teoh wrote:
 Doesn't TDD stand for Test-Driven *Design*?

It was my understanding that it was Test-Driven "Development," but I could be wrong. Let's see... Well, according to Wikipedia, it's development, not design: http://en.wikipedia.org/wiki/Test-driven_development - Jonathan M Davis
Jun 19 2013
prev sibling next sibling parent "Szymon Gatner" <noemail gmail.com> writes:
On Wednesday, 19 June 2013 at 21:21:09 UTC, H. S. Teoh wrote:
 [...]

 Doesn't TDD stand for Test-Driven *Design*?

It stands for Test-Driven "Development". Often people focus on the word "test" too much forgetting that is is actually only a "force" that drives development. TDD is not the same as Unit Testing which is all about tests. Definitions may vary but TDD folk agree that it is about evolving the design incrementally, test by test (test comes of course fists and represents new feature request) and abandoning "big upfront design". That quote actually comes from uncle Bob himself - one of the authors of Agile Manifesto and big TDD advocate. In the talk I originally linked Jim actually openly disagrees with uncle Bob on a claim that TDD improves quality of the code. And by quality they mean quality of the design not run-time correctness.
Jun 19 2013
prev sibling next sibling parent "Szymon Gatner" <noemail gmail.com> writes:
On Wednesday, 19 June 2013 at 21:59:21 UTC, Jonathan M Davis 
wrote:
 I _do_ agree with writing the tests fora function as soon as 
 the function is
 done, in which case, you're likely going to have to do more 
 work on the
 function, since it'll probably fail the test, and you'll 
 probably improve the
 tests some more at that point as well. But I completely 
 disagree with writing
 the test before the code, which is one of the key features of 
 TDD as its
 always been explained to me.

Writing the test before writing the function is exactly the point of TDD. It forces you to think about parameters it should take and value(s) it should return first. You first write the code that uses that function and only when client code looks ok and and all requirements are understood, only then you implement the feature. API change cost is way higher than fixing a bug inside a function or a class, also you will often realize that what you initially though was necessary turns out not to be after all.
Jun 19 2013
prev sibling next sibling parent "Szymon Gatner" <noemail gmail.com> writes:
On Wednesday, 19 June 2013 at 21:54:04 UTC, Walter Bright wrote:
 On 6/19/2013 4:01 AM, Szymon Gatner wrote:
 This is not strictly D related but I am very curious about D's 
 community opinion
 on the points made by non other than Jim Coplien here:

TDD strikes me as an ad-hoc way of constructing code, and is a poor substitute for thinking about the problem as a whole. For example, I don't really see how getting a square root function to pass its test cases is going to lead one to implementing one of the classic algorithms for computing square roots. I don't see how TDD for a compiler will lead one to rediscover what is known about the best way to organize the logic of a compiler. TDD is "curve fitting" which is what one does when one has no understanding. D's support of unit testing is not an endorsement of TDD.

Point of TDD for a square root function would be to create a good API for getting a square root of a number. Implementation is just a detail. Good class design is way harder to achieve then correct implementation. Libraries become unusable in time not because they did what they did wrong but because they were badly structured / designed. TDD claims to help with this by forcing you to code client code first (the test) which arguably leads to better decoupling of components. You don't have to split components for reuse later because they are already being used in unrelated contexts in tests.
Jun 19 2013
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, June 20, 2013 00:15:50 Szymon Gatner wrote:
 Writing the test before writing the function is exactly the point
 of TDD.

And that's precisely what I disagree with. I will _never_ function that way.
 It forces you to think about parameters it should take
 and value(s) it should return first.

Which you should be doing anyway. That's part of designing the class of function. I don't need tests to do that. The tests are to verify that what I wrote work correctly. That's it. So, I think that it's clear that I will never agree with TDD. If someone else wants to code that way, and it works for them, great, but it's not how I want to function. - Jonathan M Davis
Jun 19 2013
prev sibling next sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Wed, 19 Jun 2013 13:01:02 +0200
"Szymon Gatner" <noemail gmail.com> wrote:

 This is not strictly D related but I am very curious about D's 
 community opinion on the points made by non other than Jim 
 Coplien here:
 
 http://www.tele-task.de/archive/video/flash/16130/
 
 D is the only language (that I am aware of) that has first class 
 unit testing support. What do you think? Do we really just 
 "mentally masturbate"?
 
 Article about the myths of TDD referenced in the talk:
 
 http://digitalcommons.calpoly.edu/cgi/viewcontent.cgi?article=1027&context=csse_fac

The main problem I have with TDD is that it's simply the latest form of ideology-oriented-development. It follows the same pattern as OO in the late 90's, and the popularity of Haskell and such a few years later: 1. Take a good idea (objects, functional programming, unittests, short iterative cycles, etc, whatever). 2. Decide "If using some of this is good, then forcing everything into this mold must be fantastic!" 3. Some die-hards sip the kool-aid, but the rest just adapt the good parts pragmatically, stripped of the ideological baggage. 4. The die-hards run into problems eventually forcing the more level-headed among them to admit "yea, turns out there are limitations to this alleged silver bullet after all". Meanwhile everyone else continues to just enjoy the benefits because they were sensible enough to only apply the idea in situations where it actually fit in the first place. Unittests are great. Minimizing the amount of code changes in each iteration of your code-compile-test cycle is great. But there are *always* other concerns, and those concerns must always be balanced. For example: - As people have said, some things aren't well-suited to automated testing. - Sometimes you run into something that just *is* a big all-at-once change where splitting it into smaller tasks, while being ideal, would be impractical. - Writing a unittest first forces the API to be designed before the implementation is written. But implementation is necessary to flush out unexpected design requirements (If you think you can always come up with an appropriate design and interface before implementing, then you're just plain wrong). Sometimes the appropriate design and API is obvious. Sometimes it isn't. When it isn't, then TDD skirts dangerously close to some of the problems of "Waterfall Model". Sure, TDD advocates refactoring-as-needed, but I can do that with or without TDD.
Jun 19 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-06-20 00:47, Nick Sabalausky wrote:

 - Writing a unittest first forces the API to be designed before the
    implementation is written. But implementation is necessary to flush
    out unexpected design requirements (If you think you can always
    come up with an appropriate design and interface before
    implementing, then you're just plain wrong). Sometimes the
    appropriate design and API is obvious. Sometimes it isn't. When it
    isn't, then TDD skirts dangerously close to some of the problems of
    "Waterfall Model". Sure, TDD advocates refactoring-as-needed, but I
    can do that with or without TDD.

Just because you have written a test doesn't mean you cannot change it. Perhaps you come up with a better API design, then change the tests. -- /Jacob Carlborg
Jun 20 2013
prev sibling next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 06/19/2013 10:59 PM, Jonathan M Davis wrote:
 It's not necessarily the case that you write _all_ of the tests and then all 
 of the code, but as I understand it, with TDD, you write a test for a function 
 before you write that function (though both could be written well before other 
 functions and whatever tests go with them). And I completely disagree with the 
 idea of writing a test for a piece of code that doesn't exist yet, even if 
 you're going to write it immediately after writing the test.

My impression has been that in practice, in good TDD environments, it seems to work in a more co-evolutionary way than that, with code and tests being written fairly closely together. I don't carry any particular torch for it, but if you do treat it as a co-evolutionary process, it dovetails quite nicely with some modern practices in industrial safety case management (the tradition was, prepare the design, then prepare the safety case, whereas modern practice is inclining towards developing the design and the safety case together).
 I _do_ agree with writing the tests fora function as soon as the function is 
 done, in which case, you're likely going to have to do more work on the 
 function, since it'll probably fail the test, and you'll probably improve the 
 tests some more at that point as well. But I completely disagree with writing 
 the test before the code, which is one of the key features of TDD as its 
 always been explained to me.

Well, it occurred to me that with a bit more up-front testing thought, the bug in RandomSample that we discussed recently might never have happened -- it feels more like the tests are playing catch-up to the code, rather than the code to the testable design decisions.
Jun 19 2013
prev sibling next sibling parent "Szymon Gatner" <noemail gmail.com> writes:
On Wednesday, 19 June 2013 at 22:35:11 UTC, Jonathan M Davis 
wrote:
 On Thursday, June 20, 2013 00:15:50 Szymon Gatner wrote:
 Writing the test before writing the function is exactly the 
 point
 of TDD.

And that's precisely what I disagree with. I will _never_ function that way.
 It forces you to think about parameters it should take
 and value(s) it should return first.

Which you should be doing anyway. That's part of designing the class of function. I don't need tests to do that. The tests are to verify that what I wrote work correctly. That's it.

Thing is TDD enforces such thinking. You are experienced programmer and you probably can figure out good API and proper implementation fast but young programmers can't. TDD forces them to came up with simple designs because otherwise they would not be able to pass own test! Truth is it is not only about young inexperienced programmers. There are awesome implementation programmers out there. They will squeeze last bit of power from the cpu in tight loop but just can't seem to organize their code so that it is reusable. TDD is suppose to help them too.
 So, I think that it's clear that I will never agree with TDD. 
 If someone else
 wants to code that way, and it works for them, great, but it's 
 not how I want
 to function.

Anyway perfectly understandable POV. TDD is just a methodology. I myself not too long ago though it was *eghm* stupid to say the least. Why would I write the code entirely other way I was doing it until now? I am new to TDD but I must say I think it works for me. I am not ashamed to admit I am not to good in designing high level pieces of code. I know how to implement stuff but weeks/months later when requirements change I am always struggling with what i've done. Finally I see reusable pieces of code that I know I will be using in other possibly completely unrelated projects. Maybe TDD is just for stupid ppl like me? :P
Jun 19 2013
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, June 19, 2013 23:51:42 Joseph Rushton Wakeling wrote:
 Well, it occurred to me that with a bit more up-front testing thought, the
 bug in RandomSample that we discussed recently might never have happened --
 it feels more like the tests are playing catch-up to the code, rather than
 the code to the testable design decisions.

Well, I think that if it's being done right, the tests are fairly complete before the code is ever commited, and there are plenty of cases where Phobos has pretty sparse testing. However, even if you have solid testing, you sometimes miss stuff and have to add further tests later. Now, if what you're referring to is the issue where a function which used save used isInputRange in its template constraint, I would say that that was a fairly clear case of having inadequate tests before the code was submitted. But std.random is one of the older range-based modules, and we've come a long way since then (and we're still improving), so it doesn't surprise me at all if it's lacking some of the basic tests that it should have. - Jonathan M Davis
Jun 19 2013
prev sibling next sibling parent "Chris Cain" <clcain uncg.edu> writes:
On Wednesday, 19 June 2013 at 23:44:29 UTC, Walter Bright wrote:
 I just can't accept that. For one thing, implementation details 
 often must drive the interface. Just writing specs without any 
 knowledge of how it would be implemented will not produce an 
 efficient design.

 For the square root, there's a definite tradeoff between 
 accuracy and speed. With no knowledge of those tradeoffs, and 
 just coming up with a spec, how can you make the right 
 decisions?

I don't think he's suggesting that it's not important how it's implemented. He's saying that as far as TDD goes, it's not concerned with those details. TDD is not going to solve all specification issues. Personally, I agree with you that square root functions and similar "low-level" APIs probably have a different set of needs to satisfy. Realistically, I don't think anyone would actually TDD something like a square root function. Personally, I'm a big fan of TDD in general, but I think it's one of the most often misunderstood things in the field of programming. It's like if you ask a group of religious people to define what their deity wants from them... everyone's going to have a different response and everyone is going to claim that theirs is the "only one true way." So, here's my "only one true way" of TDD (tongue in cheek): TDD (Test Driven Development) has a terrible name. Too many people think it's about the tests. Tests are certainly a good side effect of TDD and I certainly can't suggest they don't have value, but the fact that tests are written is really irrelevant to what you are _really_ trying to get out of TDD. There's been numerous attempts to rebrand it (Test Driven Design, Behavior Driven Development, and Behavior Driven Design to name a few). Ultimately, the "goal" of TDD is to have the programmer sit down and look at code from a USER'S (as in, user of the API) point of view _FIRST_. "Writing a test" as the first step is really just suggesting "as a user, how would you find it convenient to use this thing?" That's really just about all there is to it. It's one of those things that helps guide your thought process. If you're already doing that without writing tests first, then great. I've always found it pretty helpful to play around with how I want code to look like before I actually code the implementation details, though. More often I can come up with a simpler and more robust design after playing with how I want the end result to look like. As a cool bonus, once everything is done, you'll have a "specification" showing all of the features of a class/package/module and how everything is used. It will also show what kinds of preconditions are necessary for using those things (for instance, does your class need particular services to be online for it to work? If so, it'll show up in the tests). The reason why a square root function seems pointless to TDD to me is because the whole reason you'd TDD it in the first place is to figure out how a user should use it. And as far as I'm concerned, if you have any ideas other than "sqrt(some floating point number) -> returns a floating point number of the same precision as the argument which if multiplied by itself is (nearly) the same as the original argument" you're likely to get a few strange looks. That problem is solved. The only questions that might remain is "which package should this be located in?" which isn't really a question that TDD is meant to solve. TDD gets more useful once you start designing a larger system (especially if you're not experienced with designing a bigger system). If you're a fan of SOLID or some of the other principles of design, you'll likely see that code developed in TDD will naturally give rise to some "good practices". For instance, a major one is IOC/DI. It's much harder to write "testable" code that doesn't use DI or some technique similar. If you're using static classes/singletons you'll almost certainly have a lot of trouble. More often than not, I see people claiming something isn't testable when they're using things like singletons or shared resources that really don't need to be shared in the end, but using TDD would have highlighted that issue in the first place. That said, if you're experienced you would have probably not made that mistake, so maybe it's not that important for everyone. Anyway, TDD isn't BS, but I do think its misused. The misusage of it is (sometimes) BS, for sure.
Jun 19 2013
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, June 20, 2013 00:55:20 Szymon Gatner wrote:
 Anyway perfectly understandable POV. TDD is just a methodology. I
 myself not too long ago though it was *eghm* stupid to say the
 least. Why would I write the code entirely other way I was doing
 it until now? I am new to TDD but I must say I think it works for
 me. I am not ashamed to admit I am not to good in designing high
 level pieces of code. I know how to implement stuff but
 weeks/months later when requirements change I am always
 struggling with what i've done. Finally I see reusable pieces of
 code that I know I will be using in other possibly completely
 unrelated projects. Maybe TDD is just for stupid ppl like me? :P

I don't think that anyone's calling you stupid, but several of us do think that TDD is flawed. However, if it works for you, it works for you. - Jonathan M Davis
Jun 19 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Jun 20, 2013 at 03:15:54AM +0200, Chris Cain wrote:
[...]
 TDD (Test Driven Development) has a terrible name. Too many people
 think it's about the tests. Tests are certainly a good side effect
 of TDD and I certainly can't suggest they don't have value, but the
 fact that tests are written is really irrelevant to what you are
 _really_ trying to get out of TDD. There's been numerous attempts to
 rebrand it (Test Driven Design, Behavior Driven Development, and
 Behavior Driven Design to name a few).
 
 Ultimately, the "goal" of TDD is to have the programmer sit down and
 look at code from a USER'S (as in, user of the API) point of view
 _FIRST_. "Writing a test" as the first step is really just
 suggesting "as a user, how would you find it convenient to use this
 thing?" That's really just about all there is to it. It's one of
 those things that helps guide your thought process. If you're
 already doing that without writing tests first, then great. I've
 always found it pretty helpful to play around with how I want code
 to look like before I actually code the implementation details,
 though. More often I can come up with a simpler and more robust
 design after playing with how I want the end result to look like. As
 a cool bonus, once everything is done, you'll have a "specification"
 showing all of the features of a class/package/module and how
 everything is used. It will also show what kinds of preconditions
 are necessary for using those things (for instance, does your class
 need particular services to be online for it to work? If so, it'll
 show up in the tests).

Thanks for your clarification! This is much more helpful than the definitions of TDD that I found online (e.g. on Wikipedia). I will say, though, that this seems to be just Yet Another Buzzword for "API design". (I'm automatically skeptical when buzzwords are involved, because all too often, buzzwords are employed to make something rather ordinary appear to be special.) I agree that many APIs are poorly designed because the implementors were more concerned with the implementation, rather than how the users will be using it through the API. OTOH, not all APIs should be designed to cater to the users' POV. I've encountered APIs that were *not* "convenient" in the way that I had preconceived, but in the end, it turned out that by forcing me to do things the "inconvenient" way, it not only opened up brand new ways of thinking about the problem, but also guided me into approaching the problem from an angle that is *efficient to implement*. It's just like Knuth said: "People who are more than casually interested in computers should have at least some idea of what the underlying hardware is like. Otherwise the programs they write will be pretty weird." So I think a balance needs to be struck between what users would like it to be, vs. what is a good way of approaching the problem. API's should not be so far removed from their implementations that the resulting code becomes "pretty weird", as Knuth puts it. TDD is good and all, but there's the danger of pushing it too far and applying it where it's not applicable. T -- Obviously, some things aren't very obvious.
Jun 19 2013
prev sibling next sibling parent "Joakim" <joakim airpost.net> writes:
On Wednesday, 19 June 2013 at 21:54:04 UTC, Walter Bright wrote:
 On 6/19/2013 4:01 AM, Szymon Gatner wrote:
 This is not strictly D related but I am very curious about D's 
 community opinion
 on the points made by non other than Jim Coplien here:

TDD strikes me as an ad-hoc way of constructing code, and is a poor substitute for thinking about the problem as a whole. For example, I don't really see how getting a square root function to pass its test cases is going to lead one to implementing one of the classic algorithms for computing square roots. I don't see how TDD for a compiler will lead one to rediscover what is known about the best way to organize the logic of a compiler. TDD is "curve fitting" which is what one does when one has no understanding.

"classic algorithms" or "the best way" that is known, then you may not get much out of TDD. But if you're exploring the solution space and trying to come up with a design that emerges with use, as opposed to thinking up a complete design in your head and then simply implementing it, I think they'd say TDD is a good solution for that. Sometimes "ad-hoc" or "curve fitting" is a better way to begin, when you're entering a new domain, ie you don't even know what the curve looks like, say designing the first social network with 1 billion users. ;) At least that is what I understand of their method, I've never used it.
Jun 19 2013
prev sibling next sibling parent "w0rp" <devw0rp gmail.com> writes:
The only time I ever did TDD was when I had to implement a really
specific set of number and date format strings for a company,
because the users were used to them. It was pretty natural for
that. This format string x must take number y, and yield the
string result z. Write a bunch of tests for those, take a step
back, implement the format strings how you think they should
work, and see if the tests pass.

Unless I'm doing really specific "this input yields precisely
this output" stuff, I won't ever bother with TDD. I often end up
with "Test a little bit after" design.

I think his point in the talk about objects taking on roles might
map nicely to the way ranges were implemented. A object doesn't
behave as a range because it inherits an interface, it behaves
that way because it has the right stuff for it. Essentially, I
think he's getting close to saying "statically-checked duck
typing." Like all things, this is good *some of the time.*
Jun 20 2013
prev sibling next sibling parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Thursday, 20 June 2013 at 01:15:58 UTC, Chris Cain wrote:
 On Wednesday, 19 June 2013 at 23:44:29 UTC, Walter Bright wrote:
[...]
 Ultimately, the "goal" of TDD is to have the programmer sit 
 down and look at code from a USER'S (as in, user of the API) 
 point of view _FIRST_. "Writing a test" as the first step is 
 really just suggesting "as a user, how would you find it 
 convenient to use this thing?"

Just left enough context to ask the following question, I hope I did not left out too much. How does TDD then help how a user of my application will use the application UI? This is the area where I usually make TDD evangelists go speechless in conferences, as they start to present ad-hoc solutions and end up changing subject. -- Paulo
Jun 20 2013
prev sibling next sibling parent "eles" <eles eles.com> writes:
On Wednesday, 19 June 2013 at 17:31:33 UTC, H. S. Teoh wrote:
 On Wed, Jun 19, 2013 at 04:52:22PM +0200, bearophile wrote:
 irritate:
 
My feelings about TDD changed when I saw that talk explaining 
TDD
in the context of double-entry bookkeeping in accounting



 Before methodologies like TDD could even begin to work, one has 
 to
 *solve* the problem at hand first -- analyse the problem, 
 explore its
 structure, invent an algorithm, then one can verify the 
 correctness of
 one's implementation with unittests. You have to already have 
 an idea
 about how things are going to work, before TDD can help you.

For me, it is like this: TDD is rather similar to bookkeeping; design is similar to doing (or managing) businesses. While being (or having) a good bookkeeper will definitely help your business to perform, the reverse is not true: being the most careful accountant in the world does not mean that you will do successful businesses.
Jun 20 2013
prev sibling next sibling parent "Joakim" <joakim airpost.net> writes:
On Thursday, 20 June 2013 at 05:05:29 UTC, Joakim wrote:
 I think their argument would be that if you're only interested 
 in "classic algorithms" or "the best way" that is known, then 
 you may not get much out of TDD.  But if you're exploring the 
 solution space and trying to come up with a design that emerges 
 with use, as opposed to thinking up a complete design in your 
 head and then simply implementing it, I think they'd say TDD is 
 a good solution for that.

to say about the design benefits of TDD. Here's an archived blog post he wrote about the benefits of TDD: http://web.archive.org/web/20120201002523/http://blog.objectmentor.com/articles/2009/10/08/tdd-triage From a design standpoint, he says you need to do all the usual up-front design, but that TDD will help you modify and iterate on your inevitably incomplete initial design quicker.
Jun 20 2013
prev sibling next sibling parent "Chris Cain" <clcain uncg.edu> writes:
On Thursday, 20 June 2013 at 07:36:41 UTC, Paulo Pinto wrote:
 ...
 Just left enough context to ask the following question, I hope 
 I did not left out too much.

 How does TDD then help how a user of my application will use 
 the application UI?

 This is the area where I usually make TDD evangelists go 
 speechless in conferences, as they start to present ad-hoc 
 solutions and end up changing subject.

 --
 Paulo

Sure. The user, in this case, isn't the user of your application. The "user" is the person using your API (other programmers). TDD has nothing to do with the visual design of your application and everything to do with the software engineering design. It's to help guide you to the answer to the questions "how should I define the behavior of this class/how should this class be used in practice?" If your question is how to use TDD for UI, the answer is that you might use TDD for defining the interface to your UI API. But defining an actual UI using TDD is usually kind of awkward and I feel it's missing the point. I'm not saying I haven't seen it done, but typically what people do to "test first" UI designs isn't really what TDD was meant to do, I feel. --- I guess the short answer is that TDD doesn't make sense to be used for that (IMO). Just like a knife can't be used to cook an egg. I suppose this is where the phrase "there is no silver bullet" should be put? :)
Jun 20 2013
prev sibling next sibling parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Thursday, 20 June 2013 at 09:02:49 UTC, Chris Cain wrote:
 On Thursday, 20 June 2013 at 07:36:41 UTC, Paulo Pinto wrote:
 ...
 Just left enough context to ask the following question, I hope 
 I did not left out too much.

 How does TDD then help how a user of my application will use 
 the application UI?

 This is the area where I usually make TDD evangelists go 
 speechless in conferences, as they start to present ad-hoc 
 solutions and end up changing subject.

 --
 Paulo

Sure. The user, in this case, isn't the user of your application. The "user" is the person using your API (other programmers). TDD has nothing to do with the visual design of your application and everything to do with the software engineering design. It's to help guide you to the answer to the questions "how should I define the behavior of this class/how should this class be used in practice?"

UI is also code, composed by functions/classes/markup, depending on the language and framework. How to use TDD to write a class for an owner drawn control in Win32 as an example? This is the issue that TDD fails, but TDD advocates keep on selling it. -- Paulo
Jun 20 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Thursday, 20 June 2013 at 10:13:53 UTC, Jacob Carlborg wrote:
 On 2013-06-20 00:47, Nick Sabalausky wrote:

 - Writing a unittest first forces the API to be designed 
 before the
   implementation is written. But implementation is necessary 
 to flush
   out unexpected design requirements (If you think you can 
 always
   come up with an appropriate design and interface before
   implementing, then you're just plain wrong). Sometimes the
   appropriate design and API is obvious. Sometimes it isn't. 
 When it
   isn't, then TDD skirts dangerously close to some of the 
 problems of
   "Waterfall Model". Sure, TDD advocates 
 refactoring-as-needed, but I
   can do that with or without TDD.

Just because you have written a test doesn't mean you cannot change it. Perhaps you come up with a better API design, then change the tests.

When I start something, it isn't always clear what mental model of the problem fit best the problem. I usually wait until I know I have a consistent mental model of the problem to write test. Otherwise, tests tend to get into your way to change the API( because you need to change tests as well. I start writing test when I know what kind of API make sense (it isn't always finished, but I know what it will look like overall). Which lead to TITMOD, test in the middle of dev.
Jun 20 2013
prev sibling next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Thursday, 20 June 2013 at 14:46:30 UTC, Andrei Alexandrescu 
wrote:
 I dislike this. I've seen conference talks and such where the 
 speakers present the inevitable throwaway implementation that 
 is patently wrong but makes the first unittest pass. That 
 mindset doesn't sit well with me at all. I don't see why I need 
 to waste time writing tongue-in-cheek code that is nonsensical 
 and will obviously be deleted next.

There is this war story about the team coding some early version of Excel. The management decided to put very severe deadline on when features. Many dev ended up writing method like this, so the feature can be delivered in time, as it is now a bug if it fails.
Jun 20 2013
next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Jun 26, 2013 at 04:24:27PM -0400, Nick Sabalausky wrote:
 On Fri, 21 Jun 2013 16:03:57 -0700
 "H. S. Teoh" <hsteoh quickfur.ath.cx> wrote:
 
 On Fri, Jun 21, 2013 at 06:12:57PM -0400, Nick Sabalausky wrote:


   Now, I'm a strong believer in being self-taught and learning
   from books, but all I need for that is a library card, not a
   $100k debt and four years of elitist attitudes from people who
   clearly don't know what they're doing anyway.

Heh. Nearly all of my programming skills are self-taught (well, and learned from experience now that I have some number of years in the industry), but I'm no reader either. I was doing online learning long before the 'Net became cool,

Actually, my first introduction to programming was the interactive tutorial disks that came with the Apple IIc. I sometimes find it kind of depressing that instruction isn't even *that* far along anymore, let alone any further advanced.

Yeah, no kidding! In the Good Ole Days (gosh I'm old :-P) the Apple PCs used to come with *full listings of the ROM code*, schematic diagrams of PC internals, and voltage/amperage levels and pin diagrams along with suggestions of how to hook it up to homemade electronics. A far cry from today's blackboxed lawyer-guarded device rentals. Sigh. T -- Political correctness: socially-sanctioned hypocrisy.
Jun 26 2013
prev sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Wed, 26 Jun 2013 13:58:18 -0700
"H. S. Teoh" <hsteoh quickfur.ath.cx> wrote:
 
 Actually, my first introduction to programming was the interactive
 tutorial disks that came with the Apple IIc. I sometimes find it
 kind of depressing that instruction isn't even *that* far along
 anymore, let alone any further advanced.

Yeah, no kidding! In the Good Ole Days (gosh I'm old :-P

Heh. I admit I have little right to be talking about "getting old", but one thing I have learned: Hearing things I still think of as "normal" be referred to by people as "retro" or "ancient" or before their time certainly doesn't help one feel young ;) I've even seen N64/PS1 stuff called "retro"! That just seems so wrong - they supported *true 3D*! I don't think I'll ever get used to anything from 1990-2005 being "old" (90's==old? That just ain't right!), and I sympathize with anyone who feels that way about the 80's. You know there are *adults* now who never lived through any part of the 80's? Again, that just doesn't seem right! I can't legitimately say I'm getting old yet, but it sure as hell feels like it!
 the Apple
 PCs used to come with *full listings of the ROM code*, schematic
 diagrams of PC internals, and voltage/amperage levels and pin
 diagrams along with suggestions of how to hook it up to homemade
 electronics. A far cry from today's blackboxed lawyer-guarded device
 rentals. Sigh.
 

Yea. I had heard somewhere that at one point electronic devices often came with their own schematics, and the demise of that was part of what put Stallman down the path leading to the FSF. (Although I can kinda understand why the Core i7, etc, don't come with printed schematics - would there even be enough paper in the world? ;) ) Luckily, there's at least a good fight being put up against orwell-computing, even if it is an uphill battle. The thing I find most encouraging thing (although this is software-side) is gog.com - there's a good article on it here: http://www.forbes.com/sites/erikkain/2013/05/30/good-old-games-gog-com-and-the-drm-free-revolution/ There's also some encouraging things happening on the game hardware front: The old guard of the Nintendo/MS/Sony ivory towers will soon have some extra competition from the Nvidia's Shield, and Valve's SteamBox. Not really *entirely* open systems, granted, as neither of them compare to, for example gog.com. And they're certainly not going to kill anything from Nintendo/MS/Sony anytime soon (maybe even never, if I'm being pessimistic). But nonetheless, Shield and SteamBox are a huge increase in openness for "primarily gaming devices" - a category of devices which have barely seen a shred of openness since the 2600 (and that was open only because Atari hadn't anticipated anything like Activision happening).
Jun 26 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Jun 20, 2013 at 06:52:26PM +0200, deadalnix wrote:
 On Thursday, 20 June 2013 at 14:46:30 UTC, Andrei Alexandrescu
 wrote:
I dislike this. I've seen conference talks and such where the
speakers present the inevitable throwaway implementation that is
patently wrong but makes the first unittest pass. That mindset
doesn't sit well with me at all. I don't see why I need to waste
time writing tongue-in-cheek code that is nonsensical and will
obviously be deleted next.

There is this war story about the team coding some early version of Excel. The management decided to put very severe deadline on when features. Many dev ended up writing method like this, so the feature can be delivered in time, as it is now a bug if it fails.

That's terrible. It encourages the kind of sloppy coding that makes so much "enterprise" code look nastier than what a highschool dropout writes in his sleep. OTOH it explains a lot of things about why early versions of Excel were so atrocious. :-P T -- It only takes one twig to burn down a forest.
Jun 20 2013
prev sibling next sibling parent Sean Kelly <sean invisibleduck.org> writes:
On Jun 19, 2013, at 2:59 PM, Jonathan M Davis <jmdavisProg gmx.com> =
wrote:
=20
 It's not necessarily the case that you write _all_ of the tests and =

 of the code, but as I understand it, with TDD, you write a test for a =

 before you write that function (though both could be written well =

 functions and whatever tests go with them). And I completely disagree =

 idea of writing a test for a piece of code that doesn't exist yet, =

 you're going to write it immediately after writing the test.

The video is quite good but takes a long time to get to the really = interesting stuff. Regarding TDD, one point he makes that seems worth = repeating is that by focusing on the behavior of single functions you're = really just doing bottom-up design. He mentions some studies or at = least anecdotes from projects that tried TDD and they all agreed that as = time went on, their code entropy increased, which seems to support the = idea that TDD at least encourages people to do bottom-up design. The more interesting point he makes, however, is about how difficult it = is to understand and therefore accurately test object-oriented programs = by virtue of how such programs are traditionally built. So the problem = really isn't so much TDD as that it's often difficult to derive anything = truly useful from the results. He then goes on to talk about DCI (Data = Context Interaction), which is essentially a different way to build OO = apps that more closely models what the apps actually do. So you could = really use TDD in combination with this DCI design approach and end up = with something a lot more useful.=
Jun 20 2013
prev sibling next sibling parent Sean Kelly <sean invisibleduck.org> writes:
On Jun 19, 2013, at 3:00 PM, Jonathan M Davis <jmdavisProg gmx.com> =
wrote:

 On Wednesday, June 19, 2013 14:19:28 H. S. Teoh wrote:
 Doesn't TDD stand for Test-Driven *Design*?

It was my understanding that it was Test-Driven "Development," but I =

 wrong. Let's see... Well, according to Wikipedia, it's development, =

 design:
=20
 http://en.wikipedia.org/wiki/Test-driven_development

Apparently, the original word was "Design" and the creator laments the = fact it's too late to change it back.=
Jun 20 2013
prev sibling next sibling parent Sean Kelly <sean invisibleduck.org> writes:
On Jun 19, 2013, at 4:44 PM, Walter Bright <newshound2 digitalmars.com> =
wrote:

 On 6/19/2013 3:23 PM, Szymon Gatner wrote:
 Point of TDD for a square root function would be to create a good API =


 getting a square root of a number. Implementation is just a detail.

I just can't accept that. For one thing, implementation details often =

how it would be implemented will not produce an efficient design.
=20
 For the square root, there's a definite tradeoff between accuracy and =

spec, how can you make the right decisions? And this is why the waterfall model (which TDD seems to encourage) tends = to be problematic. That said, I do like the idea of building a program = in a way that is testable right from the start, because it can be very = hard to bolt on testing after the fact. For example, basically all the = work I do is with distributed programming, which requires extensive = mocking to produce any sort of meaningful test. If the API wasn't = designed to allow this, about the only testing that can be done is = black-box testing using external apps.=
Jun 20 2013
prev sibling next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Thu, 20 Jun 2013 03:15:54 +0200
"Chris Cain" <clcain uncg.edu> wrote:
 
 Personally, I'm a big fan of TDD in general, but I think it's one 
 of the most often misunderstood things in the field of 
 programming. It's like if you ask a group of religious people to 
 define what their deity wants from them... everyone's going to 
 have a different response and everyone is going to claim that 
 theirs is the "only one true way."
 
 So, here's my "only one true way" of TDD (tongue in cheek):
 
 [...]
 
 Anyway, TDD isn't BS, but I do think its misused. The misusage of 
 it is (sometimes) BS, for sure.

Very interesting and helpful explanation! I don't feel somewhat unclear about it now like I did before. It sounds like instead of trying to come up with a better buzzword for it, they should just call it "Design Your API First". Calling it TDD or BDD or whatever is like referring to DRY as "Generalization Oriented Coding". Pointless obfuscation of meaning. I do, however, still find that designing an API before doing any implementation is frequently premature ("Premature design", I guess you could call it. Counterpart to "premature optimization".) Granted, I certainly agree it's often good to give at least some up-front thought to API and overall architectural design. But in my experience, the amount of up-front design is something that needs to be judged on a case-by-case basis, and the "right amount" for a given task will likely be different for every programmer. I've found that trying to do a detailed API design up-front is usually (not always, but frequently) a very bad ROI. It can easily end up taking a fair amount of effort (since there's so many unknowns at that point), and then most of it ends up needing to change anyway (since it's based on so many unknowns), which renders anything more than a rough up-front analysis wasted effort. What usually works much better for me is to first get something *working*, and *then* look at the API from a user's perspective and think "Ok, what's wrong with this? Can I improve this?". At that point, refactoring the API into something better tends to be much more straightforward since I'm dealing with a real working "thing" rather than designing in a vacuum. Granted, there have definitely been times where I did feel it was prudent or necessary to do a more detailed up-front design. In those cases, I certainly do so. And than sometimes "getting something working" is just simply more important than worrying about an ideal API. So again, it's all a case-by-case judgment call. I don't really feel that "Design Your API First" is really a *generally* correct thing for most cases and programmers. To me, that's kind of like saying "Strings are better than integers" or "Integers are better than strings". It's just too dependent on the given situation.
Jun 20 2013
prev sibling next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Wed, 19 Jun 2013 20:25:41 -0700
"H. S. Teoh" <hsteoh quickfur.ath.cx> wrote:
 
 Thanks for your clarification! This is much more helpful than the
 definitions of TDD that I found online (e.g. on Wikipedia).
 
 I will say, though, that this seems to be just Yet Another Buzzword
 for "API design". (I'm automatically skeptical when buzzwords are
 involved, because all too often, buzzwords are employed to make
 something rather ordinary appear to be special.)

I feel the same way.
 OTOH, not all APIs should be designed to cater to the users' POV. I've
 encountered APIs that were *not* "convenient" in the way that I had
 preconceived, but in the end, it turned out that by forcing me to do
 things the "inconvenient" way, it not only opened up brand new ways of
 thinking about the problem, but also guided me into approaching the
 problem from an angle that is *efficient to implement*. It's just like
 Knuth said:
 
 	"People who are more than casually interested in computers
 	should have at least some idea of what the underlying hardware
 	is like. Otherwise the programs they write will be pretty
 	weird."
 
 So I think a balance needs to be struck between what users would like
 it to be, vs. what is a good way of approaching the problem.  API's
 should not be so far removed from their implementations that the
 resulting code becomes "pretty weird", as Knuth puts it. TDD is good
 and all, but there's the danger of pushing it too far and applying it
 where it's not applicable.
 

Right. It's like creating a Vacuum Cleaner or a car or a kitchen appliance by designing the case, housing and controls first, and then trying to design working internals to fit that mold. Form needs to follow function.
Jun 20 2013
prev sibling next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Thu, 20 Jun 2013 21:38:08 +0200
Jacob Carlborg <doob me.com> wrote:

 On 2013-06-20 20:14, Sean Kelly wrote:
 
 And this is why the waterfall model (which TDD seems to encourage)
 tends to be problematic.

You don't write all the test at once and then all the implementations at once. Write tests for one function first, then implement it. Then move to the next function. Also, tests are not written in stone. They should evolve and change just as much as the rest of the code.

But then at that point, where's the benefit of writing the tests first? I can still evolve/change the tests if I wrote them after the implementation. Writing them before the implementation just means I have that much more bookkeeping *while* implementing. May as well just hold off on the tests until I have an actual interface to test against. It not like the pre-written-but-failing tests were doing me any good while I was still implementing. I'm not saying that it's necessarily bad to write a function's tests before the function itself, I just don't see what it really matters to code the tests first. The important thing is that the tests get written, ideally before you move on to something else. If you have the discipline to write the tests before the function, then you have the discipline to write the tests after the function. If the issue is to make sure the tests promptly get written promptly, then D's built-in unittests do far more to encourage that than TDD would seem to.
Jun 20 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Jun 20, 2013 at 06:37:35PM -0400, Nick Sabalausky wrote:
 On Thu, 20 Jun 2013 21:38:08 +0200
 Jacob Carlborg <doob me.com> wrote:

 You don't write all the test at once and then all the
 implementations at once. Write tests for one function first, then
 implement it. Then move to the next function.
 
 Also, tests are not written in stone. They should evolve and change
 just as much as the rest of the code.
 

But then at that point, where's the benefit of writing the tests first? I can still evolve/change the tests if I wrote them after the implementation. Writing them before the implementation just means I have that much more bookkeeping *while* implementing. May as well just hold off on the tests until I have an actual interface to test against. It not like the pre-written-but-failing tests were doing me any good while I was still implementing. I'm not saying that it's necessarily bad to write a function's tests before the function itself, I just don't see what it really matters to code the tests first. The important thing is that the tests get written, ideally before you move on to something else. If you have the discipline to write the tests before the function, then you have the discipline to write the tests after the function. If the issue is to make sure the tests promptly get written promptly, then D's built-in unittests do far more to encourage that than TDD would seem to.

+1. I'm guessing that part of TDD's drive is that you want to be explicit about what you expect from your code, so you set down your expectations beforehand; and since you have to do that, why not do it in the form of actually-runnable code (i.e., unittests)? Then, you work on making the code behave the way you expect. I have no problem with the bit about setting down expectations of what the code should do -- it may help you rethink the problem from the POV of the user of your code, and possibly find better ways of structuring the solution. But where I see TDD treading shaky ground is to singlemindedly focus on this part of the coding process to the exclusion of all else -- algorithm design, correspondence between API and implementation, efficiency considerations, etc.. I've always been a supporter of the view that the best programs are those in which code structure corresponds with data structure (a key idea in Jackson Structured Programming). Focusing on API design to the exclusion of everything else risks a mismatch between API structure and implementation structure, which leads to increased likelihood of bugs, inefficiencies, and code smells. T -- IBM = I'll Buy Microsoft!
Jun 20 2013
prev sibling next sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Thu, 20 Jun 2013 10:04:54 -0700
"H. S. Teoh" <hsteoh quickfur.ath.cx> wrote:
=20
 That's terrible. It encourages the kind of sloppy coding that makes so
 much "enterprise" code look nastier than what a highschool dropout
 writes in his sleep.
=20

I realize you're just using that as an illustrative example, but: High schools are *specifically* and *deliberately* designed for *average*-level students. The students who have trouble there are both the low-intelligence *and* high-intelligence ones. This is a *known*, documented fact. Virtually nobody accepts it though, most likely because schools are (undeniably) sacred cows. As for more direct experience, one of the smartest people I've ever known was a high school drop-out. The majority of the *worst* programmers (and just biggest dumbfucks in general) that I've ever met happened to be grad students and graduate degree holders. Only *one* of the worst programmers I've come across was self-taught (and IIRC, he did still have at least a 4-year college degree). There is *no* positive correlation between ability/intelligence and academic achievement. I don't understand where so much of society keeps getting that patently absurd notion. At best, it's a zero-correlation. At worst, it may very well be a somewhat negative-leaning correlation. (No offense to Andrei or others here - obviously there's always exceptions, and being big long time D fans kinda puts us all in the "outliers" boat anyway.) I'd also point people to this: "The results of a major national study revealed that much of the regular curriculum is redundant for gifted students (Reis et al., 1993). When as much as 60% of the curriculum was eliminated, gifted students exceeded or equaled achievement levels of matched students who were required to complete the regular curriculum. Although these findings bode ill for bright students in general, consider the plight of those who tend to be predisposed to seeking greater levels of stimulation from the environment. They are automatically at odds with the expectations schools have for students to be neat, docile, quiet for extended periods, and interested in what the teacher is teaching." -- http://www.sengifted.org/archives/articles/gifted-students-with-attention-d= eficits-fact-andor-fiction-or-can-we-see-the-forest-for-the-trees The study referenced is this: Reis, S. M., Westberg, K. L., Kulikowich, J., Caillard, F., H=E9bert, T., Plucker, J., Purcell, J. H., Rogers, J. B., & Smist, J. M. (1993). Why not let high ability students start school in January? The curriculum compacting study. (Research Monograph No. 93106). Storrs, CT: The National Research Center on the Gifted and Talented.=20 Which is available here: http://www.gifted.uconn.edu/nrcgt/reports/rm93106/rm93106.pdf And I can personally attest to the accuracy of that quoted paragraph above, as I *lived* exactly what they describe during the majority of that hellish K-12 cesspool of retards normally referred to as "school". And of course, that's just one study, there's plenty of others that all boil down to the same thing: Schools do NOT, in general, work for intelligent students. Only the average ones. (No different from the rest of society really: Only the first std deviation is ever given a shit about, everyone else is expected to just fuck off since their existence mucks up the corporate profit-maximization spreadsheets - which is also why public stocks are fucking evil, as it legally *mandates* that the corporation *must* put financial profit far above *all* other concerns such as, you know, not being an evil society-destroying sack of shit.) Bottom line is, schools are absolute garbage, and academic achievement is *at best* completely meaningless.
Jun 20 2013
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Jun 21, 2013 at 04:02:20PM -0700, Walter Bright wrote:
[...]
 /****************************************************************************
 See "Algorithms" Second Edition by Robert Sedgewick.  Boyer-Moore string
 search routine.
 ****************************************************************************/
 char *strstr(const char *text, const char * pattern)
 {

 }

Very nice! Does it help at all to cast to uint[] and search that instead of just char[]? Or do the complications caused by non-alignment outweigh the benefits? And where's the unittest block? ;-) (OK OK, I know this wasn't written in D. But I had to ask. :-P) Also, this is a prime example of code you'll *never* arrive at just by blind application of TDD. (And there's my feeble attempt to bring this back on topic. :-P) T -- This sentence is false.
Jun 21 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Jun 20, 2013 at 11:32:56PM -0400, Nick Sabalausky wrote:
 On Thu, 20 Jun 2013 10:04:54 -0700
 "H. S. Teoh" <hsteoh quickfur.ath.cx> wrote:
 
 That's terrible. It encourages the kind of sloppy coding that makes
 so much "enterprise" code look nastier than what a highschool
 dropout writes in his sleep.
 

I realize you're just using that as an illustrative example, but:

 Bottom line is, schools are absolute garbage, and academic
 achievement is *at best* completely meaningless.

I hit a nerve, didn't I? ;-) One of my previous supervisors told me that when he gets resums, as soon as he sees "Ph.D" he chucks it straight into the trash. I have an MSc, and he almost did the same with my resum until he saw that I did a lot of personal coding projects on the side, and contributed to open source projects. His reasons: (1) most CS grad students know too much theory but can't write any real code; (2) they're too opinionated to be able to work in a team. He may have been a bit extreme, but then I've personally met CS grad students who knew more about uncomputable problems than computable ones... :-P and the sight of their code makes me wanna wash my hands with detergent. T -- Questions are the beginning of intelligence, but the fear of God is the beginning of wisdom.
Jun 20 2013
prev sibling next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Thu, 20 Jun 2013 20:50:08 -0700
"H. S. Teoh" <hsteoh quickfur.ath.cx> wrote:

 On Thu, Jun 20, 2013 at 11:32:56PM -0400, Nick Sabalausky wrote:
 On Thu, 20 Jun 2013 10:04:54 -0700
 "H. S. Teoh" <hsteoh quickfur.ath.cx> wrote:
=20
 That's terrible. It encourages the kind of sloppy coding that
 makes so much "enterprise" code look nastier than what a
 highschool dropout writes in his sleep.
=20

I realize you're just using that as an illustrative example, but:

 Bottom line is, schools are absolute garbage, and academic
 achievement is *at best* completely meaningless.

=20 I hit a nerve, didn't I? ;-) =20

Heh ;)
 One of my previous supervisors told me that when he gets resum=E9s, as
 soon as he sees "Ph.D" he chucks it straight into the trash.

Classic ;) I'd be likely to do the same. And I have a friend who's moved up to a management position on a University's IT dept and he's had similar experiences: When they hire, they're expected to get their candidates via the HR dept (heh, yea, right there you know things are gonna go wrong ;) ). And their HR dept, as with most, makes their selections by blindly grepping for the biggest degrees (they don't know any better - they're HR people!). He's told me horror stories about how nearly all of the big-degree candidates they send him turn out during interview to be flat-out 100% incompetent.
 I have an
 MSc, and he almost did the same with my resum=E9 until he saw that I
 did a lot of personal coding projects on the side, and contributed to
 open source projects. His reasons: (1) most CS grad students know too
 much theory but can't write any real code; (2) they're too
 opinionated to be able to work in a team.

I'm too opinionated *and* anti-social to be able to work in a team ;)
 He may have been a bit
 extreme, but then I've personally met CS grad students who knew more
 about uncomputable problems than computable ones... :-P  and the
 sight of their code makes me wanna wash my hands with detergent.
=20

When I was a freshman undergrad (at BGSU, near Toledo), I tinkered around on the side with some PalmOS development. (PalmOS was still around back then.) A couple years or so later, shortly after I'd transferred out IIRC, I noticed on BGSU's Computer Science dept's website that they were bragging - and I mean genuinely bragging - that their genius *graduate* CS students had (*gasp!*) made software that ran on a PalmOS device! Which of course leads to the following lightbulb joke: How many academic weenies does it take to install an off-the-shelf compiler, read an API reference, and not be shown-up by the two year old shadow of a degreeless freshman?
Jun 20 2013
prev sibling next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Thu, 20 Jun 2013 11:00:32 -0700
Sean Kelly <sean invisibleduck.org> wrote:

 On Jun 19, 2013, at 3:00 PM, Jonathan M Davis <jmdavisProg gmx.com>
 wrote:
 
 On Wednesday, June 19, 2013 14:19:28 H. S. Teoh wrote:
 Doesn't TDD stand for Test-Driven *Design*?

It was my understanding that it was Test-Driven "Development," but I could be wrong. Let's see... Well, according to Wikipedia, it's development, not design: http://en.wikipedia.org/wiki/Test-driven_development

Apparently, the original word was "Design" and the creator laments the fact it's too late to change it back.

TBH, I really doubt that change would have done much to clear up the confusion.
Jun 20 2013
prev sibling next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Thu, 20 Jun 2013 10:13:39 +0200
"eles" <eles eles.com> wrote:

 On Wednesday, 19 June 2013 at 17:31:33 UTC, H. S. Teoh wrote:
 On Wed, Jun 19, 2013 at 04:52:22PM +0200, bearophile wrote:
 irritate:
 
My feelings about TDD changed when I saw that talk explaining 
TDD
in the context of double-entry bookkeeping in accounting



 Before methodologies like TDD could even begin to work, one has 
 to
 *solve* the problem at hand first -- analyse the problem, 
 explore its
 structure, invent an algorithm, then one can verify the 
 correctness of
 one's implementation with unittests. You have to already have 
 an idea
 about how things are going to work, before TDD can help you.

For me, it is like this: TDD is rather similar to bookkeeping; design is similar to doing (or managing) businesses. While being (or having) a good bookkeeper will definitely help your business to perform, the reverse is not true: being the most careful accountant in the world does not mean that you will do successful businesses.

More and more, I'm thinking that TDD is most similar to "art": Many people say they like it, but nobody, not even the creators, seem to know what the hell the term even means. ;)
Jun 20 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Jun 21, 2013 at 01:14:28AM -0400, Nick Sabalausky wrote:
 On Thu, 20 Jun 2013 20:50:08 -0700
 "H. S. Teoh" <hsteoh quickfur.ath.cx> wrote:

 One of my previous supervisors told me that when he gets resums, as
 soon as he sees "Ph.D" he chucks it straight into the trash.

Classic ;) I'd be likely to do the same. And I have a friend who's moved up to a management position on a University's IT dept and he's had similar experiences: When they hire, they're expected to get their candidates via the HR dept (heh, yea, right there you know things are gonna go wrong ;) ). And their HR dept, as with most, makes their selections by blindly grepping for the biggest degrees (they don't know any better - they're HR people!). He's told me horror stories about how nearly all of the big-degree candidates they send him turn out during interview to be flat-out 100% incompetent.

It's not entirely HR's fault. It's the way they're setup to fail. :-P The way it works at my job is that everything still has to go through HR, but it's the hiring managers (who are themselves coders actually writing code for projects) who review resums and recommend potential hires to HR. HR then arranges the technical tests and interviews, usually with the hiring managers who recommended the candidates plus one or two others whom the candidates will actually be working with, and then take care of all the paperwork and other stuff that hiring managers shouldn't be concerned with. IOW, we don't have HR decide who to interview, but actual technical people look at the resums to weed out the BS. I was involved with the hiring process for a bit some time ago -- we got to ask HR to first administer a technical test before any interviews are arranged; test results are reviewed before deciding to interview the candidate -- and we get to ask them questions about why they answered a certain test question the way they did. Makes weeding out incompetent applicants much easier. :-) If the code they submitted on the test looks obviously lousy, we don't even bother interviewing them. In spite of it all, though, we still sometimes end up hiring people who, 6 months down the road, write code that makes you scratch your head going "huh?! that genius coder we hired wrote *this* junk?!". But maybe some hiring managers are less discerning than others. *shrug* [...]
 When I was a freshman undergrad (at BGSU, near Toledo), I tinkered
 around on the side with some PalmOS development. (PalmOS was still
 around back then.) A couple years or so later, shortly after I'd
 transferred out IIRC, I noticed on BGSU's Computer Science dept's
 website that they were bragging - and I mean genuinely bragging - that
 their genius *graduate* CS students had (*gasp!*) made software that
 ran on a PalmOS device!

At least it's not as bad as "Graduates from our wonderful marvellous amazing CS program helped design our sucky non-navigable department website! Don't you wanna come study here and get a degree in HTML?". Sigh... T -- War doesn't prove who's right, just who's left. -- BSD Games' Fortune
Jun 20 2013
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 6/21/13, H. S. Teoh <hsteoh quickfur.ath.cx> wrote:
 In spite of it all, though, we still sometimes end up hiring people who,
 6 months down the road, write code that makes you scratch your head
 going "huh?! that genius coder we hired wrote *this* junk?!".

Seems like nowadays it's not too far-fetched to ask for a github/bitbucket/etc username and see the work they've done, the way they write code, contribute, etc. It should give a better "feel" than any interview.
Jun 20 2013
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 6/21/13, Jacob Carlborg <doob me.com> wrote:
 I agree. Most interviews I have been on lately is due to my github project.

Oh? That's cool. Which [rpkect?
Jun 21 2013
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 6/21/13, Andrej Mitrovic <andrej.mitrovich gmail.com> wrote:
 On 6/21/13, Jacob Carlborg <doob me.com> wrote:
 I agree. Most interviews I have been on lately is due to my github
 project.

Oh? That's cool. Which [rpkect?

Looks like I rot13'ed by accident, I meant which project?
Jun 21 2013
prev sibling next sibling parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Friday, 21 June 2013 at 06:34:09 UTC, Andrej Mitrovic wrote:
 On 6/21/13, H. S. Teoh <hsteoh quickfur.ath.cx> wrote:
 In spite of it all, though, we still sometimes end up hiring 
 people who,
 6 months down the road, write code that makes you scratch your 
 head
 going "huh?! that genius coder we hired wrote *this* junk?!".

Seems like nowadays it's not too far-fetched to ask for a github/bitbucket/etc username and see the work they've done, the way they write code, contribute, etc. It should give a better "feel" than any interview.

Except not everyone has the authorization to place their work code in such public places nor the availability or desire to code after work, just to please job interviewers. -- Paulo
Jun 21 2013
prev sibling next sibling parent "Wyatt" <wyatt.epp gmail.com> writes:
On Friday, 21 June 2013 at 05:59:00 UTC, H. S. Teoh wrote:
 In spite of it all, though, we still sometimes end up hiring 
 people who,
 6 months down the road, write code that makes you scratch your 
 head
 going "huh?! that genius coder we hired wrote *this* junk?!". 
 But maybe
 some hiring managers are less discerning than others. *shrug*

people who know how to use external libraries, what a debugger is, more than one programming language (or, indeed, any language other than Java), and any form of source control at all, I'm afraid you're hitting among the cream of the crop as far as CS baccalaureates go. And if you want them to admin a box at the same time, good luck with that. You're often better off with a passionate dropout if you're interested in hiring someone to write code. My hypothesis is this happens because the curricula are heavily skewed toward theoretical aspects of computer science. (In my view, the exemplar for this was OSU before they axed the mandatory RESOLVE/C++ series.) Cheers, Wyatt
Jun 21 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 19 June 2013 at 11:01:05 UTC, Szymon Gatner wrote:
 This is not strictly D related but I am very curious about D's 
 community opinion on the points made by non other than Jim 
 Coplien here:

 http://www.tele-task.de/archive/video/flash/16130/

 D is the only language (that I am aware of) that has first 
 class unit testing support. What do you think? Do we really 
 just "mentally masturbate"?

I finally had time to see the conf. I have to say that I started with a bad opinion of it :D But it is simply poorly marketed, the content is actually very interesting. And I'd say it is mostly about OOP rather than TDD. And I agree with most of it, while learning some other stuff.
Jun 21 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Jun 21, 2013 at 11:31:22AM +0200, Jacob Carlborg wrote:
 On 2013-06-21 05:32, Nick Sabalausky wrote:
 
[Snip]

Bottom line is, schools are absolute garbage, and academic
achievement is *at best* completely meaningless.

I couldn't agree more with your post. The problem though is that most hiring process uses the HR department, as you mentioned in an other post, and they only look at the degree. I only took a degree to have something "to show" for these HR departments, most of what I learned I learned on my own time.

+1, me too! I can say that 85-90% of what I do at work today, I learned from my personal coding projects, not from the CS courses I took in university. (That's why I like to joke about CS grads knowing more about uncomputable problems than computable ones...) Having said that, though, I *did* find that going through university helped train my mind to think logically and rigorously, even if most of the course content was not actually useful in real-life. T -- Valentine's Day: an occasion for florists to reach into the wallets of nominal lovers in dire need of being reminded to profess their hypothetical love for their long-forgotten.
Jun 21 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Jun 21, 2013 at 05:48:25PM +0200, Jacob Carlborg wrote:
 On 2013-06-21 16:56, H. S. Teoh wrote:
 
+1, me too! I can say that 85-90% of what I do at work today, I
learned from my personal coding projects, not from the CS courses I
took in university. (That's why I like to joke about CS grads knowing
more about uncomputable problems than computable ones...)

It feels like there's something wrong with the world here :)

Only "something" wrong? ;-) To be fair, though, I can understand why university programs are that way: their goal is to produce more researchers and professors who may join the faculty and produce more research. So they focus more on the theoretical aspects of computer science in order to produce such candidates, whereas focusing on more practical aspects is something for technical colleges whose goal is to produce industry workers. Unfortunately, in North America at least, the technical colleges lean more toward stuff like "how to use MS Word", "how to organize your company data in MS SQL", rather than _real_ programming, and universities that actually *teach* real programming are more interested in finding solutions to uncomputable problems than teaching students how to solve computable ones, so there's a gap in the area of producing qualified industry coders who can write functional software. There *are* pockets of competent programming education here and there, of course, but this is the impression I get from the general situation. T -- If you think you are too small to make a difference, try sleeping in a closed room with a mosquito. -- Jan van Steenbergen
Jun 21 2013
prev sibling next sibling parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Friday, 21 June 2013 at 19:14:18 UTC, Walter Bright wrote:
 On 6/20/2013 8:50 PM, H. S. Teoh wrote:
 One of my previous supervisors told me that when he gets 
 resumés, as
 soon as he sees "Ph.D" he chucks it straight into the trash.

He'd have missed out on Andrei, then.

And a lot of other people who are driven towards new ideas. There's a fair amount of inverted snobbery about academia here. Ultimately, a PhD shows the ability to conduct original research and present it. It doesn't make you a great programmer, but then again *it never purports to*. Nor does a computer science degree. If you want a normal programming job, you need to show more real world experience than a PhD, but just throwing out people who have proved their originality and in depth understanding of a topic through a PhD is nothing short of absurd.
Jun 21 2013
prev sibling next sibling parent "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
On Fri, 21 Jun 2013 21:11:51 +0200, Walter Bright  
<newshound2 digitalmars.com> wrote:

 Whether you write the unit tests before, during, or after the module is  
 written is irrelevant.

I mostly agree with this, but there is one reason to write tests before implementing - you don't know the code yet. This way you don't filter out stuff that you think is unnecessary because you know the code handles it. Of course, the converse is also true - write tests after writing the code, because you then know the code and have a better idea of what the corner cases are. -- Simen
Jun 21 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Jun 21, 2013 at 09:49:21PM +0200, John Colvin wrote:
 On Friday, 21 June 2013 at 19:14:18 UTC, Walter Bright wrote:
On 6/20/2013 8:50 PM, H. S. Teoh wrote:
One of my previous supervisors told me that when he gets resums, as
soon as he sees "Ph.D" he chucks it straight into the trash.

He'd have missed out on Andrei, then.

And a lot of other people who are driven towards new ideas. There's a fair amount of inverted snobbery about academia here. Ultimately, a PhD shows the ability to conduct original research and present it. It doesn't make you a great programmer, but then again *it never purports to*. Nor does a computer science degree.

According to my ex-supervisor (and I'm not saying I agree with him), it indicates that one is opinionated enough to originate ideas, and stubborn enough to successfully defend said ideas, which can be detrimental in a team setting if the idea wasn't a good one. (And since a PhD doesn't purport to make you a great programmer, and he was looking for great programmers rather than researchers, that could be a reason for his views on the matter.)
 If you want a normal programming job, you need to show more real
 world experience than a PhD, but just throwing out people who have
 proved their originality and in depth understanding of a topic
 through a PhD is nothing short of absurd.

Well, my ex-supervisor *did* have a reputation of having many "interesting" (i.e. extreme) ideas about many things. :) I can't say I subscribe to his views on this matter, but what I was trying to get at was the prevalent fallacious fixation on academic achievement (i.e. equating "he has good grades / a degree / a PhD" with "he is a good programmer"). Too many potential employers can't see beyond that, or are not willing / don't have the time and energy to do so, thus resulting in the situation where people are being hired because of their academic achievement, but are expected to have skills not necessarily implied by said achievement. Then when such hires consistently produce sub-par work, some people get provoked to equate "PhD" with "poor programming skills". This situation wouldn't have developed if employers evaluated candidates based on their *skills* rather than by the credentials on paper. But in this day and age where time is never enough, it's all too convenient to dismiss a candidate because he has no academic credentials rather than to spend the time / energy to review unlikely candidates on the off-chance that perhaps they might turn out to be hidden programming prodigies. Or to blindly hire a candidate *with* said credentials because someone of high academic standing is likely to be skillful enough to do the job, rather than to spend to time / energy to check if this is actually the case. (Not to mention that all too often, the ones with hiring powers may not necessarily have the ability to discern real skills from a smooth talker.) T -- "You are a very disagreeable person." "NO."
Jun 21 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Jun 21, 2013 at 12:11:51PM -0700, Walter Bright wrote:
 On 6/21/2013 3:04 AM, Jacob Carlborg wrote:
But I do some researcher about TDD and similar techniques, use my
common sense, pick some pieces from here and there and use what I
think works.

I've been around long enough to have seen an endless parade of magic new techniques du jour, most of which purport to remove the necessity of thought about your programming problem. In the end they wind up contributing one or two pieces to the collective wisdom, and fade away in the rearview mirror.

+1, words of wisdom.
From "high-level language" to "structured programming" to "OO" to

the next bandwagon, it's all the same story repeated over and over. It's supposed to be the panacea that will make you able to ship a product just by pressing a single magic button, solve all your programming problems, and cure world hunger. Then you actually get into it, and discover that you still have to (gasp!) use your brain to get your program working. And life goes on as before.
 But there is one technique that stands head and shoulders above the
 others in improving code quality, sometimes dramatically so. That's
 unit testing coupled with a coverage analyzer.
 
 I can't even count the number of times I thought "this is so simple,
 I couldn't have gotten it wrong, I doan need no steeenkin unit
 tests." But I write a couple anyway, and you can guess the rest.

Yeah, I used to be firmly in the camp of "this code is so trivial, so *blatantly* correct, that it constitutes its own proof of correctness." Until D came along and shamed me into writing unittests for said "trivial" code, and suddenly I found myself sinking in the flood of bugs, failing corner cases, and (gasp!) typos, that I hadn't imagined were there all along.
 Whether you write the unit tests before, during, or after the module
 is written is irrelevant.

Yeah, sometimes I start with one or two tests before, usually a whole bunch in the middle, and a couple after (followed by a lot more later when bugs are inevitably found and fixed). The best part about D's built-in unittests (and I keep harping on about this) is that they're so dang convenient you'd be foolish for not writing them. They can be integrated into the very process of coding -- which has singlehandedly improved the quality of my code by leaps and bounds. T -- IBM = I'll Buy Microsoft!
Jun 21 2013
prev sibling next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Fri, 21 Jun 2013 17:48:25 +0200
Jacob Carlborg <doob me.com> wrote:

 On 2013-06-21 16:56, H. S. Teoh wrote:
 
 +1, me too! I can say that 85-90% of what I do at work today, I
 learned from my personal coding projects, not from the CS courses I
 took in university. (That's why I like to joke about CS grads
 knowing more about uncomputable problems than computable ones...)

It feels like there's something wrong with the world here :)

There definitely is. I spent a total of somewhere around 6 years in college, but somewhere around the middle of that I made the deliberate decision to *not* get a degree of any sort. That was because I wanted no part in helping to perpetuate the incredibly widespread myths/lies about academic achievement. I've never regretted that decision. (The one decision I *did* regret was not leaving college entirely after the first year.)
Jun 21 2013
prev sibling next sibling parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Friday, 21 June 2013 at 20:25:28 UTC, H. S. Teoh wrote:
 On Fri, Jun 21, 2013 at 09:49:21PM +0200, John Colvin wrote:
 On Friday, 21 June 2013 at 19:14:18 UTC, Walter Bright wrote:
On 6/20/2013 8:50 PM, H. S. Teoh wrote:
One of my previous supervisors told me that when he gets 
resumés, as
soon as he sees "Ph.D" he chucks it straight into the trash.

He'd have missed out on Andrei, then.

And a lot of other people who are driven towards new ideas. There's a fair amount of inverted snobbery about academia here. Ultimately, a PhD shows the ability to conduct original research and present it. It doesn't make you a great programmer, but then again *it never purports to*. Nor does a computer science degree.

According to my ex-supervisor (and I'm not saying I agree with him), it indicates that one is opinionated enough to originate ideas, and stubborn enough to successfully defend said ideas, which can be detrimental in a team setting if the idea wasn't a good one. (And since a PhD doesn't purport to make you a great programmer, and he was looking for great programmers rather than researchers, that could be a reason for his views on the matter.)
 If you want a normal programming job, you need to show more 
 real
 world experience than a PhD, but just throwing out people who 
 have
 proved their originality and in depth understanding of a topic
 through a PhD is nothing short of absurd.

Well, my ex-supervisor *did* have a reputation of having many "interesting" (i.e. extreme) ideas about many things. :) I can't say I subscribe to his views on this matter, but what I was trying to get at was the prevalent fallacious fixation on academic achievement (i.e. equating "he has good grades / a degree / a PhD" with "he is a good programmer"). Too many potential employers can't see beyond that, or are not willing / don't have the time and energy to do so, thus resulting in the situation where people are being hired because of their academic achievement, but are expected to have skills not necessarily implied by said achievement. Then when such hires consistently produce sub-par work, some people get provoked to equate "PhD" with "poor programming skills". This situation wouldn't have developed if employers evaluated candidates based on their *skills* rather than by the credentials on paper. But in this day and age where time is never enough, it's all too convenient to dismiss a candidate because he has no academic credentials rather than to spend the time / energy to review unlikely candidates on the off-chance that perhaps they might turn out to be hidden programming prodigies. Or to blindly hire a candidate *with* said credentials because someone of high academic standing is likely to be skillful enough to do the job, rather than to spend to time / energy to check if this is actually the case. (Not to mention that all too often, the ones with hiring powers may not necessarily have the ability to discern real skills from a smooth talker.) T

Agreed on all points. As an aside: what defines a "good programmer" is of course dependant on the task. Many academics do write effective code, quite quickly, and get a lot of quite convoluted work done by coding. E.g. I would (entirely hypothetically) happily hire a physics PhD to write high-level data-analysis routines in python, but i'd be wary of getting them to work on a high security web-app. On the flip side, I would be wary of hiring an enthusiastic college dropout to write an accurate fluid simulation, but they might easily fit the definition of "good programmer" for extending a web framework.
Jun 21 2013
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Friday, 21 June 2013 at 21:33:43 UTC, Andrei Alexandrescu 
wrote:
 brute-force strstr() is fair game and I think any engineer 
 should be able to lift it off the ground quickly (to my dismay, 
 only a fraction can).

But, should the return value be const or not? :P I think I'd write in D just cuz of inout().
Jun 21 2013
prev sibling next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Fri, 21 Jun 2013 09:47:46 -0700
"H. S. Teoh" <hsteoh quickfur.ath.cx> wrote:
 
 universities that actually *teach* real programming are more
 interested in finding solutions to uncomputable problems than
 teaching students how to solve computable ones
 

That doesn't match my experience (also in the US here). Granted, all my info is from ten years ago, but what I saw was mainly a bunch of what The "Joel on Software" guy called "Java Schools" <http://www.joelonsoftware.com/articles/ThePerilsofJavaSchools.html>. The following includes what I've seen at BGSU and OSU (party schools, not that I personally attended OSU, but I did have a friend that went through OSU's "RESOLVE/C++" stuff) and also JCU (a private univ that, at least around Cleveland, is highly-regarded by everyone except me). What I've seen at these places, and apparently many others from what I understand, is that while they *do* recognize the importance of creating programmers, the problems are: - The theory is minimal to make sure they get all those high-paying students through the revolving door. - Tests and exams do *not* teach people. An yet, that's where the emphasis is, instead of on instruction. - The really *big* issue: You just simply CANNOT expect people to go from beginner to competent programmer when they spend *at most* one-third of their credits, and about 3 hours a week, over a mere 4 years on actual programming material instead of irrelevant liberal arts garbage that *belongs* in high school, not a college so-called "major". - The nasty little details like pointer/memory problems, linker errors, etc that real people have to deal with are neatly glossed over and sidestepped. - There was one CS101 teacher (I had to tutor her unfortunate students) who constantly bragged about being from a real-world software company...but she was a Java-addict (circa v1.2-v1.4) who kept trying to teach OO *before* basic flow-of-execution. Consequently, none of her unfortunate students had the slightest clue what was going on. - Many of the professors are terrible programmers themselves. For example, I had one who openly admitted the only language he knew was C, and yet at one point it became painfully obvious that he had almost no comprehension of null-terminated strings. - Many of the teachers don't even teach, they just collect the thousands of dollars in tuition and give you a book recommendation (really more of a "demand" than a recommendation). Now, I'm a strong believer in being self-taught and learning from books, but all I need for that is a library card, not a $100k debt and four years of elitist attitudes from people who clearly don't know what they're doing anyway.
Jun 21 2013
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
Just for laughs I just slapped together a strstr and it made me 
realize a little thing I do in D that I never did in C:

assert()

I use it all over the place in D, pretty much any time I make an 
assumption, I slap it down in an assert.

But I don't I ever, not once, used any kind of assertion in C or 
C++. I think this is a fairly significant gain that can be 
attributed to it being built in to D. It is just silly not to do 
it when it is there.
Jun 21 2013
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Friday, 21 June 2013 at 22:23:01 UTC, Adam D. Ruppe wrote:
 being built in to D

Or maybe it is because everybody else does it in the documentation and such, since #include <assert.h> in C isn't *that* big of a deal. idk, all I do know for sure is that I do it now and didn't before.
Jun 21 2013
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Friday, 21 June 2013 at 22:35:55 UTC, Andrei Alexandrescu 
wrote:
 Post it and I'll destroy it.

inout(char)* mystrstr(inout(char)* haystack, const(char*) needle) { assert(haystack !is null); if(needle is null) return haystack; const(char)* where = needle; inout(char)* gotit; while(*haystack) { if(*haystack == *where) { if(gotit is null) gotit = haystack; // store where the match started where++; haystack++; if(*where == 0) return gotit; } else { // if we were in the middle of a match, we'll want to // check the current character again so only advance if // we're at the beginning if(gotit is null) haystack++; else { // partial match, but not complete so no good // start over, including the current *haystack where = needle; gotit = null; } } } return null; } unittest { void test(string haystack, string needle, size_t line = __LINE__) { import core.stdc.string; import std.conv; // .ptr is fine because the test uses all string literals, which are 0 terminated automatically assert(strstr(haystack.ptr, needle.ptr) is mystrstr(haystack.ptr, needle.ptr), to!string(line)); } test("foobar", "f"); test("foobar", "ob"); test("foobar", "o"); test("foobar", "a"); test("foobar", "ar"); test("foobar", "ea"); test("bedtime bedazzled!", "beda"); } I've heard of Boyer-Moore but don't actually know it, so I'd have to look it up! The brute force loop is simple enough though without needing a great deal of thought, though my first run through did actually fail the unittest, since I neglected the commented section in the else{} part. So if I was doing it on a whiteboard, (or didn't immediately use "foo" as a test variable, with the repeated o) I might not have actually caught that!
Jun 21 2013
prev sibling next sibling parent "QAston" <qaston gmail.com> writes:
On Thursday, 20 June 2013 at 12:16:54 UTC, deadalnix wrote:
 On Thursday, 20 June 2013 at 10:13:53 UTC, Jacob Carlborg wrote:
 On 2013-06-20 00:47, Nick Sabalausky wrote:

 - Writing a unittest first forces the API to be designed 
 before the
  implementation is written. But implementation is necessary 
 to flush
  out unexpected design requirements (If you think you can 
 always
  come up with an appropriate design and interface before
  implementing, then you're just plain wrong). Sometimes the
  appropriate design and API is obvious. Sometimes it isn't. 
 When it
  isn't, then TDD skirts dangerously close to some of the 
 problems of
  "Waterfall Model". Sure, TDD advocates 
 refactoring-as-needed, but I
  can do that with or without TDD.

Just because you have written a test doesn't mean you cannot change it. Perhaps you come up with a better API design, then change the tests.

When I start something, it isn't always clear what mental model of the problem fit best the problem. I usually wait until I know I have a consistent mental model of the problem to write test. Otherwise, tests tend to get into your way to change the API( because you need to change tests as well. I start writing test when I know what kind of API make sense (it isn't always finished, but I know what it will look like overall). Which lead to TITMOD, test in the middle of dev.

You should write a book on that, it'd be a total paradigm shift for the non-yet-believers of TITMOD.
Jun 21 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Friday, 21 June 2013 at 21:33:43 UTC, Andrei Alexandrescu 
wrote:
 If there's any need to reach for documentation, the interviewer 
 has failed. When interviewing we (at Facebook) ask problems 
 that are likely to appear in a normal day's work, but for which 
 the typical libraries don't help. (E.g. many libraries don't 
 can't help with implementing unstable remove (see 
 std.algorithm).)

Amongst other thing, I did a merge sort and a quicksort in my FB interviews. It is fair to assume they can be found as libraries. But overall the process is really accurate to assert what a dev can do. Most question were technically challenging, but weren't tricks or overly complicated and useless. For the anecdote, my quicksort was buggy, but my interviewer convinced me it wasn't - when later check (after the interview) demonstrated is indeed was.
 Also it's fair to ask about implementing a stdlib function 
 itself if the interview concerns some systems-level work; e.g. 
 brute-force strstr() is fair game and I think any engineer 
 should be able to lift it off the ground quickly (to my dismay, 
 only a fraction can). Paradoxically use of stdlib functions may 
 actually hurt; I've seen people who e.g. call strlen() in a 
 loop in order to implement strstr()!

When i was doing interview, I used to ask people to program a function that sort an array of integer. No constraint of performance or anything, the good old 5 lines bubble sort would have been accepted. Most can't.
Jun 21 2013
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 6/22/13, Walter Bright <newshound2 digitalmars.com> wrote:
 Can I play, too? Mine from the Digital Mars C library. Haven't looked at it

 #if 0 /* Smaller but slower under many circumstances. */
 char *strstr(const char *s1,const char *s2)
 {   size_t len2;
      size_t len1;
      char c2 = *s2;

      len1 = strlen(s1);
      len2 = strlen(s2);
      if (!len2)
 	return (char *) s1;
      while (len2 <= len1)
      {
 	if (c2 == *s1)
 	    if (memcmp(s2,s1,len2) == 0)
 		return (char *) s1;
 	s1++;
 	len1--;
      }
      return NULL;
 }

Some things i've noticed about this implementation: - c2 is assigned by pointer dereference, but then strlen is called on s2 and you may get an early exit, meaning c2 was not needed at this point. I'd move c2 below the strlen call. - The memcp doesn't have to compare len2 amount of chars if you've already compared the first character, it can call memcmp(s2+1, s1+1, len2-1). Although I'm not sure whether it's faster to do it this way (maybe it is, if those variables are actually registers..). And notes about both implementations: - You're casting a const char * to a char * in the return, you should instead provide an overload of strstr taking char* and returning char*. - You seem to have missed the most basic and most important check. You didn't attempt to compare the memory address of both pointers against each other before doing any other work. If they both point to the same memory, the strings are guaranteed to be equal.
Jun 22 2013
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Saturday, 22 June 2013 at 11:01:53 UTC, Andrej Mitrovic wrote:
 - You're casting a const char * to a char * in the return, you 
 should instead provide an overload of strstr taking char* and
 returning char*.

I'm pretty sure the C standard says to do it with the const arg but mutable return. http://en.cppreference.com/w/c/string/byte/strstr
Jun 22 2013
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 6/22/13, Adam D. Ruppe <destructionator gmail.com> wrote:
 On Saturday, 22 June 2013 at 11:01:53 UTC, Andrej Mitrovic wrote:
 - You're casting a const char * to a char * in the return, you
 should instead provide an overload of strstr taking char* and
 returning char*.

I'm pretty sure the C standard says to do it with the const arg but mutable return. http://en.cppreference.com/w/c/string/byte/strstr

Ah, I momentarily forgot C doesn't have overloading, which is probably where Walter's function was implemented in. In C++ there are overloads of this function: http://www.cplusplus.com/reference/cstring/strstr/
Jun 22 2013
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Saturday, 22 June 2013 at 13:55:26 UTC, Jérôme M. Berger wrote:
 	I haven't tried running it, but this looks to me like it won't 
 find "ababc" in "abababc"...

You're right. I should have went backwards all the way, not just in the one case. This passes all the tests: inout(char)* mystrstr(inout(char)* haystack, const(char*) needle) { assert(haystack !is null); if(needle is null) return haystack; const(char)* where = needle; inout(char)* gotit; while(*haystack) { if(*haystack == *where) { if(gotit is null) gotit = haystack; where++; if(*where == 0) return gotit; } else { haystack -= where - needle; where = needle; gotit = null; } haystack++; } return null; }
Jun 22 2013
prev sibling next sibling parent "QAston" <qaston gmail.com> writes:
On Friday, 21 June 2013 at 16:49:26 UTC, H. S. Teoh wrote:
 On Fri, Jun 21, 2013 at 05:48:25PM +0200, Jacob Carlborg wrote:
 On 2013-06-21 16:56, H. S. Teoh wrote:
 
+1, me too! I can say that 85-90% of what I do at work today, 
I
learned from my personal coding projects, not from the CS 
courses I
took in university. (That's why I like to joke about CS grads 
knowing
more about uncomputable problems than computable ones...)

It feels like there's something wrong with the world here :)

Only "something" wrong? ;-) To be fair, though, I can understand why university programs are that way: their goal is to produce more researchers and professors who may join the faculty and produce more research. So they focus more on the theoretical aspects of computer science in order to produce such candidates, whereas focusing on more practical aspects is something for technical colleges whose goal is to produce industry workers. Unfortunately, in North America at least, the technical colleges lean more toward stuff like "how to use MS Word", "how to organize your company data in MS SQL", rather than _real_ programming, and universities that actually *teach* real programming are more interested in finding solutions to uncomputable problems than teaching students how to solve computable ones, so there's a gap in the area of producing qualified industry coders who can write functional software. There *are* pockets of competent programming education here and there, of course, but this is the impression I get from the general situation. T

Yay, a uni bashing thread. I'm a student and I like it, so here's some subjective opinion. I don't know how it's in USA, but where i live we have 2 possibilities of getting CS education at uni level: a course at "pure" university and a course at something we call "university of technology". Basically the former focuses much more on academic problems, the latter grants you with "engineer" title with all that implies - you learn how computers work from electrical level up to the high abstractions of disciplines like AI or image processing. I've been interested in programming long before I went to the "university of technology". I went there to expand my knowledge on the topic. It worked out great for me, because unlike in our "highschool" I didn't have to learn stuff unrelated to my interests. I've learned a lot and met some very skillful people. Many of them bash unis just like you guys, yet they still gained many skills there - but that's typical - I've heard a lot of whining during my entire education. I want to point out that having a CS course alone doesn't make you a good coder. First - because CS is not only about coding, that's a wide discipline. Second - your attitude is what really matters. Noone's going to make you a (very)good C programmer in 90h course - you just get enough to grasp important concepts to be able to easily expand on your own in areas you like. If you don't like CS, yet you finish the course, you've lost X years of life for getting a sheet of paper and some opportunity to get a job which you probably won't like or won't be good at. Third - unis are really not about producing enterprise workers, you don't need much knowledge to write apps which throw around data between crud app and db. Instead they give you ability to use common scientific methodology, a complete overview of CS, means to study (or even push forward) the discipline you're interested in and most important, they teach how stupid everyone (especially you - that's a painfull lesson) really is. I'm not saying that you can't get all of this on your own, yet it's easier that way. I was a self taught programmer, but I feel that my knowledge is more complete now (and some of the knowledge is very important for a systems programmer). It's obviously the same other way around - you can pass all the exams and get nothing from that. It's up to you what you're going to do with free time. Important note: public unis are free in my country, I probably wouldn't study otherwise.
Jun 22 2013
prev sibling next sibling parent "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Friday, 21 June 2013 at 21:33:43 UTC, Andrei Alexandrescu 
wrote:
 Also it's fair to ask about implementing a stdlib function 
 itself if the interview concerns some systems-level work; e.g. 
 brute-force strstr() is fair game and I think any engineer 
 should be able to lift it off the ground quickly (to my dismay, 
 only a fraction can). Paradoxically use of stdlib functions may 
 actually hurt; I've seen people who e.g. call strlen() in a 
 loop in order to implement strstr()!

I think it's fair game whatever kind of work you are doing. The tests aren't so much about the algorithm, but showing that the candidate thinks about edge cases, interfaces, possible failure cases etc. These are things you need to think about even if your job is just to essentially glue libraries together. Simple tasks like strstr just provide a nice small task that test these things.
Jun 22 2013
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 6/22/13, Walter Bright <newshound2 digitalmars.com> wrote:
 It's wrapped in #if 0 ... #endif ? :-)

Hehe. I should totally use that as an argument in a future interview. I'll wrap all my coding-answers in '#if 0', and if they're wrong I'll just say "oh, but this code doesn't *actually* exist!". :p
 The function bodies would be identical - why have two of them?

Yeah nevermind, it's C, I was thinking about C++. Then again D would blow them both out the water since it has inout.
 Most important? I've never even run across this case.

Ok, maybe not that important. It's likely a rare case. Btw, does anyone know if the typical memcmp implementation tries to compare the pointers' addresses first to see if they match?
Jun 22 2013
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Saturday, 22 June 2013 at 16:38:31 UTC, Andrei Alexandrescu 
wrote:
 It's still buggy.

What's the test case? I also forgot the empty string, blargh. Let's scrap it.
 Overly complicated, too - at 21 lines, it's about twice as 
 large as the canonical implementation.

Huh, even the shortest impl I can think of is about the same length: inout(char)* mystrstr(inout(char)* haystack, const(char*) needle) { assert(haystack !is null); if(needle is null) return haystack; while(*haystack) { auto h = haystack; const(char)* n = needle; while(*n == *h && *n && *h) { h++; n++; } if(*n == 0) return haystack; haystack++; } return null; } I like this a lot better because it is more straightforward. My first draft was trying to avoid looking at the same character twice, but obviously that didn't work so fixing that ended up doing the same thing as this anyway. I guess better to KISS than be embarrassingly wrong, eh? You could do it in about ten lines like this: inout(char)* mystrstr(inout(char)* haystack, const(char*) needle) { while(*haystack) { auto h = haystack; const(char)* n = needle; while(*n == *h && *n && *h) h++, n++; // LOL COMMA OPERATOR if(*n == 0) return haystack; haystack++; } return null; } But I really like the null check. Yes, I know the hardware does it, but it is a pain in the butt to fire up a debugger when a null hits you in the middle of something. The nice assertion failure message is so much faster to handle.
Jun 22 2013
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Sunday, 23 June 2013 at 16:07:05 UTC, Andrei Alexandrescu 
wrote:
 Still buggy. The empty string must be a prefix of any string 
 including the empty string.

Huh. Well, that makes sense. Just change while to do while and you've got that. Probably compiles to exactly the same code as your implementation in the link at that point. But this probably told you more about my methods than getting it right the first time would have: I'll just slap something together that's good enough for the test case(s) fed to it and then change it if new stuff arises where it fails. One of the nice things about software is you generally don't have to get it right the first time since it is so easy to change. This is TDD-lite isn't it! lol
 http://www.serversidemagazine.com/news/10-questions-with-facebook-research-engineer-andrei-alexandrescu/.

I definitely remember reading that when you posted it a while ago because I have code that does that <?d writeln("hello, world"); ?> laying around. http://arsdnet.net/dcode/dhp.d It works by just reading the file and translating everything outside the <?d?> into a giant writeln(string literal), pasting in the D code, then compile+running it, inserting a bunch of imports so it works. dom.d now supports those kinds of tags, it could probably be even nicer, especially with a D lexer so it doesn't think "?>" is the same as ?>. But as I'm sure I said then, this is a bad idea anyway since mixing code and html data leads to ugliness, whether the code is php or D.
Jun 23 2013
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Sunday, 23 June 2013 at 16:28:56 UTC, Andrei Alexandrescu 
wrote:
 foreach(ln; stdin.byLine(KeepTerminator.yes)) {

Yeah.
 Would be awesome if an Apache extension would make it trivial 
 to write Web pages in D.

Just use cgi or fastcgi, both are really easy to configure on apache (often needing nothing more than copying your executable into /cgi-bin/, or adding three lines to ,htaccess), and being separate processes, if you crash them it is no big deal. No need to do set up a reverse proxy or get your system administrator to load strange new Apache modules. Or if you are just playing, you can skip apache altogether and use a D http server, like my cgi.d and vibe.d both offer (vibe.d's scales way better but mine is simpler). But I kinda want to play with this now with shared libraries just for something to do so maybe I will.
Jun 23 2013
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Sunday, 23 June 2013 at 17:34:52 UTC, Adam D. Ruppe wrote:
 Would be awesome if an Apache extension would make it trivial 
 to write Web pages in D.

Just use cgi or fastcgi, both are really easy to configure on apache (often needing nothing more than copying your executable into /cgi-bin/, or adding three lines to ,htaccess), and being separate processes, if you crash them it is no big deal. No need to do set up a reverse proxy or get your system administrator to load strange new Apache modules. Or if you are just playing, you can skip apache altogether and use a D http server, like my cgi.d and vibe.d both offer (vibe.d's scales way better but mine is simpler).

Actually vibe.d embedded HTTP outperforms Apache. With high concurrency - a lot. I can see reasons for wanting to keep it behind reverse proxy like nginx, but Apache? I dream of the day this freaking monstrosity dies together with web app infrastructure it has developed. Please, don't help it survive with D module support. :(
Jun 23 2013
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Sunday, 23 June 2013 at 17:54:01 UTC, Andrei Alexandrescu 
wrote:
 On 6/23/13 10:34 AM, Adam D. Ruppe wrote:
 We should do what php does, it was very successful. I assume 
 it's a dynamic library.

And what reason behind this other than "millions of lemmings can't be wrong"? This approach is disaster when it comes to high load.
Jun 23 2013
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Sunday, 23 June 2013 at 18:02:04 UTC, Dicebot wrote:
 I can see reasons for wanting to keep it behind reverse proxy 
 like nginx, but Apache?

Generally, I don't trust random http servers connected to the open internet for correctness, stability, security, and logging. Especially not my code, There's a lot of the http protocol I never implemented, and putting apache, nginx, IIS, whatever, I'm just using Apache here because Andrei mentioned it, out in front will filter some of that out before it gets to my app. And who knows what kinds of hidden bugs my code has, so having a tested and reliable server out there to revive my process if it dies is nice too.
Jun 23 2013
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Sunday, 23 June 2013 at 18:24:05 UTC, Andrei Alexandrescu 
wrote:
 Would separate processes work better under high load? Educate 
 me.

One nice thing is you can spread separate processes across several machines. Another advantage of *cgi|embedded_httpd is that it is portable to other web servers too, you can just drop it into a shop that uses nginx or Microsoft IIS and expect it to work, whereas an Apache module is generally apache only. I think it is just an accident of history that mod_php ever got used. Classic cgi implementations were still slow enough (especially with an interpreted language) that people wanted to try something else, but the other world of options hadn't taken root yet either (I think mod_php even slightly predates fastcgi's introduction), and continues to exist just out of inertia.
Jun 23 2013
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Sunday, 23 June 2013 at 17:54:01 UTC, Andrei Alexandrescu 
wrote:
 I assume it's a dynamic library.

PHP can work as fastcgi too.
 But I kinda want to play with this now with shared libraries 
 just for
 something to do so maybe I will.

Awesome!

Eh, not so much. I started playing and got a thing saying Runtime.loadLibrary is not yet implemented on posix... so went to do it myself and got random segfaults, relating to the garbage collector (and yes, I tried Runtime.initialize). I've never done a shared library plugin before so I could be doing something wrong, but it seems to me that druntime said it wasn't implemented yet for a reason, must still have some work to do there. Oh well, maybe I'll try again next release.
Jun 23 2013
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
BTW I should mention, I wasn't actually trying to do an Apache 
module. I wanted to do a D server that watches the files for 
changes, then recompiles them as needed and reloads the resulting 
file as a shared lib.

I could just run the compiled executable too, cgi style, but here 
I was more interested in playing with shared libraries than just 
getting it to work.
Jun 23 2013
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Sunday, 23 June 2013 at 17:54:01 UTC, Andrei Alexandrescu 
wrote:
 Awesome!

I realized initializing the runtime might have been a mistake when loading the .so and moreover I was using the wrong calling convention. Fixed that and now the shared library thing works. On Linux at least, I hardcoded ".so" so don't try it anywhere else. Here's some code: http://arsdnet.net/dcode/dhp.zip unzip it, hit make. dhp2.d is the main file here, it also uses some helper libraries you can see in the arsd folder. Then run ./dhp2 in another window or something. It is a long running process that serves http. Head over to your browser and surf to http://localhost:8085/ It will take a couple seconds to load the first time, because it needs to compile the code. Subsequent loads will pull it from a cache and be faster. Anyway it will show you some stuff and a link. These pull from the files in the zip called index.dhp and dhpt.dhp. index is just html, no D code, although it will compile it since this server compiles everything. dhpt.dhp actually includes some D: <p>Hello, <% cgi.write(cgi.request("name", "user")); %>, happy to see you.</p> Note: you access that by going to localhost:8085/dhpt, NOT /dhpt.dhp. It strips out dots from the url as a way of sanitizing the filename so keep it simple. It uses ASP style <% %> tags instead of <?d ?> because my dom.d already understands them. (BTW this parses the .dhp files to be well-formed xml, so if you mismatch tags, it will throw. It might be fun to put the DOM node in scope to inspect too). There's a Cgi cgi in scope in the function it builds here. Use it to do communication instead of writeln() etc., as seen in this example. Here's where the shared library magic comes in: feel free to edit one of those .dhp files, or create your own, and go back to it in the browser. It will recompile and present it to you without having to restart the server. That's kinda cool. The downside is if you segfault in here it will take the whole server down so don't do that. If you fail compiling though, it will actually read dmd's output and translate the filename and line number to match the .dhp file input, instead of the .d file dmd actually sees. So it feels less like a filthy hack then. Feel free to look at index.d and dhpt.d in that same folder after you browse to them to see what the generated code looks like. Soooo yeah. I'll probably never use this, dom templates rok so much more than anything asp/php style, but if you wanna play, feel free and let me know if you want more features or see bugs. Probably won't be too hard to fix up now that it is started.
Jun 23 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Monday, 24 June 2013 at 02:39:30 UTC, Andrei Alexandrescu 
wrote:
 On 6/23/13 11:51 AM, Adam D. Ruppe wrote:
 I think it is just an accident of history that mod_php ever 
 got used.
 Classic cgi implementations were still slow enough (especially 
 with an
 interpreted language) that people wanted to try something 
 else, but the
 other world of options hadn't taken root yet either (I think 
 mod_php
 even slightly predates fastcgi's introduction), and continues 
 to exist
 just out of inertia.

OK so what's the way to go now? One process per request? Seems heavy to me seeing as most requests last very little.

That is what apache used to do when mod_php was introduced anyway. This whole conversation has gone out of control :D mod_php was a way to reduce the communication overhead and avoid 2 process per request in the first place (instead of one). It become plain obvious that no silver bullet exists here. The solution seems to be a mix of fibers/threads and processes where number of each depends on your workload and the actual hardware it is running on. Most high performance and high scalability system are converging toward one flavor of that pattern.
Jun 23 2013
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Sunday, 23 June 2013 at 18:24:05 UTC, Andrei Alexandrescu 
wrote:
 On 6/23/13 11:04 AM, Dicebot wrote:
 On Sunday, 23 June 2013 at 17:54:01 UTC, Andrei Alexandrescu 
 wrote:
 On 6/23/13 10:34 AM, Adam D. Ruppe wrote:
 We should do what php does, it was very successful. I assume 
 it's a
 dynamic library.

And what reason behind this other than "millions of lemmings can't be wrong"? This approach is disaster when it comes to high load.

Would separate processes work better under high load? Educate me. Andrei

It is not about "process vs dynamic library", it is about basic threaded architecture used in Apache. Every single network server that I know that cares about performance uses some sort of event model and avoids any context switches at all possible costs. It does not matter where request processing code is, only how it fits in bigger picture. Actually, server software with most impressive performance I have seen so far was implemented as single barebone process with all network stack and event library embedded. Slightly similar idea was propsoed here - http://highscalability.com/blog/2013/5/13/the-secret-to-10-million-concurrent-connectio s-the-kernel-i.html (found via reddit)
Jun 24 2013
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Sunday, 23 June 2013 at 18:42:12 UTC, Adam D. Ruppe wrote:
 On Sunday, 23 June 2013 at 18:02:04 UTC, Dicebot wrote:
 I can see reasons for wanting to keep it behind reverse proxy 
 like nginx, but Apache?

Generally, I don't trust random http servers connected to the open internet for correctness, stability, security, and logging. Especially not my code, There's a lot of the http protocol I never implemented, and putting apache, nginx, IIS, whatever, I'm just using Apache here because Andrei mentioned it, out in front will filter some of that out before it gets to my app.

Yes, that is why I have said that understand reasoning to use somthing like nginx or lighttpd - it is better to work behind something that has proper implementation of HTTP protocol and focus on application needs. I am using nginx for this on my own and perfectly satisfied with it. But using Apache is just throwing away most part of performance achived by your backend, for nothing. And creating security issues in case your app is run via Apache module is it typically runs from http user in such scenarios and using app-specific UNIX access policies becomes much harder.
Jun 24 2013
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Monday, 24 June 2013 at 02:39:30 UTC, Andrei Alexandrescu 
wrote:
 On 6/23/13 11:51 AM, Adam D. Ruppe wrote:
 I think it is just an accident of history that mod_php ever 
 got used.
 Classic cgi implementations were still slow enough (especially 
 with an
 interpreted language) that people wanted to try something 
 else, but the
 other world of options hadn't taken root yet either (I think 
 mod_php
 even slightly predates fastcgi's introduction), and continues 
 to exist
 just out of inertia.

OK so what's the way to go now? One process per request? Seems heavy to me seeing as most requests last very little. Andrei

One process/thread per CPU core + async I/O. FastCGI as simple alternative (it is similar to CGI but does not respawn process after processing request).
Jun 24 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Monday, 24 June 2013 at 09:07:54 UTC, Dicebot wrote:
 Actually, server software with most impressive performance I 
 have seen so far was implemented as single barebone process 
 with all network stack and event library embedded. Slightly 
 similar idea was propsoed here - 
 http://highscalability.com/blog/2013/5/13/the-secret-to-10-million-concurrent-connectio
s-the-kernel-i.html 
 (found via reddit)

BTW, this article is full of bullshit and hand wavy stuff. I wanted to write a article destroying it, but I then decided that I had other thing to do than loose my time :D
Jun 24 2013
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Monday, 24 June 2013 at 09:22:58 UTC, deadalnix wrote:
 BTW, this article is full of bullshit and hand wavy stuff. I 
 wanted to write a article destroying it, but I then decided 
 that I had other thing to do than loose my time :D

Yeah, it has plenty of bullshit but core idea is the same as one in project I have been engaged in and I can't write article about latter because of NDA, so using this an alternative available in the internet ;)
Jun 24 2013
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Monday, 24 June 2013 at 02:39:30 UTC, Andrei Alexandrescu 
wrote:
 so what's the way to go now? One process per request? Seems 
 heavy to me seeing as most requests last very little.

Performance wise, I've found heard very good things about async i/o (like vibe.d) and had very good experience with a process pool (on linux anyway, cgi.d's fastcgi and new embedded_httpd_processes do this, the latter hitting 6,000 requests per second for hello world on my machine!). The process pool forks ahead of time, then they take connections as they become available. Basically it puts the concurrency in the kernel's hands, which does a good job. The zip I linked to yesterday does a simple one thread per connection, multiple requests per connection model. Doesn't scale very well, but works well and with simplicity on light loads.
Jun 24 2013
prev sibling next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Thu, 20 Jun 2013 00:15:50 +0200
"Szymon Gatner" <noemail gmail.com> wrote:

 On Wednesday, 19 June 2013 at 21:59:21 UTC, Jonathan M Davis 
 wrote:
 I _do_ agree with writing the tests fora function as soon as 
 the function is
 done, in which case, you're likely going to have to do more 
 work on the
 function, since it'll probably fail the test, and you'll 
 probably improve the
 tests some more at that point as well. But I completely 
 disagree with writing
 the test before the code, which is one of the key features of 
 TDD as its
 always been explained to me.

It forces you to think about parameters it should take and value(s) it should return first.

TBH, I've never found a lot of value in that. What I find "write tests first" does is force you to *guess* about params and return values. In my experience, implementation automatically roots out the necessary set of ins and outs.
 
 API change cost is way higher than fixing a bug inside a function 
 or a class

I very much disagree with that, unless you're talking about "write implementation, use it everywhere, do a big public release, and *then* write unittests", which is a bad thing to do anyway. Fixing a bug can often be non-trivial, but changing an API right after, or while, you're implementing it is always trivial (although it's less trivial if you've already written a lot of tests that rely on it).
 also you will often realize that what you initially 
 though was necessary turns out not to be after all.
 

Like I said above, I find that writing the implementation does a far better job of that then making guesses about what will/won't be necessary before implementing.
Jun 27 2013
prev sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Sat, 22 Jun 2013 06:27:34 +0200
"QAston" <qaston gmail.com> wrote:

 On Thursday, 20 June 2013 at 12:16:54 UTC, deadalnix wrote:
 Which lead to TITMOD, test in the middle of dev.

You should write a book on that, it'd be a total paradigm shift for the non-yet-believers of TITMOD.

The coolest acronym in programming history, ever :)
Jun 27 2013