Comments on: Bulding smart teams

By: The Wisdom of Crowds and groupthink in Agile Software Development at Mark Needham

Wed, 03 Sep 2008 14:19:21 +0000

[...] Adzic posted a summary of a talk James Surowiecki gave at Agile 2008 and it got me thinking how we use the Wisdom of [...]

By: Vinayak Joglekar

Vinayak Joglekar — Mon, 25 Aug 2008 10:34:52 +0000

While defining software requirements we often depend on the product manager or business analyst. In fact these subject matter experts are one of the busiest people in the software product development supply chain. They tend to become the bottleneck. One way of putting wisdom of crowds to work is to take opinion of the end users and aggregate it using a web 2.0 like mechanism to facilitate posting reviews and ratings for the proposed features of the product. This will not only relieve the pressure on the BAs and Product Managers – but hopefully give more accurate requirements.

By: gojko

gojko — Wed, 06 Aug 2008 09:52:26 +0000

Hi Michael, If you see me in the hall, just stop me and it would be great to meet you. Btw, Marisa and I are doing a talk on Thursday (at 16h at the Tools stage) so that might be a nice time to meet as well if we don't catch up between sessions. Regrading the scalability question, I remember someone asking about different distributions after the talk but I don't remember what the answer was. I guess that most of the keynote material, apart from the software development references, was simply taken from the presenter's book, so that might be a good place to look for an answer.

By: Michael Nygard

Michael Nygard — Wed, 06 Aug 2008 04:11:52 +0000

Gokjo,

I didn’t know you were at Agile 2008. I hope we get the chance to meet while here. I’ve admired your writing and thinking for some time now.

I missed the keynote this morning, so your explanation was very helpful to me.

One question that occurs to me is this: How does the wisdom of crowds vary depending on whether the thing to be determined is a scalable or non-scalable quantity?

From “The Black Swan”, “non-scalable” quantities are those that follow normal distributions. Like human height or weight, is becomes enormously unlikely to observe non-scalable quantities far from their mean. That means that the bulk of the probability distribution is relatively near the average value.

Scalable quantities are those like annual income, ones where the probability of finding a value 10, 100, or 1000 times the average follows a power-law distribution.

When observing a scalable quantity, people tend to treat it as intuitively similar to a non-scalable one. That is, they will dramatically underestimate the likelihood of finding a value far away from the average.

It seems to me that the wisdom of crowds should work best with non-scalable quantities. On the other hand, “lines of code in Visual Studio” is a highly scalable value. In fact, I spoke to people tonight who had guessed everything from 85,000 lines to 150,000,000. (I guessed 5,500,000, myself.)

To have the average of all guesses come so close to the real value is, frankly, almost chilling.

Cheers,
-Michael Nygard