I’m currently reading Exploring Requirements: Quality Before Design by Donald Gause and Gerald Weinberg. The book was written twenty years ago but it is still spot-on, which is really incredible for a software-development related book. Gause and Weinberg described an experiment from one of their workshops that was supposed to show how even the simplest things can cause a lot of misunderstanding. I honestly could not believe the results, so I decided to repeat it yesterday. The outcome absolutely amazed me.

They conducted the experiment by showing a seven-point star picture before one of their seminars, then after a lecture and a coffee break asked the attendees to tell them how many points the picture had. They got quite a wide spread of results. According to Gause and Weinberg, everyone knows what a five-point star looks like but seven-point star was not that usual, so people remembered it differently and recalled different images, especially after the coffee break. That accounted for the spread. People also understood the task differently, which explained clusters in the answers.

I am not so interested in people not remembering things off the top of their head, but I wanted to check the problem of differences in understanding. After my talk on Selenium yesterday, I handed out index cards and put a picture of a standard five-point star on the projector, asking people to write down how many points they see in the picture there and then. I used an image of a familiar figure (the one on the right) and it was on display while people wrote down their answers. So any differences in answers could only be caused by people understanding the task differently.

After initial reluctance and my explaining that the question might sound stupid and obvious, but I’d really like people to answer it anyway, I got about 50 cards back. And the results were simply amazing. I expected that most people would write 10 as the number, and that a few would possibly write “infinite” referring to the mathematical axioms that any line contains an infinite number of points. But the answers were much more surprising.

Most people (25) voted for 10 points in the star, counting inner and outer points. The second most popular answer (7) was five points, where people counted just the outer points. Number 14 got a lot of votes as well — one of the people at the pub after the talk explained to me that this is probably the ten points on the star and four corners of the picture. There was a single vote for number 9, which can theoretically be explained by five outer points and four corners of the picture, but there were also four cards which I can not explain at all: numbers 11 and 15 got two votes each.

If anything, this experiment has confirmed that even a simple thing as a familiar image and a straight-forward question can cause a lot of misunderstanding, or rather differences in understanding. Some people thought that the edges of the image counted as part of the image, some did not consider them. Some people just counted outer points, some counter inner as well. This is essentially why giving people screenshots or wireframes with simple descriptions does not work as an effective technique to pass knowledge. This is a very effective demonstration why we need workshops that stimulate discussion to define requirements and specifications, ideally using realistic examples which could then be converted to acceptance tests.

I'm Gojko Adzic, author of Impact Mapping and Specification by Example. To learn about discounts on my books, conferences and workshops, sign up for Impact or follow me on Twitter. Join me at these conferences and workshops:

## Product Owner Survival Camp

- Oslo, NO, 5-6 May
- Munich, DE, May 20-21
- Rijswijk, NL 3-4 June
- Frankfurt, DE, 14-15 October
## Specification by Example Workshop

- Vienna, AT, May 13-14
- London, UK, May 15-16
- Oslo, NO, 12-13 Jun
- Oslo, NO, 16-17 October
## Conference talks and workshops

- Craft, Budapest, HU, April 24-25
- Vienna, AT, May 13

How about this…

- 5, 10, and 14 are what you say

- 9 and 11 are miscounts for what should have been perceived as 10

- 15 is a miscount on the 10 points on the way to 14.

I’m sad to say that I came up with 11 first as well, so I can shed some light. You visually try to figure it out by starting at the top, and with your finger or mouse, trace the star, counting the outer and inner points.

But the mistake is, once you get back to the start, you count the top point [again]. This gives you the 11.

“This is essentially why giving people screenshots or wireframes with simple descriptions does not work as an effective technique to pass knowledge. This is a very effective demonstration why we need workshop that stimulate discussion to define requirements and specifications, ideally using realistic examples which could then be converted to acceptance tests.”

I’m not sure I’d interpret the results that way. You’re asking people in the presentation to give the opposite of what you’d ask them as a deliverable in a project. You wouldn’t hand the development team a finished product and ask them to describe it. You’d tell them you want a 5 point star and they’d deliver your picture, more than likely.

If you wanted a 5 point star in a box, you could tell them so.

I doubt you’d be disappointed in what you got if you asked for a 5 point star.

Regarding 15: some people may have looked at one arm of the star, saw it had 3 points, and multiplied by 5 to get that number, forgetting or ignoring that each arm shares two of its points with 2 other arms.

Interesting test, but it doesn’t show to me what it shows to you. =)

What’s the answer? I counted 146.

Great post, it’s got me wanting to try the experiment sometime soon !

Would be interesting to know how people came up with 11 and 15…

When you’re counting the 10 points, it is easy to re-count the point you start from. Being unsure of whether there are fewer inner points will cause the person to manually count the points, and to over-count the starting point by returning to it (while completing the polygon, as if drawing it) during the manual count.

This accounts for 11 and 15 (11 +4 corners)

I counted 11.

I was coming from a computer graphics perspective – to draw a 5-pointed star, you have to plot point A, then plot the remaining points all the way round, and then plot A again.

“The second most popular answer (7) was five points, where people counted just the outer stars.”

I think you mean “outer points” in that.

That’s a very interesting experiment. It might be fun to have people write a justification for their answer, if you ever do it again, both to see if it changes the clustering at all, and also to see just what the heck some of them are thinking.

The logic behind 15 could be each arm has the central and two flanking points. Times 5 that would be 15 points.

11 makes sense if you consider plotting the outline as a polygon, drawing lines between each point – you have to draw the final line between point 10 and point 1 again or you end up with the final line segment unjoined – this might be what is going on, as you’re using the first point twice.

I can think of a total of 15+x, if we’re thinking poetically:

- 10 for the vertices of the star.

- 1 for the center point of the star (balance point)

-4 for the corners of the picture

-x for whatever point or points you’re trying to make by showing the picture.

11 and 15 would be counting the center “point”

11 points can be explained easily… Draw the star as a line strip instead of a line loop, and you would need to input 11 points to make it (the first point has to be repeated at the end.

I’d say 11 is a miscount.

15 could be the inner and outer points (10) + 5 larger arrow shapes comprised of a point, and it’s two corresponding opposing points.

the answer of 15 probably refers to the number of points were the lines of the star to be joined together, as in a pagan pentagram. What is the answer then? Logically it has to be 5, because this star literally has 5 points, no?

I suspect the 11,15 guessers were counting the center of the star as a point.

Like Laurie, I thought of it as a vector graphics problem and got 11. You would need to include the first point as the last point as well.

lewey and icecow got it right. 11 & 15 answers are counting a center point. it’s not enough to establish the boundary. you must also establish it’s location…hence a center point. a star pattern is a circular (or polar or radial) array…and circle is partly defined by it’s center.

That’s just weird. A five pointed star has five points. Anything else is just people trying to be to clever for their own good.

@Sammy Larbi yeah you’re absolutely correct here.

If you take the example from your starting point it becomes a matter of translating words into an object instead taking an object and describing it in words.

He may say draw me a 10 pointed star. Because the person providing the requirements thinks that a 5 pointed start has 10 points, the person drawing the star will probably draw the wrong thing.

There are a number of flaws in requirements drafting. Usually it all boils down to assumptions and vagueness. If your providing requirements, don’t be vague, if your receiving requirements, don’t assume.

Do you really not understand that a big fraction of your respondents were just having a piss at your expense?

It’s a trick question. The correct answer is ZERO. If you actually take the time to look at the picture you will see TEN distinct line segments and ZERO points.

Right. So just the noise in the responses demonstrates the point.

I can’t think of any point in a 5 point star. Other than for the illustration of the differences between (n) point stars. So 1 point really.

Now lets see how many corners can be found inside a circle.

assuming that it is impossible to create a truly mathematically perfect shape, a star would have infinite or near infinite points

i was just wondering, point isn’t the actual term you were thinking of right? its vertices. i figured this because you expected ten, there are ten vertexs (either word can be used). that is what I counted as well. So could the problem be more on the question, if a question is not specific enough, is it really the fault of the person answering?

just a thought =]

Well I got the only 20 apparently. I agree the 15 (my first number) is the outer points, plus the two points at the base of each of the 5 arms- But then I had also imagined a drawn 5- point star, with 1 line, which generally consists of a pentagram in the center. Probably pays to scroll down before figuring it out

Some other amazing things about this experiment are:

1. How many people doubt the result.

2. How many of those people don’t test it for themselves.

3. How many people don’t doubt the result.

4. How many of those people don’t test it for themselves.

5. How many people don’t vary the experiment, as you did.

All these behaviors underly lots of the problems with understanding requirements. People tend to believe what’s written down, even when testing it is quite simple. Be careful what you write down.

If the average Joe draws out a 5 pointed star freehand ( as in 1 fluid motion ), which one may do in their mind to visualize the image, it may be perceived to have 15 points.

There are 10 distinctive points, but only 3 are required to reproduce it: center, one tip, and one neighbor valley.

If it’s always like a pentagram you only need 2: center and one tip (for example).

Any kind of start can be described with only 5 numbers: center, radius 1, radius 2, number of corners, and an angle.

I hate to be a wet blanket, but this experiment is not so amazing. People were expecting it to be some kind of trick question (which it was), so they were trying to stay one step ahead by counting extra “points”.

If you asked a bunch of people “hey, how many points on that star over there?” in a way suggesting that you actually didn’t know, and just wanted an answer, and weren’t going to shout “AHA YOU FORGOT THE INNER POINTS, NO GOOGLE JOB FOR YOU” if they failed to think laterally enough, 99% of them would just say “five”.

my guess for 9:

you have of course 5 outer points and for each outer point one inner point… now the beer starts working and logic fails… and people think since you go round a circle with the points the 10th is actually the first one you made.

recently I’ve been asked a question:

what is rectangle ?

I was wondered why that question… if somebody looked onto internet first article is Wikipedia and … this is definition

In geometry, a rectangle is defined as a quadrilateral where all four of its angles are right angles.

I stopped wondering after that…

nice article,

mozda smo i zemljaci

I counted 14 points in my head after only reading the title (without seeing the image). My reasoning was “every arm of the star has a peak and two points at its base (i.e. triangle), so 5 x 3 = 15 points, but I don’t want to double count one of the starting base points, so it must be 14″.

Obviously I was being retarded, but that’s one way to reach 14 as the answer.

I actually first thought that there would be an infinite amount of points. After all, inj mathematics there truly is infinite amounts of points in any line.

There’s 90000 points. Its a 300×300 image.

I suspect the 11 was a miscount, its easy to go around the star counting and accidentally count your start/finish twice.

As for the 15 it could be down to people trying to be clever, eg:

each spike of the star has 3 points each, 5 spikes to the star, therefore 5×3 = 15, tadaa.

Obviously this missed out on the overlap but thats why its wrong

Toby

Pingback: Google Chrome: A Fresh Take On The Browser | Wombat Diet