Statistics for the reasoning impaired

From The Independent:

Using the IQ tests taken from the military records of 241,310 Norwegian conscripts, the scientists have found that eldest siblings are, on average, significantly “more intelligent” than second-borns. It may not seem like much, but 2.3 points on the IQ scale – the average difference between first and second siblings – could be enough to determine whether or not someone gets into a good college.

Whatever the cutoff mark is to get into a given college, there will always be someone who just misses out by a sliver. Any difference in IQ could be the difference between getting into a good college or not, especially as most academic entrance tests examine the same skills as IQ tests.

There is another flaw in this article. It’s not exactly an error, as we shall see, but it is worth comparing the quote above with what the actual abstract says:

We show that intelligence quotient (IQ) score levels among nearly 250,000 military conscripts were dependent on social rank in the family and not on birth order as such, providing support for a family interaction explanation. (my emphasis)

How is it possible that the journalist has come to the opposite conclusion to the research? The answer is that he hasn’t. Later in the article he goes on to explain that the research suggests that it is not strict birth order but authority within the family that matters. First-born children tend to be given a higher rank within the family, and it is the role within the family that appears to account for the IQ difference. So while it is true that the journalist is aware of this and has reported on it, the problem remains that in report after report (check Google to see what I mean), the wrong explanation is given first, discussed at length with little indication that the explanation is wrong, and only later is the correct interpretation given. One particularly bad piece at Boston.com takes eight paragraphs to get to the finding about social rank, and compounds my irritation with an even worse example of the statistical error above.

But the extra smarts conferred by birth order are significant enough in the broad sense. Spread over tens of thousands of individuals, the tiny difference could translate into a higher likelihood of acceptance into better colleges, improved chances of landing a good job, or even better luck at winning a brainy spouse, according to some child experts.

Which is the same bulldust, at greater length, and with new bulldust added. Winning a brainy spouse? Give me a break. Show me the person who can recognise a difference of 2 IQ points in others. Then show me that this difference affects mate selection. Then I might take it seriously as a finding, but I still won’t consider it to be a social process of any significance. Then the Boston.com article gets even worse:

With a sample size of nearly a quarter-million test subjects, that’s compelling scientific evidence.

No no no no no! Sample size is only one part of assessing the strength of a given finding. The real test of evidence is not sample size but significance score, which is not reported in this article here (and unfortunately I can’t access). You can have a billion samples and still not have compelling evidence. And even if there is strong significance, it does nothing to exclude biases or confounding variables.

Fortunately other journalists, such as those as the Washington Post and CNN, reported the social rank upfront as the key finding and avoided blathering to readers about how important this trivial IQ difference is. Kudos to them.

Kudos also to this pseudonymous poster who, despite providing compelling evidence that his parents deserve a refund, appears to have more insight than most professional commentators:

This type of ’scientific study’ is always so subjective and time wasteful. Every little factor of family life comes into play. I am the middle child. I went to private school. I am tested smarted then my older brother, who is a twin (faternal) to my very much smarter older brother. My younger brother dropped out of high school to go to college, and my youngest brother was in the youngest of us all to own his own home. We were all raised in Kentucky. Try am fit those oddities into the ‘findings’ and see how it is squewed.

Sorry, comments for this entry are closed at this time.