Some early comments
Thanks for sending me “Sorry, wrong number!”. What a pleasure to read!
James Le Fanu, Medical Correspondent, Daily and Sunday Telegraph.
Get this book. It affords infinitely more entertainment than a month of insipid TV situation comedies.
Warning Signs (www.anxietycenter.com) .
Another friend of mine is Dr. John Brignell, a gentleman who has taught industrial instrumentation for twenty years at the University of Southampton in Great Britain. John has a delightful Internet site calledwhere he comments monthly about "math hysteria" and the way statistics are used by government agencies, special interest groups, and various charlatans to advance junk science and other bad ideas. When I learned that he had published Sorry, Wrong Number! The Abuse of Measurement you can bet I was eager to get a copy and I can tell you that it is both very entertaining and very instructive. If I could understand it, anyone can!
Pick of the month, November (www.bookviews.com).
I am reading your book "Sorry, wrong Number!", and it is one of the best reads I have had in a long time. In fact I am only about half way through but have started boring my students with it.
Philippe Guyard, University of Portsmouth
……a treasure. I read a little every morning at breakfast and am enjoying it. Thank you for writing it.
S Fred Singer, President, Science and Environment Policy Project (www.sepp.org) .
Thanks for an entertaining read in “Sorry Wrong Number”. You lost me to a certain extent with the UK university content but overall I enjoyed your style. Of course my wife thinks I am a lost cause for thinking about this stuff let alone BUYING a book on the subject. Congratulations and keep up the good work.
Graham Austin, Australia
……should be required reading for students of statistics and for all postgraduate students.
Dr B. M. Craven, Reader, Newcastle Business School, University of Northumbria.
The Sunday Telegraph December 17 2000
Number crunching that makes you feel ill
By Dr James Le Fanu
I have recently reached that well recognised parental milestone – being asked for help with the maths homework. No doubt I will soon be out of my depth, but for the moment I am entranced –in a way I never was as schoolboy – by the beauty and clarity of simple mathematical calculations. Nor is this merely theoretical knowledge, as Western civilisation, and particularly its architecture, is rooted in the fact that the area of a circle can be deduced by squaring its radius and multiplying by pi.
The main application of maths to medicine is, of course, in the field of statistics, which can, again with considerable elegance, reveal what was previously obscure. Thus, when the number of deaths in a town or region are recorded over a period of time, it is possible to see they may represent, in aggregate, the distinct biological phenomenon of an infectious epidemic.
The great power of this "numerical method" is undoubtedly its ability to identify major causes of ill health, and thus make possible their prevention. In the 19th century the quaintly titled Compiler of Abstracts in the General Registrar's Office, William Farr, provided the intellectual impetus for the great sanitary reform movement by tabulating the yawning differential in childhood mortality rates between rich and poor.
Similarly, a century later, Sir Austin Bradford Hill, using statistics alone, proved unequivocally that the rising epidemic of lung cancer was due to smoking. In both instances the statistical methods were of the most elementary kind, well within the competence of a 12-year-old. Medical statistics, observed Sir Austin, were little more than "the application of common sense to figures"
Since his day statistics have become ever more sophisticated and the findings ever more unreliable. Nowadays, sensible people view any sort of statistically derived knowledge with the utmost suspicion. Nor is this just a problem in medicine. Rather, as Professor John Brignell, formerly of London's City University, argues wittily and incisively in his recent book, Sorry, Wrong Number: The Abuse of Measurement, it has become a generalised phenomenon.
Everybody's doing it – not just the charlatans and admen, but "journalists, politicians, lawyers, bureaucrats, single-issue fanatics (especially environmentalists and health activists) and, last but not least, scientists".
"It's a sad fact," says Brignell, "that scientists are amongst the most powerful generators of wrong numbers."
And why, he asks, do all these people "want to tell you lies with numbers?" - to which he gives a characteristically robust answer: "Some want your money or your vote, some want to control your life. Some need simply to justify their own existence (or salary)."
Prof Brignell, as an electronic engineer, has used computers for 40 years but believes their invention has been "a catastrophic event, fuelling the abuse of measurement by providing more numbers than human beings can digest." This vast quantity of computer-generated statistics must therefore be interpreted – which is where the problem lies, because they can so readily be transformed into a decoy, concealing what is really going on behind an apparently objective accumulation of hard data.
Brignell introduces us to the many tricks of the trade. Two examples illustrate the main purpose of abused measurement – proving that some- thing is much more important than it really is.
The first is the "Quantum Leap", where an observation from a limited number of people is generalised by extrapolating to apply to the whole nation. In this way, "three out of 10 mobile phone users, who have difficulty in sleeping", can only too easily become "three million mobile phone users suffer from insomnia”.
The second example is the "Trojan Number", where very large numbers are used to give credibility to a very small effect. Thus, the finding that 2,000 people given the cholesterol. lowering drug cholestyramine reduced their chances of a fatal heart attack by a quarter makes it sound as if hundreds of lives have been saved. But it is not so because only 30 of those in the study had a heart attack.
Hence, it would be equally true to say that 1 ,970 healthy middle-aged men took the drug in sufficient quantities to cause unpleasant side effects for 10 years in order to increase their chances of surviving a heart attack by just 0.5 per cent.
Medical statistics, which in the past have helped to save tens of thousands of lives, have become a vehicle for making people much more neurotic about their health than they need be and inducing them to take many more drugs than they require. Bring back common sense!
John Brignell, Sorry Wrong Number! ISBN 0.9539108.0.6. Price £14. See also his website at www.numberwatch.co.uk
Sorry, wrong number
One of the most often quoted statistics about computer security is that 80%
of attacks come from the inside. This figure has been used so frequently
that it is usually treated as an accepted fact. Well, until the recent
Compsec 2000 conference in the UK that is.
Apparently in one of the opening sessions this conventional wisdom was
challenged, with the new figure being about 50-50 between internal and
external attacks (I missed the speech that started it all as I caught the
cheap train to London!) Anyway, the argument between the 80-20 fans and the
new 50-50 crew seemed to run throughout the conference.
I must admit that I found this all rather worrying.
My biggest concern is with the provenance of the statistics in the first
place. Unjustified statistics are like smiling cats - not to be trusted.
There are a number of big question marks over both the 80-20 and 50-50
First, and perhaps most importantly, they both come from voluntary surveys.
This drops them straight into the "self selecting sample" bin ("90% of
people in our survey said they liked to fill in surveys"). So straight away
we need to clarify the figures with a caveat of "out of people who
responded". Now I suspect that there are a number of organizations out there
who flatly refuse to fill in surveys; admitting to security breaches is not
always good publicity, and although these surveys are anonymous there's
always the chance it might sneak out (the ever present risk of mistyping the
email address or using the wrong button on the fax).
Then there's the fact that these figures can only possibly refer to security
breaches that have actually been detected. Say the 80-20 figure is valid in
other respects, perhaps all it means is that internal attacks are easier to
There's also the question of what exactly is a security breach. This rapidly
becomes an "apples and oranges" comparison. Counting breaches is not
particularly useful unless you apply some grading to them according to their
potential for harm and possibly even some sort of exchange rate for
comparisons ("I'll swap you one dictionary password attack for four ping
Merely counting the numbers is like performing medical triage by counting
the cuts, ignoring their location, depth and the amount of blood. For
example, in one major UK survey there was a huge leap in incidents between
1998 and 1999. The reason was simple, and in the small print: the method for
counting had been changed, but the old figures on the graph had been left
unchanged. However, the graph makes impressive viewing if you're trying to
justify a growing security budget.
I could go on, but I guess you're starting to get the idea. Such throwaway
numbers do more harm than good, and to see respected members of the security
community describe them as "hard facts" is appalling. Incidentally, the same
person then went on to admit that he failed dismally when asked at a
security conference to gauge the relative risk of various common risks such
as cancer, motor accident and the like.
Risk is a complex issue and it is far more than a simple numbers game fed by
the latest paranoia and industry surveys. As I mentioned in my very first
Muse, we have a natural tendency to exaggerate exciting risks and
underestimate the boring ones.
We all have to deal with risk analysis, and measurement of the risks
involved always includes an element of subjectivity. However it is essential
to bear this in mind when drawing conclusions, rather than falling into the
advertiser's trap of selective quoting and pseudoscientific analysis.
[The title of this Muse comes from a superb book by John Brignell that
covers the misuse of statistics and measurement in a much more eloquent and
comprehensive fashion. I recommend it thoroughly, and the accompanying web
site at http://www.numberwatch.co.uk]
Journal of Economic Issues
Sorry, wrong number! is a book written by John Brignell,
Professor of Industrial Instrumentation, University of Southampton about the
abuse of measurement. This is not a book about economics but the student
of economics, and especially econometrics, would do well to read it. It is
also a book that should be read by all post graduate students and students of
statistics. When Brignell set out to write this book it was intended to be
a scientific book for the general lay reader with a few entertaining diversions
from everyday life "but as soon as I started the research I found a can of
worms reeking of the stench of corruption." Brignell challenges the notion
that scientific and academic research is conducted accurately and
dispassionately. Nothing new in that you might say. But what makes
this book fascinating is his use of numerous examples, of seriously flawed
research publications; some cited hundreds of times, to make his case.
Brignell recognises that equipment failures cause some errors but that other
human factors, such as carelessness, incompetence, avarice, greed for power
cause most errors. He demonstrates, for example, that if there are many
researchers studying events with low probability then chance will generate some
statistically significant associations. These findings - the type 1 error
- are more likely to be published than are the many more studies which,
properly, support the null hypothesis. Thus the research literature is
biased; a fact widely researched and verified in medical circles (see special
issue BMJ September 1997).
In chapter 4, Brignell examines dozens of recent
scare stories including the perceived deleterious effects on health of salt, red
meat, fats, AIDS and DDT It will turn many readers into sceptics and
confirm the prejudices of many others. Brignell devotes a later chapter to
the scientific methodology that is also the basis for positive economics;
observation, hypothesis, prediction, test, theory, back to test. This is
often a difficult area for students of economics but Brignell's approach should
make the task of understanding significant deviations and confidence intervals
accessible to all but the lazy. From this Brignell turns his attention to
medicine. He explains the major flaw in the reasoning that which enables
rats to be used to test the effects of chemicals on humans; it is the belief
that the effect of small doses can be extrapolated from the effect of massive
doses. The human stomach, for example, contains dilute hydrochloric acid
but to drink the concentrated form would be fatal. There was a successful
attempt to create a scare about the use of Alar, a growth regulator on apple
crops. Many farmers and processors lost millions of dollars but to
replicate the dosage given to the hapless rats you would need to drink 38,000
pints of apple juice a day. By this methodology almost everything we eat
Not surprisingly, it is where Brignell ventures into the
social sciences where his analysis is weakened. He is very critical, for
example, of macroeconomics and in particular of the Monetary Policy Committee of
the Bank of England seeming to believe that huge distributional and regional
imbalances could be cured with better management. This is a pity because
his own values tend to take over from detached analysis. To his credit
Brignell recognises this point and the book's content and probity is little
compromised. Students of economics should read this book if only to view
number crunching with a little less conviction and a little more scepticism.
Times Higher Educational Supplement
Imagine walking through today's headlines with a mentor by your side. Are mobile phones dangerous? What about overhead power lines? Should you believe the statistics on lung cancer and smoking? Who is telling the truth about drinking and driving? Reading this book is like taking such a stroll. Whatever the current media scare, John Brignell has been there and found the fallacy in the figures.
Brignell is an electrical engineering academic, with experience in forensics, but there are few areas of technical expertise that escape his scrutiny. Aids, BSE, CJD, DNA, EMF, ERM, ESP - no acronym or abbreviation has been left unexplored. His witty investigation of the ills and threats of modern living make this an ideal work for sixth-formers broadening their horizons with general studies, but anyone feeling overwhelmed by the latest media-induced panic should buy it. It is better than Prozac at calming the nerves.
Particularly attractive is the material's up-to-the-minute feel, confirmed by the "webography", which is maintained at Brignell's website (www.numberwatch.co.uk). This means the book needs only a modest conventional bibliography as the cutting-edge data and comment are to be found, regularly refreshed, in cyberspace. In addition, the book inculcates a habit of mind - probing, prying - that will long serve the reader.
Brignell's concern is the abuse of measurement. Every "government expert" or "industry spokesman" backs up his questionable propositions with statistics, charts and figures. Should we take these on trust? "No", answers the book. Brignell recognises that we are often misled with the best of intentions. But he emphatically rejects the proposition that it is justifiable to lie for a good cause - first, because such lies tend to get found out, and when they do the lay person loses faith in experts altogether; second, because if science "accepts but one lie it ceases to be science".
He exposes the logical fallacies that underlie misleading statistics, along with the multitudinous causes of wrong numbers, from indolence to prejudice and fashion. This work is a rollercoaster ride around every politically correct and incorrect media fetish. After reading this, you will think twice before asserting baldly that "smoking is bad for you" or "drink-driving kills".
Brignell encourages scepticism of official pronouncements not merely by puncturing official balloons inflated with questionable data, but also by his own comments on the issues. I would take a different view on many of these, but there is no doubt that Brignell has dug out the facts, digested the figures and set out a coherent case for why he adopts the position he does.
Take anti-smoking statistics. Brignell contends that the oft-used figure of 400,000 premature deaths a year in the United States because of smoking is a "total fabrication". He asserts that the figure depends on 60 per cent of those "premature deaths" occurring in smokers aged over 70, and 17 per cent of them at age 85 and above. Brignell argues that the same data can be used to "prove" that tobacco saves 200,000 lives each year. He also quotes a study from the University of Athens that asserts that Greece has the highest per capita consumption of tobacco in the world but the lowest rate of lung cancer. And Japan, with a high smoking rate, has an average life expectancy of 79.1 years.
Brignell also addresses passive smoking in some detail. He claims that nicotine is found in many common vegetables such as potatoes, tomatoes and aubergines, and he cites research that suggests that eating a normal portion of potatoes equals three-and-a-half hours in a smoky room. He further asserts that US Department of Transportation figures reveal a negligible difference in exposure between smoking and non-smoking flights, and quotes figures from a pro-smoking lobby indicating that flight attendants working exclusively on smoking flights are exposed to, at worst, the equivalent of about six cigarettes a year. Since non-smoking flights began, he says, airlines have reduced filtering to save money. The result, he says, is an increase in lung infections among travellers on non-smoking flights.
Having questioned the statistics against smoking, Bignell sets out the positive benefits of lighting up. He quotes a report apparently showing that "generally, the health of smokers is better than that of many former or non-smokers". It seems that the worst sufferers of hypertension caused by stress are ex-smokers and the "never-smoked". Figures for steady smokers are much lower. The same is apparently true for Parkinson's disease. It is also claimed that non-smokers suffer 50 per cent more Alzheimer's disease than smokers.
Such iconoclastic material made me sceptical of the official figures. It made me even more sceptical of the figures purporting to show that smoking might be good for me. I suspect this is the reaction Brignell wants.
Some of the issues addressed by the book are tackled superficially, which is inevitable given its range. But it is an easy-to-digest overview of issues that bombard us daily. A healthy scepticism is the outcome - what better mindset for the preservation of modern democracy?
Robert Gaitskell QC is a chartered engineer and former vice-president, Institution of Electrical Engineers.
The Sorry story
The appearance of a savage review of the book Sorry, wrong number! on the pages of Amazon UK, brought back memories of its painful birth. Of course, hostile reviews are part of the game, and one against a couple of dozen favourable ones is nothing to get excited about, though all authors are by nature protective of their babies.
As the millennium was coming to an end I had been increasingly irked at the abuse of measurement in politics and the media and the impulse to write a book about it became irresistible. I was also becoming less and less satisfied with the job I used to love, that of a university professor, as it became more and more bureaucratised and subject to a constant downward political pressure on academic standards. So I opted for early retirement, but continued for a while to work part time supervising student projects, which was the on part of the work I still enjoyed. I had previously written a couple of technical books and several chapters in edited books and was continually solicited by publishers to commit to more of the same, but I was not enamoured with the activity and a different sort of writing had appeal. I submitted a synopsis and a couple of sample chapters to publishers and began to get the usual run around. Penguin, for example, said that they already had a similar book on the stocks. I am still waiting for it to come out. It was the comment of a literary agent, however, that set me on the route to self-publishing. His opinion was “This is just a series of complaints about the way we do things today.” True, but not very helpful. It dawned on me that what I was writing was politically incorrect and would be anathema to the very PC publishing industry. I spoke to a colleague in the university printing department and he said that I could use their facilities as a private customer paying the commercial rate, so I arranged a bank overdraft to cover the estimate and pressed on with the writing.
The colleague went to a great deal of effort to get a good bargain for the actual printing costs, but it was tied to a very exact deadline, which would have to be met at all costs. I completed my manuscript and, though I sez it as shouldn’t, it was immaculate. I was particularly proud of the index, which was accurate and comprehensive.
I heard nothing for a couple of weeks and inquired about progress, to be told that the person assigned had done the type-setting and gone on holiday leaving it completed. When I looked at the result I could have wept. He had down no laying-out and the result was exactly the same as my layout in Word, except that it had been translated into the printers’ language, Quark. It had been reduced to A5, with the result that you needed a magnifying glass to read it.
By this time the deadline was getting ominously close, so my colleague mounted a rescue operation in a different sub-department. There followed a few desperate days punctuated by crises and panics. One thing that happened was that all the fractions had disappeared in the translation and they had to be re-inserted. One was done incorrectly, which ultimately resulted in my telling the world that the probability of getting either a head or a tail on tossing a coin was one half. In the laying out process many other typos had been introduced, about a hundred of them. I was left with two days to do the proofing, when it transpired that my immaculate index could not be translated and I would have to renumber it by hand. With the help of another colleague I spent a day doing the proof reading. Inevitably a number of errors remained undetected or newly emerged and exist in the book today, but apart from that fraction they were relatively minor and obvious. I worked through the night on the index, sustained by several pots of black coffee. At four in the morning, trembling with excess caffeine intake, I realised that I was not going to make it, which is why the first half of the book is well indexed and the second half only done superficially.
When the book emerged from the printers it was rather disappointing. In order to keep to the planned number of pages, and hence the cost, it had been printed in quarto size, and the chosen font was rather dense, which made it less than ideally readable. So it was a relief to find that readers and reviewers were rather tolerant of the defects. Over the succeeding years I have received a substantial and touchingly appreciative correspondence. I had almost forgotten the pain of it all, when that hostile review, ironically entitled Desperately in need of an editor, brought it all back. Perhaps writing about it will help to lay the ghost.