It’s in the news (e.g. WUWT), cosmetic operations on experimental data to improve their looks. That is, reducing noise.
Noise is defined as any “unwanted” signal, just as a weed is any unwanted plant. It includes such things as mains-induced signals (50 or 60 Hz), but usually refers to “random” noise, the adjective often being dropped. Smoothing is generally adopted to eliminate high-frequency noise – those nasty wiggly bits that uglify our desired smooth curves. We must always be aware that it is an irreversible process (the discarding of information), which might be crucially destructive. Often we are only allowed to see a titivated version, which could be misleading.
Here are some simple rules for those who are not sure of their understanding of signal theory (both continuous and discrete):
1. Use only simple procedures whose effects are well understood.
2. Do not try to invent new smoothing procedures (unless it is your intention to mislead). All the trustworthy ones have been tried (to say nothing of even more of the other sort).
3. Never trust anyone who purports to have invented a new smoothing process.
4. Never trust anyone who purports to predict the future by means of a smoothing process (or, indeed. any other sort).
In our treatment in these pages we have confided discussion to the very simplest methods, because this is a field full of hazards, even for specialists. Just simply to discuss the possibility of smoothing the end values of truncated time series (or even more so, extrapolating from them) requires an in-depth knowledge of such matters as the uncertainty principle, the related window problem, discrete function transforms etc. Even these concepts falter in the presence of non-linearity and non-stationarity, which preclude the powerful and wide-ranging linear algebra. In the present state of mathematical knowledge, if at all, it is not amenable to solution. People try to develop ad hoc methods. If they do so, they should at least test them to destruction with various demanding artificial input functions, but they do not; which exposes their motivation, to produce a result in accordance with their beliefs rather than promoting understanding.
There is a tendency to forget that signals such as global temperatures are real time data. The only difference is that when the sampling interval is one year, rather than one nanosecond, there is more time for human mischief in between.
It takes considerable effort to demonstrate the fatuity of some of these home-made processes, whether they are produced by relative unknowns, such as Roger Coppock, or more notorious deceivers, such as Chris Huhne or Michael Mann. It is dispiriting work (of which your bending author, for one, has had quite enough) but fortunately there are always an energetic few willing to take it on; while the internet, free of the corruptible shackles of “peer review”, ensures that they get an airing.
Only a mathematician of great powers could make progress in such a field, if any is possible. It requires hubris of Mannian proportions for an amateur mathematician to propose a method of smoothing and extrapolation of noisy evolutive processes. People have tried unsuccessfully to predict the future since the dawn of language, so they are not going to stop now.
Incidentally, in defence against possible
accusations of sticking to simple examples just because of lack of knowledge of
others I have scanned in a reprint of an article published in Electronics
Letters in June 1968, which I believe gave the first demonstration of
real-time online digital filtering in the UK, using an adapted Pegasus machine.
The filter is fourth order Chebyshev class low-pass. See here:
Filter page 1
Filter page 2
Woke up this morning,
Global warming on the floor.
Woke up this morning,
Global warming on the floor.
Seems like I ain’t
Gonna see the Sun no more.
If any reader in Northern Europe ever wondered why the term “stationary high” occurs so often in Number Watch (18 times), they do not now. This is the weather pattern that, in the wrong place, causes abnormally extreme temperatures in both winter and summer, bringing misery and death to the poor, the vulnerable and the old. The first mention occurred in a blithe spring just a decade ago, when global warming propaganda was at its fiercest, vying temporarily with WMD as the convenient scare. Another was in September 2009 when we indulged in a fictional (not a lot) account of future events, which have been adumbrated by this bitter, sunless March, with Britain just days away from an energy supply disaster engineered by its clueless political class.
In response to a comment on filtering:
The Gaussian function has a special place in signal theory as it is the one function whose shape is unchanged by Fourier transformation. It is free of oscillatory behaviour (ringing), which helps to make it very useful as a filter in such applications as image processing.
In application to a time-series, the Gaussian filter is non-causal: it uses future values and hence is not realisable by physical means. That is a heavy hint for those operating on real time events.
Both the frequency and impulse responses of it are unbounded. Therefore it has to be truncated to be realised; which means, in particular, that it runs into the buffers at the beginning and end of a real time sequence: hence the need for fancy methods to smooth the sequence near the end-point (i.e. the now in real time, which is usually the most portentous moment) that generally involve “padding” with various sorts of artificially generated numbers. Such problems are related to the uncertainty principle and, as students of quantum physics well know, the product of the width parameters in the two domains (the standard deviations) is a constant (1/2π).
As with the provision of statistical packages to the likes of sociologists and epidemiologists, making such techniques available in easy-to-use form is like giving sharp tools to children as playthings.
Owing to convolution and the central limit theorem, the Gaussian response can be approximated to any desired order by repeated applications of the rectangular response (running mean)
We first commented on the MMR scare eleven years ago (and subsequently 15 more times). Back then we did not know the sordid details that were later to emerge, but the characteristics of junk science were already there. Now, a serious outbreak of measles in Swansea underlines the gravity of the consequences. Britain was once almost free of the disease, but now is second only to Rumania among European case statistics.
The wages of junk science is death.
At one with Watts, my heart dropped on reading Delingpole’s latest missile. Its heavy jocularity misfired and as a result overstepped the mark. There is every reason to be angry, but it is unproductive. The sedate, though highly competitive, world of scientific disputation was shaken when the thuggish new boys appeared on the block. They launched savage ad hominem attacks on opponents, excluded them by rigging the peer review process and access to research funds. So much was well known, but successfully suppressed in the public domain until the dramatic revelations of “Climategate” and the openly corrupt official whitewashes that followed it. All this, in the light of the poisoned political background, was comprehensively covered by Delingpole in his excellent recent book. This latest tirade is, however, at minimum a tactical error. Most people do not logically analyse what is written, they absorb the tenor through the use of language: “by their metaphors shall ye know them”. It enabled, for example, the egocentric Mann to claim that Delingpole had called for his murder, which he specifically did not do. Mann knows that his blinkered followers will not bother to check the context.
Personally, I bridled at the suggestion that possession of an arts degree endows a superior gift of reasoning. There are many professors in the arts, some of whom I have counted as friends, who are serial dealers in sophistry. His frustration is quite understandable after a trawl through the largely unreadable guff that constitutes a typical Telegraph comments section. A rambling discussion with thousands of largely irrelevant, frequently abusive (and often barely literate) entries serves no useful purpose at all. While there is evidence of a strong correlation between adherence to zealot causes and infelicity of language, that is no justification for people of reason to respond in kind. The rugby dressing room as a source of an apt metaphor, for example, is nothing short of bizarre.
We habitual ironists frequently run into trouble through being taken literally. It is a great irony that in the USA, the home of some of the finest ironists in English literature, irony is routinely misunderstood. I for one am not going to start adding the “\sarc” tag to my writings. It is like the pub bore, insisting on explaining the joke he has just told you. It does, however, lay you open to attack from ruthless opponents, who can make hay from a single sentence quoted out of context.
“Know thy enemy” advises Sun Tzu in The art of war. This requires careful observation of patterns of behaviour.
A long time ago I saw a film called A Face in the Crowd. Apart from the fact that I fell in love with Patricia Neal, its story of how fame and its pursuit can turn a decent person into a monster made a lasting impression on me. It is something I have observed all my life. Lapses in standards of conduct are rarely isolated: further examination reveals that they are parts of patterns of behaviour that develop in some individuals. So it is with the small cast list of the page you are reading (Mann, Huhne, Wakefield). A dodgy graph, a traffic law violation or an unethical publication by press conference are all, perhaps, to some extent forgivable; but, when they turn out to be the tip of a behavioural iceberg, that is a very different matter indeed.
Link to this piece
The object of power is power.
Orwell: Nineteen eighty four.
This video is shocking, but few people will see it so. As in the UK, America’s anti-salt fanatics are deviously practising political control for its own sake. The manoeuvres, which they have learned from the tobacco freaks, are blatant but they don’t care. The scientific evidence is against them but they don’t care. As we have said before, they will not be satisfied until they have you shivering in a cave, sipping thin gruel. Their goal is to have dominion over their fellow man; yet they are all of a once honourable profession.
Link to this piece
Everything is poison and nothing is poison. It all depends on the dose.
Extraordinary! It gets a whole front page of the broadsheet Daily Telegraph plus the inevitable accompanying inside page of dire dietary advice. Journalist Victoria Lambert has, rather late in the day, caught up with the new kid on the zealotry block, Robert (sugar is poison) Lustig of (where else could it be but) San Franciso, California.
It is a tawdry piece, full of the usual clichés, such as “some scientists believe”. If they believe they are not scientists, while if they are scientists they do not believe. All you need to know is that a healthy diet is varied and balanced. Excess or dearth of any component is likely to be deleterious.
We noted the apotheosis of Lustig at the beginning of this year, during the celebrations of the seasonal diet-fest, but just a year ago the sugar is poison movement was already a phalanx in the great march of the zealots and could be traced back to the likes of that great numerical necromancer, the mysterious Dr Hu.
The food fascists are in full cry and the political class are always on the look-out for means of misdirection from their failures. Anyone for thin gruel?
Footnote: just for information, your bending author does not add sugar to beverages or any other food items within his control.
Link to this piece
which leads us to:
Back in the old routine
Back where the corn is green
Back in the old routine
With that mind-numbing inevitability the epidemiologists queue up to manufacture evidence to support the latest trend in scaremongering. Sugars are the now in vogue for horror stories. Out come the surveys, observational studies based on anecdotal evidence, dismissing the possibility of confounding factors and yielding derisory relative risks.
How readily the popular media embrace the new zealots on the block! In retrospect this step in the chain was inevitable. The carbophobic faith now being well established, it was only natural that the next zealot attack would be against the next stage of food production. Plant photosynthesis not only made the planet habitable for us by oxygenating it, the process also locked in solar energy to create, from carbon dioxide and water, the basic building blocks of all life on earth, namely sugars.
Encouraging, though, to observe a healthy scepticism among commenters on the Daily Mail splash on this scare.
Everyone should read, learn and inwardly digest this timely slogan from SPONG:
There is only one safe food; thin gruel. Get used to it.
Link to this piece
Our revels now are ended.
These our actors,
As I foretold you, were all spirits, and
Are melted into air, into thin air:
And like the baseless fabric of this vision,
The cloud-capp'd tow'rs, the gorgeous palaces,
The solemn temples, the great globe itself,
Yea, all which it inherit, shall dissolve,
And, like this insubstantial pageant faded,
Leave not a rack behind. We are such stuff
As dreams are made on; and our little life
Is rounded with a sleep.
The integrating web sites (such as gwpf) are reporting a widening debate on the “pause” in global warming. What is rarely considered is whether the putative phenomenon actually happened in the first place. When the scaremongers abruptly reversed their position in the late seventies, jumping from the coming ice age to its antithesis, they caught the opposition (traditional, i.e. sceptical, scientists) on the hop. As the new catastrophism was convenient to a large range of political, media and business interests, they were able to build up such a head of steam that over a few years they established a virtual monopoly of influential opinion. In the UK, for example, only five MPs voted against the monumentally destructive Climate Act of 2009, introduced by Ed Milliband. Their case rested on a number of computer models, which may be discarded as hopelessly unreliable, and the claim of a measured increase of global temperature of 0.7ºC over a century (or a wide choice of smaller intervals).
The edifice, however, being built on sand, is beginning to crumble. The hegemony, so swiftly established, is slowly tottering. In recent times, for example, we have witnessed the collapse of the voluntary and virtually total media self-censorship. Comedians are even making jokes about it (not on the BBC, of course).
Now would be a good time to initiate the debate we never had on whether global warming ever really happened. Here are a few random jottings, debating points that might help to kick off the discussion:
|0.7ºC is a very small temperature change, one that you would not notice when passing from one room to another: yet it is claimed to be extraordinarily responsible for an incredible list of catastrophes.|
|At first the believers had a clear run, boasting of a high quality instrumentation network. Gradually, doubters began to organise themselves and examine just how high that quality was.|
|It is an interesting coincidence that, as the amount of surveillance of data gathering and manipulation increased, the flow of evidence in favour of warming faded.|
|There is evidence of malpractice in the gathering and manipulation of global temperature data (e.g John Daly, man-made, kiwigate etc.). The intriguing question was – if the believers had faith in their theory, why did they feel the need to resort to such practices?|
|The believers had set about trying to find or create evidence to support their hypothesis; quite the reverse of the scientific method, which is based on falsification.|
|They made forecasts on the basis of the theory, which is proper science, but ignored their failure, which is not.|
|The nadir of the new “climate science” came when its most prominent practitioners declared that they would rather destroy their data than release them to other scientists. Why?|
|Would it be unreasonable to suggest that the probable error is greater than 0.7ºC, in the absence of access to the data?.|
|Sensor stations are not only badly sited in terms of their immediate locality; they are badly distributed (all clumps and gaping spaces) while their numbers have declined significantly, especially where they were already sparse.|
|The methods of the believers have been alien to the traditions of science. They include vicious ad hominem attacks, promotion of the idea and existence of a “consensus” and calculated corruption of the process of disinterested quality control in scientific publication.|
|Now that the political class have established control over education and science, there is little hope that integrity will ever be restored.|
Which leads us to:
A trivial number that irrationally became one of the most consequential in human history. It brought down a devastating economic plague upon the West, causing it to surrender world leadership.
This is the number that triggered a concerted move by the leaders of a whole continent (including the UK) to adopt a programme of economic suicide. They drove vital indigenous industries into the welcoming arms of their international competitors and forced some of their own most vulnerable citizens into choosing between food and warmth, to mention just two of their many related misdeeds, while spending wildly on extravagant nostrums.
Perhaps this number is best viewed as zero plus experimental error.
Link to this piece
Index, search box, begging bowl and FORUM