Why do records always increase? There is one obvious reason. By definition they cannot go down, so if they change it must be in an upward direction. One further possible reason for records to increase is that the quantity being observed is not stationary (e.g. the average value is actually increasing with time), but even if the process is stationary there is still a reason for records to increase. As we keep adding data, the size of the statistical sample is increasing.
Let us invent a stationary process and create a little allegory. In the eighteenth century an eccentric amateur astronomer looks through his low-power telescope each night, pointing roughly in the same direction, and counts the number of stars in its field of vision. He is keen to establish a record so that he can apply for entry in the contemporary equivalent of the Guinness Book of Records. The conditions are such that the average number of stars he sees is 100 and the average scatter (standard deviation) is plus or minus ten stars.
We can model this process by generating numbers from a normal distribution (for the cognoscenti the function rnorm(N,100,10) in Mathcad).
Our astronomer takes one reading each night and only writes down the current record, the highest value so far, so after a hundred nights he gets something like:
What he finds is that his record goes up quite quickly at first and then a lot more slowly. It is getting harder and harder to break the record, so he passes the task on to his son and it carries on down the generations. Eventually, after nearly three hundred years, the family have accumulated 100,000 attempts to break the record. Such a large number of points cannot be plotted against a linear scale of days, so they have to plot it logarithmically:
By now we have reached the tenth generation. It is the late twentieth century and the age of rationality has come to an end. The new scion of the family, realising that it has obtained no benefit from all these years of effort, publishes a paper warning of the increase in star density that is occurring and calculating all sorts of disasters that arise from the increased gravitation. He is appointed to a professorial chair in a new university and founds a new Department of Celestial Change. The cause is taken up by the Deputy Prime Minister and large sums of money are diverted from other research areas to a new programme for the study of celestial change. Disaster stories are regularly fed to the media, who sell more papers and advertising time.
A few old-fashioned scientists point out that the results are completely explained by the well-known statistics of extremes, which predict that the largest value will increase as the logarithm of the number of observations. There is no money in that, so they are ignored and the bandwagon rolls on.
The problem posed by the logarithmic relationship is that it is one of diminishing returns. Here is the theoretical record (the characteristic largest value) corresponding to the first figure.
Returning to our allegorical astronomer, he now finds that it is increasingly difficult to establish new records over a reasonable time scale. This poses a number of problems. He now has an established department with mouths to feed. He therefore needs to keep the pot boiling with regular press releases in order to justify more research grants. There are now two basic methods open to him. The first is the Olympics method, which is based on improving the accuracy of recording so that smaller and smaller increments of record are established, in his case by improving the optics on his telescope. The second is the Climatologist's method, which involves extending the range of techniques of measurement, so that he can cherry pick the ones that best suit his purpose. He therefore expands to radio, infra-red, x-ray and ultra-violet telescopes, including satellite mounted versions. As a result he continues to hit the headlines and corner a substantial share of the limited funds available for scientific research. Thus, as with all good fairy stories, they lived happy ever after.