Interplay between theory and experiment (March 2006)

Lubos Motl assistant professor, Cambridge, USA

file: interplay.html, 12Mar2006

full original article see here (highlighting by meteoLCD)
....

Background vs. signal

It is extremely important to know what is the “natural background” if we try to figure out whether there is a new “effect”. Some people like Rasmus Benestad just don’t want to study the natural background at all - they immediately want to get effects (and attention of the press in which they're pretty successful because many journalists are pretty dumb) - which is why I think that they are crackpots. As mentioned previously, one of the defining features of crackpots is that they want to make big discoveries before they learn what is the science describing the “simpler” phenomena before their discovery.

Let me say why their research is defective in one more way.

Whenever we try to design scientific theories that describe something, we must know which quantities in reality will be described by our theories and we must be able to isolate them.
By isolating them, I mean both theoretical as well as experimental isolation. In theories we must know - or at least feel - that the effects we have neglected do not change our predictions too much. In experiments we must know - or at least have rational reasons to believe - that the effects we observe are not caused by something else, something “more ordinary”. When we try to observe telepathy, for example, we must know that the people are not communicating by some more "natural" methods.

The climate modellers almost never try to follow these lines. They have a completely vague, sleeky set of ideas that predict anything and everything - warming, cooling, bigger variations, smaller variations, more hurricanes, less winds, increased circulation, diminished circulation, more ice in Antarctica, less ice in Antarctica, and so forth - and then they’re arguing that the data agrees with these predictions. Of course they emphasize the points whenever they agree and de-emphasize them whenever they disagree. This is not science.

Of course there is no direct way how one can ever construct a scientific framework out of this mess. To do science, one must focus on a limited class of questions that are sufficiently well-defined and that have a chance to be “cracked” by a theory. I am sure that there are many nice laws about the climate that we don't know yet, and I am equally sure that the work of most of the "mainstream" climate scientists today is not helpful in revealing these laws.

When we try to argue that the humans are suddenly dictating the climate trends - after 5 billion years when they were dictated by other, more natural things - it is a rather extraordinary conjecture that deserves extraordinary evidence. For getting any evidence, it is absolutely necessary to understand how the climate was behaving for 5 billion years before the hypothetical “revolution” occured around 1917. We must know what were the fluctuations and how they depended on the time scale. We can only learn such things reliably by observing the real world. Only once we know the background, we can study the additional effects.

Studying additional trends above a background that we don’t need to understand is equivalent to the Biblical literalism.

Summary

Some readers may feel that the two parts of this text contradict each other because I defend theory in the first part and the observations in the second part. However, I am convinced that every sane scientist (and informed layman) knows that both theory as well as experiments are important. My goal was certainly different from changing the balance to one side. My goal was to emphasize that science should be looking for robust conclusions and theories and it should be attempting to find the situations in which the phenomena exhibit themselves in the sharpest possible way. And a necessary principle to achieve this goal is to try to follow these principles:

• try to isolate the "signal" that you are interested in as well as you can
• when your signal exists above a certain "background", you must definitely try to understand the background first
• if you can find an idealized situation in which one signal is isolated from some other effects that you're not interested in, study this situation
• if you cannot find an idealized situation and if everything looks like quantitatively undescribable chaos to you that you want to match by a computer-generated chaos, then it means that you still misunderstand what's going on; avoid the quagmire and return to the point #1
• if you have a theory, be sure to deduce and decide what kind of quantities the theory should be able to predict
• if your theory only agrees with some observations, never fool yourself and never try to de-emphasize the observations you know to disagree with your theory
• if your theory or model only agrees "roughly" with 24 features of the data but there are 25 parameters or assumptions that led to your model, be sure that you can't claim that you established your model or its assumptions
• isolate the assumptions of your theories (and open questions) from each other and try to test them separately whenever you can
  
• if you try to explain experimental data, always ask whether there exists a simpler and more natural theory than yours that would be able to do approximately the same job
• if there is a more natural theory with less parameters, go for it
• never believe that your theory is superior just because it is using the buzzwords - or approximate concepts and laws - that are more frequent in physics; this is not how the better theories are identified.

______________________________________

See also here on temperature and autocorrelation

Luboš Motl's reference frame E=mc2 a test ... interplay between theory and experiment