Scientific method

The scientific method is how scientists investigate phenomena and acquire new knowledge. It is based on observable, empirical, measurable evidence. Scientists propose hypotheses to explain phenomena, and test those hypotheses by examining the evidence from experimental studies. Scientists also formulate theories that encompass whole domains of inquiry and bind hypotheses together into logically coherent wholes.

"Science is a way of thinking much more than it is a body of knowledge." (Carl Sagan).

Elements of scientific method

According to Charles Darwin ,

". . .science consists in grouping facts so that general laws or conclusions may be drawn from them."

This simple account begs many questions. What do we mean by ‘facts’? How much can we trust our senses to enable us to believe that what we see is true? How exactly do scientists ‘group’ facts? How do they select which facts to pay attention to, and is it even possible to do this in an objective way? And having done this, how exactly do they go about drawing any broader conclusions from the facts that they assemble? How can we know more than we observe directly? The English philosopher, Francis Bacon is sometimes credited as the leader of a scientific revolution with his 'observation and experimentation' theory, the template of the scientific method as conducted ever since. He recognised clearly that interpreting nature needs more than observation and reason:

...But the universe to the eye of the human understanding is framed like a labyrinth, presenting as it does on every side so many ambiguities of way, such deceitful resemblances of objects and signs, natures so irregular in their lines and so knotted and entangled. And then the way is still to be made by the uncertain light of the sense, sometimes shining out, sometimes clouded over, through the woods of experience and particulars; while those who offer themselves for guides are (as was said) themselves also puzzled, and increase the number of errors and wanderers. In circumstances so difficult neither the natural force of man's judgement nor even any accidental felicity offers any chance of success. No excellence of wit, no repetition of chance experiments, can overcome such difficulties as these. Our steps must be guided by a clue... (Francis Bacon)

We live in a world that is not directly understandable. We sometimes disagree about the ‘facts’ we see around us, and some things in the world are at odds with our understanding. What we call the “scientific method” is an account of how scientists attempt to reach agreement and understanding, how they gather and report observations in ways that will be understood by others and accepted as valid evidence, how they construct explanations that will be consistent with the world, that will withstand critical logical and experimental scrutiny, and that will provide the foundations for further increases in understanding.

The success of science, as measured by the technological achievements that have progressively changed our world, have led many to the conclusion that this must reflect the success of some methodological rules that scientists follow in their research. However, not all philosophers accept this conclusion; notably, the philosopher Paul Feyerabend denied that science is genuinely a methodological process. In his book Against Method he argued that scientific progress is not the result of applying any particular rules. Instead, he concluded almost that "anything goes", in that for any particular ‘rule’ there are abundant examples of successful science that have proceeded in a way that seems to contradict it. To Feyeraband, there is no fundamental difference between science and other areas of human activity characterised by reasoned thought. A similar sentiment was expressed by T.H. Huxley in 1863: "The method of scientific investigation is nothing but the expression of the necessary mode or working of the human mind. It is simply the mode at which all phenomena are reasoned about, rendered precise and exact."

Nevertheless, in the Daubert v. Merrell Dow Pharmaceuticals Inc. [509 U.S. 579 (1993)] decision, the U.S. Supreme Court recognised a special status to ‘The Scientific Method ‘, in ruling that "… to qualify as ’scientific knowledge’ an inference or assertion must be derived by the scientific method. Proposed testimony must be supported by appropriate validation - i.e., ‘good grounds,’ based on what is known." The Court also stated that "A new theory or explanation must generally survive a period of testing, review, and refinement before achieving scientific acceptance. This process does not merely reflect the scientific method, it is the scientific method."

Hypotheses and theories

Hypotheses and theories play a central role in science; the idea that any observer can study the world except through the spectacles of his or her preconceptions and expectations is not sustainable. As these preconceptions change with progressively changing understanding of the world, the nature of science itself changes, and what was once considered conventionally scientific no longer seems so in retrospect.

A hypothesis is a proposed explanation of a phenomenon. It is an “inspired guess”, a “bold speculation” , embedded in current understanding yet going beyond that to assert something that we do not know for sure as a way of explaining something not otherwise accounted for. Scientists use many different means to generate hypotheses including their own creative imagination, ideas from other fields, induction, Bayesian inference. Charles Sanders Peirce described the incipient stages of inquiry, instigated by the "irritation of doubt" to venture a plausible guess, as abductive reasoning. The history of science is filled with stories of scientists claiming a "flash of inspiration", or a hunch, which then motivated them to look for evidence to support or refute their idea. Michael Polanyi made such creativity the centrepiece of his discussion of methodology.

If a hypothesis has any scientific content, then it will lead to predictions, and by doing experiments to see whether these predictions are fulfilled of not, the hypothesis can be tested. If the predictions prove wrong, the hypothesis is discarded, otherwise the hypothesis is put to further test, and if it resists determined attempts to disprove it, then it might come to be accepted, at least for the moment, as plausibly true.

The philosopher Karl Popper , in The Logic of Scientific Discovery, a book that Sir Peter Medawar called one of the most important documents of the 20th century, argued forcefully that argued that this 'hypothetico-deductive' method was the only sound way by which science makes progress. He argued that the alternative process of induction - of gathering facts, considering them, and inferring general laws, is logically unsound, for any number of mutually inconsistent hypotheses might be consistent with any given set of facts. He argued generally that "historicism", the process of constructing a plausible explanation after the event, is intellectually disreputable, a vehicle for insidiously introducing ideological values while maintaining the mere superficial appearance of objectivity.

He concluded that a proposition or theory cannot be called scientific if it does not admit the possibility of being shown false. It must, at least in principle, be possible to make an observation that would show the proposition to be false, otherwise the proposition is vacuous, with, as Popper put it, no connection with the real world. For Popper, explanations without any predictive content were unscientific, and he argued that the explanations of Freudian psychoanalysis, those of Marxism, and those of astrology, were all examples of ‘empty’ unscientific theories in this sense.

For Popper, a theory was the context within which hypotheses are developed, and which determined which things were important to investigate and which were not. The theory encompasses the preconceptions by which the world is viewed, and defines the ways we study it and understand it. A theory thus has a profound importance, without a theory no science is possible. He recognised that you do not discard a theory lightly, and that a theory might be retained long after it had been shown to be inconsistent with many known facts (anomalies). However, the recognition of anomalies drives scientists to elaborate or adjust the theory, and if the anomalies continue to accumulate, will drive them to develop alternative theories. He also explained that theories always contain many elements that are not falsifiable, but he argued that these should be kept to a minimum, and that the content of a theory should be judged by the extent to which it inspired testable hypotheses. This was not the only criterion in choosing a theory; scientists also seek theories that are "elegant" or "beautiful". These notions are subjective and hard to define, but they express scientists' expectations that a theory should yield clear, simple explanations of complex phenomena, that are intellectually satisfying in the sense of appearing to be logically coherent, rich in content, and involving no miracles or other supernatural devices.

Popper thus argued that progress in science depends upon attempted falsification of hypotheses, and that most progress came by success in falsifying them; disproof is logically sound, support by induction is logically unsound. "Verifiability" in Popper's view was not the object or intent of science, just a weak by-product of a failed attempt at falsification.

Popper's views were in many respects in marked contrast to those of his contemporary, the historian of science Thomas Kuhn. Kuhn's own book "The Structure of Scientific Revolutions" was no less influential than Popper's, but its message was markedly different. Kuhn analysed times in the history of science when one dominant theory was replaced by another different world view - such as the replacement of Ptolemy's heliocentric model of the Universe with the Copericus Copernican geocentric model, and the replacement of Newtonian laws of motion with Einstein's theory of Relativity. In many respects Popper was asserting what he held to be the rules for "good science", Kuhn on the other hand considered himself to be reporting what scientists actually did, although he believed that as what they did was undeniably successful, probably there was merit in what they actually did. Kuhn concluded that falsifiability in fact had played almost no role in these "scientific revolotions" where one paradigm was relaced by another. He argued that scientists working in a field form a closed group, mutually supporting each other, resisting attempts from outside to offer alternative interpretations, and tenaciously defending their world view by a process of continually elaborating their shared theory, by "puzzle solving" in a way that constantly extended the scope and explanatory power of the theory. He argued that when one theory is eventually replaced by another, this does not happen because scientists are "converted" to a different world view; rather a new theory starts as an unfashionable alternative that gains more and more adherents as the advantages of the new theory over the old become apparent to new scientists entering the field. Seldo if ever is it the case that experiments are decisive in refuting one theory and imposing a new one; Kuhn argued that theories are "incomensurable", one theory cannot be tested by the assumptions of a different theory, and for the adherents of a theory, "once it has been adopted by a profession ... no theory is recognized to be testable by any quantitative tests that it has not already passed".

Experiments and observations

Werner Heisenberg in a quote that he attributed to Albert Einstein , stated [Heisenberg 1971]:

The phenomenon under observation produces certain events in our measuring apparatus. As a result, further processes take place in the apparatus, which eventually and by complicated paths produce sense impressions and help us to fix the effects in our consciousness. Along this whole path—from the phenomenon to its fixation in our consciousness—we must be able to tell how nature functions, must know the natural laws at least in practical terms, before we can claim to have observed anything at all. Only theory, that is, knowledge of natural laws, enables us to deduce the underlying phenomena from our sense impressions.

For much of the 20th century, the dominant approach to science has been reductionism – the attempt to explain all phenomena in terms of basic laws of physics and chemistry. This driving principle of scientific methodology has ancient roots - Francis Bacon (1561-1626) quotes Aristotle favourably as declaring "That the nature of everything is best seen in his smallest portions." In many fields, however, reductionist explanations of complex phenomena are impractical, and all explanations involve 'high level' concepts. Nevertheless, the reductionist belief has been that these high level concepts are all ultimately reducible to physics and chemistry, and that the role of science is to progressively explain high level concepts by concepts closer and closer to the basic physics and chemistry. For example, to explain the behaviour of individuals we might refer to motivational states such as hunger or stress or anxiety. We believe that these reflect features of the activity of the brain that are still poorly understood, but can investigate the brain areas that house these motivational drives, calling them, for example, “hunger centres”, These centres each involve many neural networks – interconnected nerve cells, and the functions of each network we can again probe in more detail. These networks in turn are composed of specialised neurons, whose behaviour can be analysed individually. These specialised nerve cells have distinctive properties that are the product of a genetic program that is activated in development – and so reducible to molecular biology. However, while behaviour is in this sense reducible to basic elements, explaining behaviour of an individual in terms of these basic elements has little predictive value, because the uncertainties in our understanding are too great, so explanations of behaviour still largely depend upon the high level constructs. Historically, the converse philosophical position to reductionism has taken many names, but the clearest debate was between “vitalism” and reductionism. Vitalism held essentially that some features of living organisms, including life itself, were not amenable to a physico-chemical explanation, and so asserted that high level constructs were essential to understanding and explanation.

The reductionist approach has asigned a particular importance to precise measurement of observable quantities. Scientific measurements are usually tabulated, graphed, or mapped, and statistical analyses of them; often these representations of the data using tools and conventions that are at a given time, accepted and understood by scientists working within a given field. The measurements often require specialized instruments such as thermometers, microscopes, or voltmeters, whose properties and limitations are familiar to others in the field, and the progress of a scientific field is usually intimately tied to their development. Measurements also demand the use of operational definitions. A scientific quantity is defined precisely by how it is measured, in terms that enable other scientists to reproduce the measurements. In many cases, this ultimately involves internationally agreed ‘standards’. For example, electrical current, measured in amperes, can be defined in terms of the mass of silver deposited in a certain time on an electrode in an electrochemical device that is described in some detail. The scientific definition of a term sometimes differs substantially from their natural language usage. For example, mass and weight overlap in meaning in common use, but have different meanings in physics. Scientific quantities are often characterized by their units of measure which can later be described in terms of conventional physical units when communicating the work. Measurements are not reports of absolute truth, all measurements are accompanied by the possibility of error in measurement, so they are usually accompanied by estimates of their uncertainty, This is often estimated by making repeated measurements, and seeing by how much these differ. Counts of things, such as the number of people in a nation at a particular time, may also have an uncertainty due to limitations of the method used. Counts may only represent a sample of desired quantities, with an uncertainty that depends upon the sampling method used and the number of samples taken.

The scientific method in practice

The UK Research Charity Cancer UK gave an outline of the scientific method, as practised by their scientists [4]. The quotes that follow are all from this outline

[Scientists] start by making an educated guess about what they think the answer might be, based on all the available evidence they have. This is known as forming an hypothesis. They then try to prove if their hypothesis is right or wrong. Researchers carry out carefully designed studies, often known as experiments, to test their hypothesis. They collect and record detailed information from the studies. They look carefully at the results to work out if their hypothesis is right or wrong…

Once predictions are made, they can be tested by experiments. If the outcome contradicts the predictions, then explanations may be sought before the hypothesis is discarded as false. Sometimes there is a flaw in the experimental design, only recognised in retrospect. If the results confirm the predictions, then the hypotheses might still be wrong and if important, will be subjected to further testing. Scientists keep detailed records, both to provide evidence of the effectiveness and integrity of the procedure and to ensure that the experiments can be reproduced reliably. This tradition can be seen in the work of Hipparchus (190 BCE - 120 BCE), when determining a value for the precession of the Earth over 2100 years ago, and 1000 years before Al-Batani.

Peer review

…Once they have completed their study, the researchers write up their results and conclusions. And they try to publish them as a paper in a scientific journal. Before the work can be published, it must be checked by a number of independent researchers who are experts in a relevant field. This process is called ‘peer review’, and involves scrutinising the research to see if there are any flaws that invalidate the results…

Manuscripts submitted for publication in scientific journals are normally sent by the editor to (usually one to three) fellow (usually anonymous) scientists who are familiar with the field for evaluation. The referees may or may not recommend publication, publication with suggested modifications, or, sometimes, publication in another journal. This helps to keep the scientific literature free of unscientific work, reduces obvious errors, and generally improves the quality of the scientific literature. The peer review process has been criticised, but is very widely adopted by the scientific community. Nevertheless, there are inevtable weaknesses; first it is very much easier to publish data that are consistent with generally accepted theory than to publish data that contradict accepted theory: the 'bar' for acceptance of work is higher the more remarkable the claim. This helps to ensure the stability of the body of accepted theory, but also means that the appearance of the extent to which a conventionally accepted theory is supported by evidence might be misleading - boosted by poor quality supportive work and protected against higher quality opposing work.

On the other hand, originality, importance and interest are particularly important in 'high impact' general journals of science -see for example the author guidelines for Nature, thus if controversial work appears to be very convincing then it stands a good chance of being published in such journals Criticisms (see Critical theory) of journal publication priorities are that they are so vaguely defined, highly subjective and open to ideological, or even political, manipulation, that they can sem to impede rather than promote scientific discovery. Apparent censorship by refusing to publish ideas unpopular with mainstream scientists has soured the popular perception of scientists, by apparently contradicting their claim to be objective seekers of truth.

The scientific literature

…If the study is found to be good enough, the findings are published and acknowledged by the wider scientific community…

However Thomas Kuhn argued that scientists are

Sir Peter Medawar, Nobel laureate in Physiology and Medicine in his article “Is the scientific paper a fraud?” answered yes, "The scientific paper in its orthodox form does embody a totally mistaken conception, even a travesty, of the nature of scientific thought." In scientific papers, the results of an experiment are interpreted only at the end, in the discussion section, giving the impression that those conclusions are drawn by induction or deduction from the reported evidence. Instead, explains Medawar, the expectations that a scientist begins with provide the incentive for the experiments, and determine their nature, and they determine which observations are relevant and which are not. Only in the light of these initial expectations that the activities described in a paper have any meaning at all. The expectation, the original hypothesis, according to Medawar, is not the product of inductive reasoning but of inspiration, educated guesswork. Medawar was echoing Karl Popper, who proclaimed that

Confirmation

…But, it isn’t enough to prove a hypothesis once. Other researchers must also be able to repeat the study and produce the same results, if the hypothesis is to remain valid…

Sometimes experimenters make systematic errors during their experiments, Consequently, it is a common practice for other scientists to attempt to repeat experiments, especially experiments that have yielded unexpected results. Accordingly, scientists keep detailed records of their experiments, to provide evidence of their effectiveness and integrity and assist in reproduction. However, it is not possible for a scientist to record everything that took place in an experiment. He must select the facts that he believes are relevant to the experiment. This may lead to problems if some supposedly irrelevant feature is questioned. For example, Heinrich Hertz did not report the size of the room that he used to test Maxwell's equations, and this turned out to account for a deviation in the results. The problem is that parts of the theory must be assumed in order to select and report the experimental conditions. Observations are thus sometimes described as being 'theory-laden'.

It seems to be only very rarely that scientists falsify their results; any scientist who does so takes an enormous risk, because if the claim is important it is likely to be subjected to very detailed scrutiny, and the reputation of a scientist depends upon the credibility of his or her work. Nevertheless there have been many well publicised examples of scientific fraud, and some have blamed the insecurity of employment of scientists and the extreme pressure to win grant funding for these instances. Under Federal regulations "A finding of research misconduct requires that: There be a significant departure from accepted practices of the relevant research community; and The misconduct be committed intentionally, or knowingly, or recklessly; and The allegation be proven by a preponderance of evidence."

Honor in Science, published by Sigma Xi , quotes C.P. Snow (The Search, 1959): "The only ethical principle which has made science possible is that the truth shall be told all the time. If we do not penalise false statements made in error, we open up the way, don’t you see, for false statements by intention. And of course a false statement of fact, made deliberately, is the most serious crime a scientist can commit." It goes on to say: "It is not sufficient for the scientist to admit that all human activity, including research, is liable to involve errors; he or she has a moral obligation to minimize the possibility of error by checking and rechecking the validity of the data and the conclusions that are drawn from the data."

Statistics

…If the initial study was carried out using a small number of samples or people, larger studies are also needed. This is to make sure the hypothesis remains valid for bigger group and isn't due to chance variation…

Statistical analysis has become a standard part of hypothesis testing in many areas of science. This use of statistics formalises the criteria for disproof, essentially by allowing statements of the following form "If a given hypothesis is true, the chance of getting the results that we in fact observed is (say) only 1 in 20 or less (P<0.05), Accordingly, there is a strong likelihood that the hypothesis is wrong, and accordingly we reject it.

This leads to a notion of a hypothesis that is quite different to Popper's. For instance, we might predict, boldly on the basis of our inspiration combined with a deep appreciation of theory, that a certain chemical will produce a certain surprising effect. However the hypothesis that we will test in such cases is often not this, but the complementary null hypothesis - the hypothesis that the chemical will have no effect. The reason for this shift is simple but inescapable - If our original hypothesis only tells us that there will be an effect but is vague about its expected magnitude, we can still logically disprove the null hypothesis, by showing an effect, but we can never disprove the hypothesis that the chemical is effective, as we cannot exclude the possibility that the effect is there but smaller than we can measure reliably.

The answer might well be to choose hypotheses that give quantitatively precise predictions, but in many areas of science this is often unrealistic. In medicine for example, we might reasonably expect a new drug to be effective in a particular condition on the basis of our understanding of its mechanism of action, but might be very uncertain of how big the effect is likely to be because of numerous and considerable uncertainties - how many individuals in a genetically variable population will be resistant to the drug for example, and how quickly will tolerance to the drug develop in individuals that respond well?

This use of statistics can attach a level of certainty to the rejection of a null hypothesis, but does not alllow any level of certainty to be attached to the hypothesis if the outcome of the experiment supports the original hypothesis. This is confusing, because conventionally we are using the word "hypothesis" for two quite different things when we talk about the original hypothesis and the null hypothesis.

To make this clearer; the scientist starts with an original hypothesis, a bold speculation, consistent with existing theory but extending that theory in some way. The scientist then tests the hypothesis by deriving from it a prediction -some proposition that will be true if the hypothesis is true but would not be expected to be true otherwise. The scientist then may design an experiment to test the null hypothesis - the assertion that the prediction is false. If the null hypothesis is disproved, the original hypothesis survives, and may be said to be supported by the experiment.

In fact this is not hypothesis testing in Popper's sense at all, and verification of this type is something that Popper considered to be at best a weak form of corroborative evidence. Part of that weakness comes because Popper argued that it is impossible to put any sensible measure on the degree of support that such evidence gave to a hypothesis. In appendix ix to The Logic he states: "As to degree of corroboration, it is nothing but a measure of the degree to which hypothesis h has been tested...it must not be interpreted therefore as a degree of the rationality of our belief in the truth of h...rather it is a measure of the rationality of accepting, tentatively, a problematic guess."

There is an important school of Bayesian statistics that seeks to provide a statistical basis for support by induction, and some areas of science use these approaches; but in much of science this approach is not tenable because of the difficulty of attaching a priori probabilities in any meaningful way to the alternative predicted outcomes of an experiment.

Progress in science

…Over time, scientific opinion can change. This is because new technologies can allow us to re-examine old questions in greater detail.

Einstein's prediction (1907): Light bends in a gravitational field

Einstein's theory of General Relativity makes several specific predictions about the observable structure of space-time, such as a prediction that light bends in a gravitational field and that the amount of bending depends in a precise way on the strength of that gravitational field. Arthur Eddington's observations made during a 1919 solar eclipse supported General Relativity rather than Newtonian gravitation.

See Also

Models of scientific inquiry Pseudoscience

Further reading

• The Keystones of Science project, sponsored by the journal Science has selected a number of scientific articles from that journal and annotated them, illustrating how different parts embody the scientific method. Here is an annotated example of the scientific method example.
• Bacon, Francis Novum Organum (The New Organon), 1620. Bacon's work described many of the accepted principles, underscoring the importance of Theory, empirical results, data gathering, experiment, and independent corroboration.
• Dewey, John (1991) How We Think, D.C. Heath, Lexington, MA, 1910. Reprinted, Prometheus Books, Buffalo, NY
• Heisenberg, Werner (1971) Physics and Beyond, Encounters and Conversations, A.J. Pomerans (trans.), Harper and Row, New York, NY pp.63–64
• Latour, Bruno, Science in Action, How to Follow Scientists and Engineers through Society, Harvard University Press, Cambridge, MA, 1987.
• McComas WF, ed. The Principle Elements of the Nature of Science: Dispelling the Myths, from The Nature of Science in Science Education, pp53-70, Kluwer Academic Publishers, Netherlands 1998.
• Poincaré H (1905) Science and Hypothesis Eprint

External links

• An Introduction to Science: Scientific Thinking and a scientific method by Steven D. Schafersman.
• Introduction to a scientific method
• Theory-ladenness by Paul Newall at The Galilean Library
• Scientific Method
• Analysis and Synthesis: On Scientific Method based on a study by Bernhard Riemann From the Swedish Morphological Society
• Using the scientific method for designing science fair projects from Science Made Simple