John Droz is a prolific commentator on matters of education, science and policy issues like the quest for net zero that has been given legs by bad science and scientifically illiterate politicians, university administrators and journalists.

This is a particulalry telling piece that he published this morning and it prompted me to suggest that a revival of Karl Popper’s “crtical rationalism” will be helpful. He asked me for a comment on his site to advance the discussion and I am posting this to provide a link to put on his site.

Popper’s Evolutionary Theory of Knowledge

This piece was prepared in 1983 for a radio talk on a science show on ABC, the national broadcaster. The  presenter wrote to say that this talk was not required because the philosopher Max Charlesworth gave a talk covering the same ground. No doubt Max Charlesworth mentioned Popper, Kuhn, Feyerabend and Lakatos but I suspect that he did not indicate that Popper represented an improvement on their views because he referred to Popper’s popularity among scientists as something of joke on the scientists.

This article explains how Popper’s ideas promote creativity and effective problem-solving in science and elsewhere. In 1934 he started a new era in the philosophy of science with a book written in German, and translated in 1959 under the title The Logic of Scientific Discovery. He rejected the traditional idea that scientific knowledge was based on a method called induction whereby theories are verified by observations. Popper argued that the logical process of induction simply does not exist. Theories are forever tentative and the most useful function (and the only logically decisive effect) of observations is to act as tests or attempted falsifications of theories.

Since 1960 Popper’s work has been dominated by biological themes. He has revived an approach to knowledge called evolutionary epistemology. This approach was very popular last century but has lapsed in modern times due to the obsession with physics in the philosophy of science.

Evolutionary epistemology applies Darwin’s principle of natural selection to scientific theories and to other forms of knowledge. It is concerned with problem-solving and error-elimination under various forms of selective pressure. This contrasts with most theories of knowledge that are concerned with the foundations of belief or the probability of theories.

Popper started with the old idea that knowledge grows by trial and error, or in more learned terms, by conjecture and refutation. He generalised this theory to encompass all forms of learning and problem-solving, including the evolution of life on earth.

On his account every organism, from the amoeba to Einstein, is constantly engaged in problem solving. In the plant and animal world this involves the production of new reactions, new organs, new forms of life. For humans it involves the production of new ideas. When these forms of life or theories appear they confront selective pressures. These may come from the biological environment or from competing forms of life. Ideas meet the competition of alternative theories, critical arguments and experimental tests.

The central motif of Popper’s evolutionary epistemology is the four-step problem-solving schema:

P —> TS —> EE —> P

The starting point is a problem, which evokes tentative solutions. These are subjected to the process of error elimination by way of critical discussion and experimental testing. In the course of these activities new problems emerge.

The Significance of this Scheme

First of all, it can be used to correct many false views about science that promote over-specialisation and antagonism between science and the humanities. For example, Sir Peter Medawar in his book Pluto’s Republic describes the tension between the romantic and the rational views of science. The romantic points to the poetic inspiration involved in creating new theories. The rationalist makes much of data collection, experimentation and logical analysis. This conflict has broad cultural implications. The triumph of Newtonian mechanics was widely perceived as the full flowering of the so-called inductive method to find the truth by accumulating observations. But this achievement provoked a revolt by romantics and poets who could not stomach a view of human activity that had no place for the imagination. Nor could they accept the mechanical universe. The result of this collision has been a kind of cultural schizophrenia, with imagination set against reason, the organic set against the mechanical, the inspiration of the poet set against the empiricism of the scientist.

Popper’s theory cures this cultural schizophrenia. It harmonises the relationship between the various elements of the situation for both scientists and artists, and indeed for anyone. These elements include traditional beliefs, criticism, logic, imagination and experimental trials. These elements each have a role to play and so there is no need for the tensions and antagonisms that flow from partial and narrow views of problem-solving and creativity, whether in science, art, technology or daily life.

Secondly, the schema brings out the importance of recognizing problems and working on them. In this schema a problem functions as an ecological niche to be colonised by tentative solutions. Problems are not a nuisance because they provide a habitat for new species of ideas. This provides a theory of discovery, based on the creative function of criticism. Criticism is often regarded as a negative kind of thing and critics are supposed to be second-rate people, rather like teachers, who cannot actually do anything. To grasp the full power of evolutionary epistemology it is necessary to understand this creative function of criticism. Problems are the habitat where new ideas grow and criticism has two functions, which are about equally valuable: (a) to eliminate error and (b) is to generate new problems, i.e. new habitats. Thus Popper’s theory brings out both the error elimination and the creative function of criticism. And we need to maximise the free play of criticism to get the best out of both its functions.

This brings us to the third function of the evolutionary schema. It reveals the protective devices and the conservative limits on criticism that are imposed by many theories of science. I will criticise four of these, namely the traditional theory of induction and the supposedly radical innovations of Kuhn, Lakatos and Feyerabend.

Conservative Limits on Criticism

The theory of induction claims that there is a method or a process called induction, whereby scientific laws or generalisations are derived from repeated observations. But observations are only useful if they relate to a problem and in practice the inductive method can lead to mere accumulation of data. Even if this extreme is avoided the inductive approach tends to cramp the imaginative search for new ideas. It does not assign any role at all to criticism. It encourages over-specialisation (sticking to data collection in your own field). It provides no incentive to explore the wider theoretical, technological and moral implications of problems and theories.

We now turn to Thomas S. Kuhn and his ideas about normal science and paradigms. These were spelled out in The Structure of Scientific Revolutions (1962). Kuhn rejected the view that science grows in a steady fashion as observations accumulate. He suggested instead that periodic revolutions occur, with whole world-views changing in the process. These world-views he called paradigms. In the period between revolutions we have what Kuhn called “normal science”, performed by “normal scientists” who work on relatively minor “puzzles”. In contrast with a Popperian scientist who accepts no limits to criticism, normal scientists never try to explore or criticise the wider framework of ideas (the paradigm) in which the puzzles are located.

The secret of the success of Kuhn’s ideas lies in their symbiotic relationship with inductivism. A symbiotic relationship is a partnership between two species where both derive benefit. The inductivist approach produces “normal scientists” who uncritically accept the “paradigms” that they inherit. Kuhn made his reputation by describing this situation and this is his debt to inductivism. In return his theory legitimates whatever scientists are doing, and so he repays his debt by providing support for inductivism. At the surface level his ideas can be seen as a challenge to some ideas about induction which Popper demolished in 1934. At the deeper level Kuhn’s ideas are thoroughly conservative and unhelpful for working scientists.

We now proceed to Imre Lakatos. He appeared to be following Popper in resisting Kuhn’s ideas about paradigms and he produced a complicated methodology of scientific research programs to provide a rational reconstruction of scientific progress. In contrast with the symbiotic relationship between Kuhn and inductivism, Lakatos formed a parasitic relationship with both Popper and Kuhn. From Popper he took the idea of research programmes. From Kuhn he took the idea that the central part of the program should be protected from criticism. He used some exciting new terms; his program has a “hard core” of theories. A “protective belt” of lesser theories that can be modified or discarded surrounds it. But beneath the verbal froth and bubble this is a recipe for conservatism. It prohibits the most important and fruitful criticisms which are directed at the framework assumptions of the program.

Finally a look at Paul Feyerabend and his assertion of methodological anarchism, summed up in the formula “anything goes”. Reverting to the four-stage problem solving schema, we welcome the notion that anything goes at the stage of Tentative Solutions (the more of these the better). But if this permissive attitude is allowed at the error elimination stage, then everything stays.

So in conclusion, contrast the protective devices in the four theories sketched above with Popper’s advice to use unlimited criticism at all levels including the often unstated framework assumptions which dominate the selection of problems and the formulation of solutions.

Objections to Popper

One might ask: What manner of counter-arguments do these schools use against Popper? They have two main lines of defence, first, they recycle arguments that Popper raised and refuted in his first book (in 1934). Second, they claim that Popper does not adequately describe what the scientific community is actually doing. This is a very perverse criticism. Popper wants to help scientists to solve their problems more effectively by mobilising the creative energy of critical thought. His ideas have to be criticised as tentative suggestions to that end. What sort of criticism is it to say that most scientists are not really very critical in their thinking? Imagine someone making suggestions for famine relief in Ethiopia who is criticised by people who point out that there is famine in Ethiopia? This is the level of logic that has been used to discredit Popper’s ideas.

If scientists take these ideas to heart then the practice of many will change. And if they are taken up by people other than scientists then we can hope to move towards a more problem-solving community. Immense resources of creative and critical problem-solving energy are locked up by conservative theories of knowledge in their traditional and radical forms. These resources may be released by the evolutionary approach championed by Karl Popper. But these ideas are threatened because Popperian philosophers are an endangered species and they could be extinct in the universities within a generation.

This places a heavy responsibility upon people outside the academies to keep these ideas alive, (so long as they stand up to criticism) and to preserve them for future generations who in time may break down the barriers that the ruling academic guilds have erected against Popper’s ideas.