J Scott Armstrong and Kesten Green, The Scientific Method: A Guide to Finding Useful Knowledge. Cambridge University Press, 2022.
Science and academic life at large have changed out of recognition since the second world war under the influence of rapid growth, increasing government control and the politicisation of the allocation of research grants.
The world of science is in a bad way and academic studies in the philosophy and social studies of science have not helped. The philosophy of science took a particularly unhelpful turn in the 1930s when the subject became an academic speciality in Vienna and elsewhere on the Continent. The school of thought known as logical positivism or logical empiricism became embedded in the universities of the anglosphere when many exponents, notably Rudolph Carnap and Karl Hempel, occupied prestigious chairs when they fled to the west to escape from Hitler.
The most popular reaction in the form of Kuhn’s paradigm theory and social studies of science has been equally unhelpful because it was led by exponents of cultural Marxism or fellow-travellers who rapidly took control of the humanities and social sciences.
This book is not a contribution to the academic literature, it is much more important and helpful than that. It is actually more than one book, in a single set of covers. One of the books is a practical handbook or an operating manual for working scientists who are trying to make sense of information and solve a scientific or practical problem while in contrast philosophers of science are concerned with the fashionable problems in the academic literature at the time.
The core of that book is a series of checklists to help scientists to navigate on the journey from the beginning of a research career to particular research projects from the start (selecting a topic) to the end, including publication and verbal presentation of the findings. Good practice in all of the elements of the process should become second nature for researchers who are well taught, well supervised during their doctoral studies and well mentored by senior colleagues later on. However not all postgraduate students are well supervised because supervision is an art and a science in itself which not all academics master
The first list is “self assessment of self-control” in a chapter on what it takes to be a good scientist. The prospective scientist is advised to think hard about the pros and cons of a research career. The 1925 novel Arrowsmith by Sinclair Lewis could be prescribed as a text to learn about the trials and tribulations of idealistic researchers.
The hero of the book is a radical and independent medical researcher who adheres to strict principles of scientific methods. The central drama of the book is Arrowsmith’s discovery of a special kind of cell in the blood that destroys bacteria and his dilemma in the face of an outbreak of bubonic plague on a fictional Caribbean island. His scientific principles demand that he should test the therapy before its mass use on the Island, even at the expense of lives that might be saved.
As for the exhilaration of advancing the frontier of knowledge, the authors cite a survey of students in the doctoral programs in economics at eight leading universities in the US. The survey found that 18% of the students experience moderate or severe depression and 11% think about suicide in a two-week period. Not surprisingly, economics is dubbed “the dismal science.”
There is a short checklist on identifying important problems and a long list of things to attend to in planning and executing the data collection and analysis.
After drawing out conclusions, the scientist turns to disseminate the findings. There are progress reports and seminar papers on preliminary findings along the way but the critical products to maintain tenure and ongoing grants are papers published in peer-reviewed literature. Hence the talk about “publish or perish” that has been an ongoing refrain at last since the 1960s when I first became aware of it.
The checklist starts with “Explanation of findings and why they are credible and useful.” It has to be said that many claims of usefulness in the softer sciences tend to strain credibility. These are the projects that regularly attract criticism from conservative critics of the major grants allocated by bodies such as the Australian Research Council.
The largest checklist concerns writing the paper. By the time the researcher gets to this point he or she should be thoroughly familiar with this particular literary form but familiarity does not guarantee that the beginner or even experienced scientists will do a good job. The most important suggestions are at the end of the list: “Use editors to improve clarity and rewrite until the report is clear and interesting.” By editors I presume they mean colleagues, preferably experienced in report writing.
The last three checklists are concerned with sales and marketing. This is the work that some of the most conscientious and committed truth-seekers overlook, find distasteful or dismiss as beneath the dignity of scholars. This probably did not matter so much a generation ago when there were less scientists and good work would usually be picked up by influential workers and given due attention including all-important citations. This is no longer the case unless the researcher has the necessary connections in place, otherwise they have to be reached by well-organized efforts to contact and cultivate them.
Talks and oral presentations are important and it is surprising how badly some leading scientists perform, without attention to the details and the preparation that make all the difference. Serious work, practice and coaching are required to develop advanced skills in talks and presentations and the effort will be well rewarded when the scientists has an opportunity to put their work on display at seminars and conferences.
The authors provide valuable advice to scientists and there is more! The book begins with a survey of the problems that afflict science at present and, it ends with practical suggestions for improvement that can be taken up by the range of stakeholders in the scientific enterprise. There are chapters on assessing the quality of scientific practice, the problem of advocacy, concerns with the effectiveness of peer review and the complications that arise with government funding and direction of research. The positive suggestions are offered to university managers, journal editors, governments, courts, the media, and interested individuals.
The Scientific Method: A Guide to Finding Useful Knowledge. Cambridge University Press, 2022.
9 thoughts on “Book review: The scientific method”
This comment regarding the issue of the scientific method is an extremely critical one, given we now seem to live in a world where there is a distinct lack of the use of a scientific method or even a logical or rational method for observing and describing cycles, events and processes.
Much of the problem in my view lies in the fact that hardly anyone these days goes back to first principles. Most take as their starting point some claim by someone else and rarely go back to check whether it was an accurate claim in the first place. Much of this attitude I suspect stems from the comment regarding over reliance on the accuracy of peer reviewed journal articles. Whilst these are supposed to be the gold standard in truth, in fact those over the past 4 -5 decades are woeful and anyone who relies on them to the degree that they are usually finds themselves digging down an intellectual rabbit hole.
Secondly, our dysfunctional education systems has created a class of people who simply cannot think for themselves. Education these days is just a process of memorisation, and hardly ever about working their through an issue or problem to understand the pattern or process that is really at play, instead it’s just about the result or outcome. Additionally, given the lack of training in mathematics and an over reliance on calculators and computers, hardly anyone knows if a mathematical result is accurate or plausible, mostly they just accept it because they can’t think of an alternative answer and consequently inaccuracies get baked into the system (see Climate Change).
There is no easy answer, as we now have almost two generations who have been inducted into this illogical non-thinking process and progress is only going backwards. There is also no current political leadership who is capable of changing this course and so it could be that the Western Empire and Enlightenment is almost at the end and we are in the first stages of collapse.
Recommended for all politicians and jismists.
David Armstrong or J Scott Armstrong?
Great comment – sad but so very true.
In-house proofreading fails again:)
Charles, I hope the situation that you describe can be improved and it will be a difficult task, at least this book is a weapon in our hands for the contest although as usual the people who most need to read it will not do so:)
At the moment we are on the defensive, just keeping valuable traditions alive for future use.
Yes, it is. When the powers that be use science to boldly lie, and the people know they are bold lies, so you can never trust anything published by “science” again.
That is the consequence of “science” systemically lying.
People know that climate change is fake and gay, that COVID vaccines and much of the rhetoric (masks, church and business closures, etc) was fake and gay. We also know that 50% of what’s published in science journals is unreproducible garbage.
Whoa to those who now listen to and make decisions from mainstream “science” as it comes from the Father of Lies.
So now that science is dead, where to next?
I’ll mostly stick with engineering, which is the provable application of science in the real world and leave “science”to the NPCs of the world.
Many thanks for this Rafe. I’ve arranged to have a copy placed in the library of the University where I used to work.
Good work Mak!
“Much of the problem in my view lies in the fact that hardly anyone these days goes back to first principles. “
Start with the basics and don’t skip any steps – I’ve lost count of the number of times doing this has made me look “brilliant” because I solved the problem no-one else could. Most of the time, they weren’t even “hard” ones, just assumptions that got in everyone else’s way of seeing what was wrong – that, and the willingness to ask “dumb” questions.