Philosophy of science and metascience — the method
Area. Popper, Lakatos and the Duhem–Quine problem; the reproducibility literature (many-analysts studies, preregistration, Registered Reports); reliability statistics.
Introduction. Popper made falsifiability the demarcation of science; Duhem and Quine showed that no claim faces a test alone, so any refutation can be deflected onto an auxiliary. Lakatos’s answer is to judge research programmes, not claims: a programme is progressive when its adjustments yield novel corroborated predictions, degenerating when they only absorb anomalies. Metascience since 2011 has measured how much researcher freedom shapes results (29 teams, one dataset, opposite conclusions), how little voluntary preregistration constrains (three-quarters of preregistered studies deviate), and how much a gatekeeper changes (96% positive results in the standard literature, 44% in Registered Reports). Cox, Arnold and Villamayor-Tomás did for Ostrom’s design principles what the paper proposes for Beer’s: coded 91 studies against them and reformulated the principles.
Three names. Imre Lakatos; Brian Nosek and Chris Chambers (Registered Reports); Klaus Krippendorff (alpha).
Importance for cybernetics / VSM. Van der Zouwen inventoried the cybernetics literature twice and found the proportion of empirically tested claims declining. The VSM has no protocol of its own for saying what would count against it.
Importance for the article. The receipt protocol (§4), the programme-level appraisal (§4.7, §12.8), the preregistered low-agreement prediction in Demonstration III (§8), and the Registered Reports adoption route (§10.6) all come from here. A reviewer will ask why a register would work when preregistration mostly does not; §10.6 is the answer.
Start here.
- the appraisal criterion the paper adopts.
- why the register must be enforced at publication.
- the base rate Demonstration III predicts against.
- the executed precedent.
One outside text. Chambers, C. (2017). The Seven Deadly Sins of Psychology: A Manifesto for Reforming the Culture of Scientific Practice. Princeton University Press — the case for Registered Reports from their originator.
More Details
This theme supplies the paper’s method rather than its object. The receipt protocol is a piece of applied philosophy of science (what a claim must specify to be capable of losing, and how a programme rather than a claim is appraised) fitted with instruments borrowed from metascience (preregistration, Registered Reports, many-analysts designs, reliability statistics, coded evidence reviews) and checked against one precedent of a neighbouring field correcting itself without any of that apparatus. A co-author must be able to say why the paper cites Lakatos and not just Popper, what the metascience numbers actually show, what Krippendorff’s alpha does and does not settle, and why the scale-free episode both supports and threatens the paper’s case. Take the sub-themes in spec order: 2.1 gives the appraisal logic, 2.2 the machinery, 2.3 the one statistic the protocol names, and 2.4 the precedent the paper must confront last.
Falsification, research programmes and the Duhem–Quine problem
Popper’s criterion was that a scientific claim must forbid some observation, and that a theory is tested by attempts to produce the forbidden case. Duhem and later Quine showed why single refutations rarely bite: a prediction follows from a theory only together with auxiliary hypotheses about instruments, initial conditions
Metascience: many-analysts studies, preregistration, Registered Reports
Metascience studies research practice with research methods. Three findings matter here. Many-analysts studies give the same data and question to independent teams and measure the spread of conclusions: Silberzahn et al. (2018) had 29 teams test whether referees give more red cards to dark-skinned players and obtained odds ratios
Measurement, reliability and analyst agreement
Content analysis is the technique of making replicable, valid inferences from texts to their contexts; Krippendorff’s textbook gives its design logic (unitising, sampling, coding, reducing, inferring) and its treatment of reliability. Reliability among coders is the degree to which independent coders, working from the same instructions, assign the same
Self-correction in a neighbouring field: the scale-free network episode
Barabási and Albert (1999) proposed that many real networks have power-law degree distributions generated by growth with preferential attachment, and "scale-free" became the headline result of network science, extended to metabolic, protein, social and Internet topologies with claims about robustness to random failure and vulnerability to targeted attack. Within