What are the classics of the "Science of Science" or "Meta Science"? If you were teaching a class on the subject, what would go in the syllabus?

Here's a (very disorganized and incomplete) handful of suggestions, which I may add to. Suggestions welcome, especially if you've dug into relevant literatures.
1. The already classic "Estimating the reproducibility of
psychological science" from the Open Science Collaboration of @BrianNosek et al. https://t.co/yjGczLZ6Je

(Look at that abstract, wow!)
Many people had pointed out problems with standard statistical methods, going back decades (what are the best refs?). But this paper was a sledgehammer, making it impossible to ignore the question: what, if anything, were we actually learning from all those statistical studies?
2. Dean Keith Simonton's book "Creativity in Science: Chance, Logic, Genius, and Zeitgeist". If an essentially scientometric book could be described as a fun romp through science & creativity, this would be it https://t.co/RQ935H1fKs
3. From the philosophy of science literature, I especially like Lakatos's "Proofs and Refutations", Feyerabend's "Against Method", and Kuhn's "The Structure of Scientific Revolutions". I should probably dig deeper into other schools (recs?) (Yes, I've read Popper's main works.)
4. Speaking of Feyerabend, Steve Weinberg had some surprisingly sympathetic & characteristically insightful comments on Feyerabend, which I expect would bear re-reading. I've lost the reference.
5. Switching genre, there's the work of @pierre_azoulay and collaborators, studying the HHMI versus NIH approaches to discovery in the life sciences: https://t.co/9eta708dVf
6. From 2018, a nice review paper on "The Science of Science", coming from a network science / scientometrics point of view. There are tonnes of interesting observations in the paper, many of which I bundled up in this thread: https://t.co/potbylhgxt

https://t.co/xduj2A8c8q
7. One of our best long-term observers of science and science policy was Daniel Greenberg (who passed away last year). Many possibilities to choose from, but here's one I got a lot out of: "Science, Money, and Politics": https://t.co/8QANkuL6ZA
8. Harry Collins has done some wonderful work on the central role of tacit knowledge in science. Here's one of his classics, on the role of tacit knowledge in figuring out how good sapphire is as a lasing material: https://t.co/Oaz2VWySmM
That sounds very specialized. It's not. It goes to questions at the very heart of science, both institutionally and methodologically. Rather, Collins' paper is a beautiful detailed study of tacit knowledge.
(Tangentially: one way my thinking has changed is in gradually understand how tied together our institutions and our methodologies are. There's a kind of Conway's Law in action: our institutions tend to mirror our methodology, and vice versa.)
9. The institutions around us are, of course, all made up, out of ideas - things like universities, PhDs, journals, etc, even the notion of "Science", are first and foremost conceptual innovations. I'd love to understand the history of those ideas better.
One striking text in this vein is Francis Bacon's 1627 "The New Atlantis", which introduces "Salamon's House", which strongly influenced the design of the Royal Society (1660), and modern universities. https://t.co/bznUG1eTH5
10. Another good one in this vein is Vannevar Bush's "Science: the Endless Frontier", which helped establish the concepts underpinning the modern basic research ecosystem https://t.co/bpc1nPg8A7
11. Indeed, I've heard it argued that Bush is the person most responsible for developing the concept of "basic research", and this was done in part as a way of winning a political fight to motivate funding for pure research. The argument is made here: https://t.co/QPMCAEYSLE
12. David Lang suggests Paula Stephan's book here: https://t.co/zQiuTlWOdR

The book has been in my queue for some time, and is almost certainly a good overview of a huge chunk of economic thinking about science.
Interlude: thanks for the many wonderful replies!!

Twitter threading makes it a little hard to skim the thread. Expandable tree version here, thanks to @paulgb's great treeverse Chrome extension: https://t.co/vF4JF0Gf4J
I'll indulge myself a bit, and ask @dabacon, @AndrewDohertyQu, @quantum_aram, @uncatherio, @albrgr, @DGoroff, @BrianNosek, @juliagalef, @juanbenet, @AdamMarblestone, @patrickc, @pierre_azoulay if you have any particular favorite additions for the list?

More from Science

@mugecevik is an excellent scientist and a responsible professional. She likely read the paper more carefully than most. She grasped some of its strengths and weaknesses that are not apparent from a cursory glance. Below, I will mention a few points some may have missed.
1/


The paper does NOT evaluate the effect of school closures. Instead it conflates all ‘educational settings' into a single category, which includes universities.
2/

The paper primarily evaluates data from March and April 2020. The article is not particularly clear about this limitation, but the information can be found in the hefty supplementary material.
3/


The authors applied four different regression methods (some fancier than others) to the same data. The outcomes of the different regression models are correlated (enough to reach statistical significance), but they vary a lot. (heat map on the right below).
4/


The effect of individual interventions is extremely difficult to disentangle as the authors stress themselves. There is a very large number of interventions considered and the model was run on 49 countries and 26 US States (and not >200 countries).
5/
It was great to talk about reproducible workflows for @riotscienceclub @riotscience_wlv. You can watch the recording below, but if you don't want to listen to me talk for 40 minutes, I thought I would summarise my talk in a thread:


My inspiration was making open science accessible. I wanted to outline the mistakes I've made along the way so people would feel empowered to give it a go. Increased accountability is seen as a barrier to adopting open science practices as an ECR

It also comes across as all or nothing. You are either fully open science or your research won't get anywhere. However, that can be quite intimidating, so I wanted to emphasise this incremental approach to adapting your workflow

There are two sides to why you should work towards reproducibility. The first is communal. It's going to help the field if you or someone else can reproduce your whole pipeline.


There is also the selfish element of it's just going to help you do your work. If you can't remember what your work means after a lunch break, you're not going to remember months or years down the line

You May Also Like

I'm going to do two history threads on Ethiopia, one on its ancient history, one on its modern story (1800 to today). 🇪🇹

I'll begin with the ancient history ... and it goes way back. Because modern humans - and before that, the ancestors of humans - almost certainly originated in Ethiopia. 🇪🇹 (sub-thread):


The first likely historical reference to Ethiopia is ancient Egyptian records of trade expeditions to the "Land of Punt" in search of gold, ebony, ivory, incense, and wild animals, starting in c 2500 BC 🇪🇹


Ethiopians themselves believe that the Queen of Sheba, who visited Israel's King Solomon in the Bible (c 950 BC), came from Ethiopia (not Yemen, as others believe). Here she is meeting Solomon in a stain-glassed window in Addis Ababa's Holy Trinity Church. 🇪🇹


References to the Queen of Sheba are everywhere in Ethiopia. The national airline's frequent flier miles are even called "ShebaMiles". 🇪🇹