Organisme intervenant (ou équipe pour les séminaires internes)
Université de Turin
Nom intervenant
Niccolò Campus
On the prediction of replicability of social science experiments

In recent years there has been much debate about common statistical practices in many fields (medicine, biology, genomics, psychology, economics) and about the spread of false positive findings in the scientific literature. Several large-scale replication projects (OSC 2015, Camerer et al. 2016, 2018 and others) obtained poor replication rates (as little as 36%) and weaker effects. Existing attempts at predicting replicability of experimental claims all rely on ‘black-box’ machine learning techniques (Altmejd et al. 2019, Yang, Youyou and Uzzi 2020). We design a general framework to predict experiment outcomes (statistical significance and effect size) based on Specification Curve Analysis (Simonsohn, Simmons and Nelson 2015) and propose new estimators of replicability derived from meta-analysis and Bayesian Model Averaging. These estimators should better capture the ‘inner workings’ of any given study and therefore have higher explicative and predictive power than ML/NLP algorithms. We will test the predictive performance of our estimators on a large population of experimental studies drawing upon the aforementioned large-scale high-powered replications.

Salle de réunion 142, bâtiment 210
Date du jour