66 min listen
Auxiliary Turtles All the Way Down
FromThe Black Goat
ratings:
Length:
68 minutes
Released:
Feb 12, 2020
Format:
Podcast episode
Description
The path from theory to study consists of a thousand decisions, big and small. How and how much do these decisions matter? We discuss a recent crowdsourced meta-study that tried to find out. Fifteen teams of researchers were given 5 different hypotheses and told to design a study to test them, then they ran all the studies and got widely varying results. What are the implications of this study for how we should think about the role of theory in study design? What does it say about the different functions of direct and conceptual replications? Is this evidence of hidden moderators? How predictable were the differences in results, and were they predictable because of differences in the study designers' expertise, biases, or something else? Plus: We answer a letter about getting scooped on a systematic review.
Links:
Crowdsourcing hypothesis tests: Making transparent how design choices shape research results by Landy et al. in Psychological Bulletin
200 researchers, 5 hypotheses, no consistent answers. Coverage at Wired by Christie Aschwanden
The Black Goat is hosted by Sanjay Srivastava, Alexa Tullett, and Simine Vazire. Find us on the web at www.theblackgoatpodcast.com, on Twitter at @blackgoatpod, on Facebook at facebook.com/blackgoatpod/, and on instagram at @blackgoatpod. You can email us at letters@theblackgoatpodcast.com. You can subscribe to us on iTunes or Stitcher.
Our theme music is Peak Beak by Doctor Turtle, available on freemusicarchive.org under a Creative Commons noncommercial attribution license. Our logo was created by Jude Weaver.
This is episode 75. It was recorded on January 30, 2020.
Links:
Crowdsourcing hypothesis tests: Making transparent how design choices shape research results by Landy et al. in Psychological Bulletin
200 researchers, 5 hypotheses, no consistent answers. Coverage at Wired by Christie Aschwanden
The Black Goat is hosted by Sanjay Srivastava, Alexa Tullett, and Simine Vazire. Find us on the web at www.theblackgoatpodcast.com, on Twitter at @blackgoatpod, on Facebook at facebook.com/blackgoatpod/, and on instagram at @blackgoatpod. You can email us at letters@theblackgoatpodcast.com. You can subscribe to us on iTunes or Stitcher.
Our theme music is Peak Beak by Doctor Turtle, available on freemusicarchive.org under a Creative Commons noncommercial attribution license. Our logo was created by Jude Weaver.
This is episode 75. It was recorded on January 30, 2020.
Released:
Feb 12, 2020
Format:
Podcast episode
Titles in the series (85)
Significant Feelings: Everybody's talking about p-values. An important part of the replicability discussion has been about the correct use and interpretation of p-values, and the potentially distorting incentives attached to getting one below .05. And recently, Simine was a c... by The Black Goat