mastodon 4.7.3A mathematician/entrepreneur in social science. I write here about psychology, open science, critical thinking, philosophy, tech, and society.
I looked at 325 psychology study replication results today, and the best simple rule I found for telling if a study would replicate is this:
When the original study's p-value was at most 0.01, about 72% of the papers replicated.
Whereas when p>0.01, only 48% replicated.
Others have found similar results.
For instance, @uebernerd@nerdculture.de has suggested p=0.005 as a cutoff:
We're doing a live taping of the Clearer Thinking podcast in Manhattan on Jan 30th, 2024 on the "Moral Status of Insects and A.I. Systems, and Other Thorny Questions in Global Priorities Research" - with Jeff Sebo - it's free to attend! Sign up here:
https://docs.google.com/forms/d/e/1FAIpQLSc4SgsjvHXCueNASskgr5p2_ZXRNPh3bouT9NYbgLHtlc7_8A/viewform
Time socializing correlates with well-being. But does it matter how diverse the set of social connections you socialize with is (e.g., do people who socialize with a mix of friends + colleagues + family have higher well-being)?
Our replication attempt: https://replications.clearerthinking.org/replication-2022pnas119-43/
We analyzed Elon Musk's predictions to see how accurate they have been (only including those that had an explicit or implicit time frame, and where that time has passed). Here are the results:
A call to academic psychologists (and those who trained in related fields) - it would be fantastic if you'd participate in our study on controversial and hotly debated topics in psychology! We'd also really appreciate it if you'd share this as we need more participants:
Link to participate in the study on hotly debated topics in academic psychology:
https://www.guidedtrack.com/programs/bek8ujv/run?source=mastodon
It covers topics like:
• effects of social media on teen mental health
• fraud in academic psychology
• the interpretation of controversial findings
Thank you!
Academics from psychology, behavioral science, and adjacent fields, here's a challenge for you! Test your skill predicting psychological correlations for a new study. You'll receive $5 for participating.
https://www.guidedtrack.com/programs/18blaeo/run?expert=1
We'd also appreciate you sharing with colleagues!
(this version of the study is only for academics / people with formal training in these fields)
A question for folks in (and interested in) psychology:
We're planning a big survey on academic psychologists (and related fields) to understand their perceptions of the reliability of their own field and the impact (or lack thereof) of the replication crises (where many major findings have failed to replicate).
What question would you want to know academic psychologists answer to?
To sign up to receive the survey (and hear the results when it's done) just go here:
My “five rules for good research” that I aim to use when I run studies:
(1) don’t use a net with big holes to catch a small fish
That means you should use a large enough sample size (e.g., number of study participants) to reliably detect whatever effects you’re looking for!
🧵[thread]
We just launched an interactive learning module on "nuanced thinking", which teaches what are, in my view, some of the most important critical thinking skills (that, unfortunately, are very rarely taught in schools):
https://programs.clearerthinking.org/nuanced_thinking_techniques.html
People support behavioral nudges less for court appearances because they believe that missing court is intentional, according to a 2020 paper in Science.
For our Transparent Replications project, we ran a replication study to see if the findings hold up:
https://replications.clearerthinking.org/replication-2020science370-6517/
