This is a blog about general problems in Psychology (not always directly related to problems we are discussing, but perhaps interesting to have a look at when you have a spare couple of minutes): http://fixingpsychology.blogspot.nl/
In this blog we review the major themes in the discussion of Thursday 25/9/2014.
Why (not) replicate?
A true finding should be replicable.
Replication is no QRP’s detector: if you want to you can make a finding happen. However, without QRP’s (and fraud) replication is meaningful.
How can we stimulate replication:
Reward system: give extra point to researchers that replicate findings.
Punish system: punish studies that cannot be replicated. For example, by linking the original study to the failed replication (there was no consensus about this point).
Change the research policies: replicate your study a couple of times before publication to be sure that you publish a meaningful finding.
Let students do a replication in their internship instead of (mostly failing) new research.
It is impossible to replicate all studies. Editors could determine which study needs to be replicated and researchers need to focus their replications on:
Studies on which policies are based.
High impact studies.
Lastly we discussed some changes that would benefit science and psychology:
Researchers need to change their mindsets: they do research in isolation. We need building blocks that combine expertise and knowledge.
Theory can function as a boundary in maintaining quality within science. However, there are too many useless theories: every finding can be explained by a different theory. We need better theories, for example theory based on mathematical models.
Supervisors and teachers need training in doing research the right way: preventing QRP’s.
Every researcher need to use Bayesian statistics (of course…).
Studies need to be pre-registered.
Prevent testing small samples because they will always bias the results (unless you pre register).
Every department needs a methodologist. UvA is planning to set up a ‘methodenleer winkel’ for the staff to get methodological advice.
Attitude change: we won’t discover large effects anymore, researchers will from now on probably discover only small effects.
Combine research effort: results are no personal baby.
Finally an optimistic note: We are moving in the right direction, in the sense of creating a lot of awareness and good initiatives to improve replication and replicability in psychology.