Reasons to implement replication to the Research Masters Program.

As you may know, psychology has been in a state of crisis in the last years. Social psychology, for example has been shaken by several cases of scientific fraud and there have been other allegations of ‘bad research’. However, the problem of ‘bad research’ is a more general problem. In addition to fraudulent research; questionable research practices (QRPs) or the selective reporting of positive results have broadly distorted the interpretation of scientific data. In some cases, the distortion between what the data are and what the data are interpreted to be is so large that current research cannot be used by the private sector. For example, the pharmaceutical company Bayer reports that they are unable to use ‘scientific findings’ from universities, stating that sometimes as many as 75% of the results cannot be reproduced. This means that some of the ‘knowledge’ that universities produce is inaccurate or false, too unreliable to develop new medication.

However, psychology needs reliable results, if we want to apply psychology to gian something from it. Therefore, I think it is necessary and urgent that universities start to to implement replication into psychology via education. We are likely to need a lot of researcher that can replicate in the future, because many studies indicate that replication rates in psychology and other sciences are low (see Begley and Ellis, 2012). Psychology needs an overhaul in order to advance as a science and therefore it needs a lot of researchers who are able to adequately replicate studies. However, they will not appear out of nowhere. If we do not educate people, we risk that psychology gets a bad reputation among scientists and it may hurt the ties to the private sector. The private sector will not invest in a field that cannot produce applicable, reliable science, which will affect funding for psychological research negatively in the long run.

In order to make the process of reviewing and advancing our science easier, universities should raise a new generation of scientists that is able to increase the reliability of scientific findings by replication. This can only happen if universities provides the means to educate students who are willing and capable to do this. Replication is a corner stone of science and formal education is needed to do it.

Begley, C. G., & Ellis, L. M. (2012). Raise standards for preclinical cancer research. Nature, 483, 531–533.

How students could help to replicate data

Currently, a growing amount of researchers express doubt about the quality of scientific evidence in many research areas. There are several debates approaching solutions to this problem one of which aims to improve the quality of scientific evidence by introducing rigorous replication to research. I think a pragmatic way to implement replication into the scientific field is to teach students about research by actual replication of research. Since scientists are predominantly invested into their own research instead of replicating the work of their colleagues, we should consider alternative solutions to encourage replication.

The University of Groningen teaches methodology using a format were students have to fill out Questionnaires that resemble those used by other psychologists. For example, the RuG methodology course simulated a study based on the anchoring effect. The anchoring effect occurs when people are asked to estimate results of e.g. a math problem such as 8*7*6*5*4*3*2*1 or 1*2*3*4*5*6*7*8. People’s estimates of the results are based on, or anchored onto, the first number of the sequence in the equation. Therefore they estimated the result to be 2250 for the first equation and 512 for the latter case (the actual result is 40,320).

Later on, the data of those Questionnaires is given to students to practice statistical analysis and the writing of papers. Within small classes detailed writing instructions were given to students. Even though, students put in a lot into the writing, their work will only have an impact on grades, instead of science. But we could change that!

Instead of losing the data we could use the format of the methodology course to replicate studies. Many students may find the possibility of a co-authoring a paper desirable, especially if they want to become researchers themselves. I think it is reasonable to expect that among 250 undergraduates, at least one will produce a decent introduction, a decent method section and so forth, which could be rewritten into a good paper that withstands the peer review process. Results of these study replications could then be published in a student journal dedicated solely to replication, similarly to already existing student magazine’s such as the Honour’s Review.

I believe that we would improve undergraduate education in the long run and additionally could use methodology courses to actually contribute replications to the scientific community. What do you think? I would be happy to see a discussion of this topic.