Science is currently in a crisis. Results across the board do not replicate, null results are hardly published and sloppy science to get favourable results are at an all time high. At the heart of this issue lies publication pressure the pressure for scientists to publish as many articles as possible. To solve this issue, I propose a performance interview to make a more qualitative evaluation. Furthermore, an interview could better take into account all different goals of a scientist: science, education, and societal benefit.
For a branch of mathematics, statistics has reached an extraordinary amount of applications for people with no mathematical background at all. For example, correlation went from this:
On one hand, this is an extraordinary feet of science and applications. The fact that such difficult mathematical techniques are available to use for everyone, is a great example of making scientific theoretical results (the correlation) available for everyone (SPSS). On the other hand, because it is so easy, people may not know what they actually talk about anymore. For example, Oakes (1968) asked researcher, including professors, six questions about the nature of the p-value. Only 2 out of 70 got all six right.
However, is not knowing about statistics a problem? I argue it is not, as long as there are clear guidelines available. Let’s take one of the easiest test available: the independent t-test. We’re currently at a level of statistical accessibility only knowledge about following a flow chart and basic computer skills are required to perfectly execute and interpret an independent t-test, or its non-parametric counterpart. To take the first steps: do a Komorgonov-Smirnov test (click here, here, and here). If a certain value is above 0.05, proceed with the t-test, otherwise do a Mann-Whitney test. For a t-test, click such and so, etc. As long as a researcher follows this flowchart perfectly, very little can go wrong.
So if you want to do statistics with little knowledge, a very clear analysis algorithm is required. However, here statisticians still have some work to do, as for even little more complicated tests, the whole flowchart is not perfectly clear. For example, recently I had a long debate with two fellow methodologists on how to exactly follow up a 3×2 ANOVA to test all interactions. This is quite a basic design in any clinical trial (compare three drugs before on a pre and a post time point), so these basic problems should be clear for everyone involved.
In cases where statistical knowledge is not 100% clear, as in the example above, I see only one solution. Researchers need to know what they are talking about statistics wise. Only then is it possible to truly do “the right” analysis. Again, a big role can be played by statisticians. The knowledge of practitioners of statistics like psychologists starts with (and usually ends with) the education on the bachelor’s and master’s levels. In most statistics courses, the examples are always “textbook” perfect. No complex assumptions are violated, no difficult interactions need to be tested, etc. This makes people unable to do statistics in practice, because this is what practitioners face every day.
So in conclusion, to make people handle statistics better, statisticians need to do two things: make clearer flowcharts to follow, and give education that connects better to the problems people face in practice.