These are exciting times for experimental political science. The community of experimental researchers is growing quickly, and these scholars are applying increasingly sophisticated techniques to the study of an ever-widening range of substantive problems. I'm excited, and feel honored, to have been chosen as the new editor of JEPS during this phase of the field's evolution.
I'm immensely grateful to Rebecca Morton and Joshua Tucker, the founding editors of JEPS, not only for putting in place such a well-functioning institution, but also for their helpfulness to me personally during the transition process. Learning how to be a journal editor isn't easy – and I'm sure I'll make my share of mistakes, for which I must beg everyone's forgiveness in advance – but without their openness, their generosity with their time, and their frank advice, it would have been far more difficult than it was. Their insights and their hard work have created something of incredible value to political science, and I'll do my best over the next four years to maintain and hopefully to extend their contribution.
In many respects, JEPS will look, and function, the same under my editorship as it did under theirs. The journal continues to welcome submissions from across the substantive and methodological universes that experimental political science encompasses. It will construe “political science” broadly, and it will hope to be a place where interdisciplinary connections can be forged. JEPS will continue to be friendly to null results, and will value well-crafted experiments on theoretically motivated questions regardless of what the findings may have been. The “strong Associate Editor” model remains in place, and the journal continues to aim for a relatively high desk reject rate, to save authors time and overburdened referees unnecessary effort.
But it's also true that times are changing, and JEPS will change with the times. As the experimental social sciences continue to mature, it's become clear that academic journals must take new steps to ensure the reliability of published research. As we all know well by now, in fields like social psychology, many well-known studies have failed to replicate. Improving the reliability of published research is a complex problem, because the potential sources of unreliability are many in number. Journal editors and referees need to be more aware of publication bias, and accordingly prioritize quality research design over the specific nature of experimental results. Null results that flow from well-designed, sufficiently powered, theoretically motivated studies should be welcomed, as they always have been in this journal. Authors need to be more transparent about their research process, and the distinction between ex ante hypothesis testing and ex post exploratory analyses, to avoid “harking” (hypothesizing after the results are known) – and editors and referees must be sophisticated readers of manuscripts, who value such transparency and who recognize that exploratory analyses have their proper place. And, of course, authors and journals need to be more diligent in archiving data in a way that is transparent and easily accessible to other researchers.
JEPS occupies a special place in the discipline, and it should play a vigorous role in advancing social scientific research standards. Over the coming years, JEPS will continually reexamine its submissions, review, and archiving procedures in an attempt to address publication bias, promote innovative new research, encourage replication of influential existing studies, and increase the transparency with which research is carried out and disseminated. Nothing less is required to ensure the reliability of the research that it publishes, and to keep pace with a rapidly changing field. Over the course of 2016, the journal will focus on several key structural changes as a first step in what will doubtless be an unending quest for improvement.
First, to address publication bias, and to incentivize innovative research, JEPS will introduce a new “pre-acceptance: original research” submission track later in 2016. Under this submission track, scholars can choose to submit a fully-specified research design, power analysis, and detailed pre-analysis plan, before carrying out the experiment. Such submissions will be peer-reviewed based on the quality of the research design and the pre-analysis plan, the extent to which the proposed study is innovative or substantively important, the perceived feasibility of the study, and other factors. Successful submissions will be “accepted in principle” – that is, JEPS will commit to publishing the subsequent completed study, regardless of the ultimate results, so long as the research was carried out according to the specifications of the submission, and professionally written up and analyzed (and within a timeframe agreed upon in advance). The instructions for authors and instructions for reviewers corresponding to this submission track are currently being drafted. This new submission track addresses publication bias by committing to publish a study before the data even exists. I also very much hope that this new track will help stimulate the production of innovative, important research that might not otherwise have taken place. By guaranteeing publication at the design stage, the journal may help shift the risk calculations of authors who have novel ideas that are exciting and important, but who also face career concerns that sometimes seem to incentivize “safe” research. Knowing that a good idea will be published makes it that much easier to go ahead and invest the blood, tears, and money it takes to run a study.
To be clear, JEPS will also continue to maintain the traditional “completed manuscript” submission track alongside this new “pre-acceptance” track. The proportions of the journal devoted to these separate tracks will depend squarely on the mix of submissions it receives.
Second, JEPS will at the same time open a further new “pre-acceptance: replications” submission track, to incentivize replications of important results in the field. Under this submission track, scholars would submit a fully-specified replication plan, and detailed pre-analysis plan, before carrying out the replication. Such a submission would then be peer-reviewed, based on the quality of the replication design and the pre-analysis plan, the extent to which the prior research has been highly-cited or otherwise influential, and other factors. As above, this track could have a number of desirable effects, including incentivizing replications that might not otherwise take place, evangelizing for the importance of replication in social science, and increasing the perceived status of replication studies in the discipline. Replication articles will be short (significantly shorter than typical JEPS articles), therefore displacing little original content from the journal.
Third, JEPS will undertake a series of changes that heighten the salience of data transparency and accessibility. Among these changes, the journal is raising the standards for the completeness and ease of use of authors’ research design and data replication packages. The new standards will be clarified in a revised set of instructions to authors that will be posted soon. JEPS will also require authors to submit replication datasets earlier in the process; rather than requesting data only after an article has been conditionally accepted, authors will need to submit it earlier (ultimately, once the reform has been completely phased-in, at the R&R stage when revisions are resubmitted). The journal is also working on launching an improved, permanent online replication archive later this year.
Thanks to Profs. Morton and Tucker, JEPS has gotten off to a terrific start. Through the above changes to the journal's structure, and other modifications that will be announced in the months and years to come, my goal is to keep that great start going and to help keep JEPS at the forefront in an ever-changing world.
