Fear and Reviewing in Academia

I've got at least two things going against me. For one, most of human communication is non-verbal. Whatever I say or write in critique of a paper is always more easily misinterpreted than if I were to say the exact same words to the authors in person. Second, it's likely that inherent biases in our brains will always influence how you read and interpret a critique. Malcolm Gladwell sold a lot of books on this premise.

Within the last year I was a reviewer on a paper for a journal where technical aspects of experiments within the manuscript are the most important factor in acceptance. As a reviewer I had absolutely no problem with the technical aspects of the manuscript, but I personally think that the introduction and discussion should be completely rewritten to de-emphasize what ends up being the take home story. I wrote that I didn't think the manuscript should be accepted in this state and suggested a variety of other ways to report and analyze the data which would allow the paper to be received by a larger percentage of the relevant audience. I was essentially arguing over subjective differences between the authors and I, even though the paper was technically OK. Ultimately the paper was published without the changes. This is how the system works, and I'm OK with this outcome (again, the paper is technically OK).

I want to be able to describe the specifics of this experience in a blog post, and maybe even a manuscript because I think it highlights one major downside of the "publish if experiments are technically OK" suite of journals. I want to write a post-publication critique of this article and include my actual review. I'm motivated enough to write a paper highlighting the dangers of crystallizing subjective interpretations in the form of a manuscript that glosses over this subjectivity. All this being said, I am currently an assistant professor on the tenure track. I don't want to make enemies, even though (as anybody who knows me will attest) nothing I say is ever meant as an ad hominem attack. I can be direct and this is off-putting to some, but I do this for the sake of making the story better (It's the New Yorker in me). I think that science advances much further without in-fighting and with colloquiality. I simply want science and research to progress in an efficient way with self-corrections of confusing statements. We can disagree, but let's do this over a beer and shake hands at the end.

Since I'm currently untenured, I'm absolutely terrified at inadvertently pissing the wrong people off and therefore tanking my career (and my family's well being). There are always camps in science which disagree with one another. Some of the best examples are described in Provine's "The origins of Theoretical Population Genetics" and Hull's "Science as a process". Ultimately, I'm OK if I'm lumped into a camp in some way or another but I want this to be for strictly science reasons not personal ones. In order to get tenure in the US, I must have outside letter-writers from peer institutions (some chosen by me, some by the college). These letters will hopefully describe how I make worthwhile contributions and further research in my area of expertise. One bad letter can tank my career. It's possible that someone may read a blog post (or critique of a paper) and simply take it the wrong way. Since I'm reviewing these papers, they are definitely within my realm of expertise, and so the authors have a chance at being selected by my higher ups as letter writers. I worry that critiquing a paper I've reviewed will be looked down upon by the editor, who in many cases is within the ballpark of potential letter writers. If I critique a paper over subjective and controversial interpretations, there are others out there who may hold the same viewpoints (who aren't authors on the manuscript) that could be off put by my critique. Letters are just one aspect of tenure. What if these critiques limit the chances of me being asked to speak about my work at conferences? What if these critiques make it more difficult for me to publish my own papers or get grants due simply to psychology? Is that risk worth it even if post-publication review might make a difference or open up an important discussion?

There are a bunch of folks that describe a utopian world where post-pub review is the norm, reviewers are always named, and reviews made public. I want to live in a world where I can sign my name to reviews and comment and critique papers in blog form or in a comment box next to the article. I want to be able to write papers with an opposing viewpoint. Often times fears of this world are stated hypothetically. I'm aching for a real and open discussion about the topics I raised in my original review, I think this would hugely benefit the field. I'm terrified, at least at this point in my young career, at what happens if I become the dog that catches the car. Maybe anonymity and pseudonyms are best for some things...

 I don't know that there is a fix because of the way human brains work.

Update: For some the comment box works, for others not so much. Feel free to email me comments and I'll post (I'm pretty easy to find).

Comment from Rich Lenski (http://telliamedrevisited.wordpress.com):

+++

I think there are three issues here that I’ll try to unpack.  Issue #1 is the worry over potential repercussions for your career from the authors of the paper. That’s obviously important, but let’s set it aside and look at the other two issues.

Issue #2 is that the authors of this paper ignored your useful suggestions.  Nonetheless, the paper was accepted and published.  That’s annoying.  But from what you wrote, it seems you don’t think that particular paper is a very important one in the grand scheme of science.  So I think you can let it go with respect to #2, and focus on the interesting and important work that you yourself are doing.

Issue #3 is your broader concern that journals that require only technical correctness may be weakening or diluting the scientific literature.  In that case, if you feel strongly about it, then I suggest you look for an outlet where you could write a short editorial or perspective on this issue.  You could mention that you were involved in such a situation, but there's no need to name authors or even the journal (or you might mention several journals where this is the policy).  To illustrate what you’re talking about, you could construct a strictly hypothetical case where: a paper is technically correct but ignores some issue; a reviewer asks that issue to be explicitly noted; the authors ignore the advice; and, because the paper is technically correct, the editor gives the go-ahead and it’s published.  Given all the subtleties and complexities of real science, it will probably be easier for you to construct and explain a hypothetical case than to explain the actual case that bothers you.  Plus, notice that issue #1 has gone away!

+++