I was at an academic conference recently where an researcher I respect was talking about paper reviews at a top journal much the same way. His point, if I can attempt to do it justice--was that we sometimes get so caught up in proposing ways to "polish the diamond" that sometimes we forget that it's a diamond we're polishing. Some things are good in their own right, and while yes, they are likely improvable, we as "reviewers" don't usually offer enough recognition of the submission's value, or do it only in a token way in order to excuse the criticism that follows.
The researcher used as an example a recent submission with a novel research method. The other reviewers were caught up in the minutia of the paper, and missed the forrest for the trees: the method itself was more than a worthy enough contribution to the field to be accepted, and they were arguing about significance of the results. Over the course of the next couple of review rounds, this researcher's focus shifted from reviewing the paper to defending it to the other researchers. In the end, after multiple rounds (and likely a couple of years), the vote was 2-1 to reject. At this point the Senior Editor stepped in and published the paper without further revision.
Do we do the same with submissions here?
That conversation was hit me pretty hard, and I hope it will change the way I review.
Comments
I was at an academic conference recently where an researcher I respect was talking about paper reviews at a top journal much the same way. His point, if I can attempt to do it justice--was that we sometimes get so caught up in proposing ways to "polish the diamond" that sometimes we forget that it's a diamond we're polishing. Some things are good in their own right, and while yes, they are likely improvable, we as "reviewers" don't usually offer enough recognition of the submission's value, or do it only in a token way in order to excuse the criticism that follows.
The researcher used as an example a recent submission with a novel research method. The other reviewers were caught up in the minutia of the paper, and missed the forrest for the trees: the method itself was more than a worthy enough contribution to the field to be accepted, and they were arguing about significance of the results. Over the course of the next couple of review rounds, this researcher's focus shifted from reviewing the paper to defending it to the other researchers. In the end, after multiple rounds (and likely a couple of years), the vote was 2-1 to reject. At this point the Senior Editor stepped in and published the paper without further revision.
Do we do the same with submissions here?
That conversation was hit me pretty hard, and I hope it will change the way I review.