Prof. David M. Schultz

How Geophysical Research Letters Suppressed Scientific Discussion

If a Comment is submitted on a published article, does the original author whose article is being commented upon entitled to publish a Reply?

This is the question that this essay raises.

On 17 October 2007, Geophysical Research Letters accepted our manuscript "Weekly precipitation cycles? Lack of evidence from United States surface stations" for publication. This article culminated over ten years of working with this dataset of 219 surface station data, before getting the encouragement of scientists in Finland to complete the work and hone the statistical analyses. In this article, we show that "neither the occurrence nor amount of precipitation signifcantly depends upon the day of the week."

In late November 2007, Geophysical Research Letters published our article.

  • Schultz, D. M., S. Mikkonen, A. Laaksonen, and M. B. Richman, 2007: Weekly precipitation cycles? Lack of evidence from United States surface stations, Geophys. Res. Lett., 34, L22815, doi:10.1029/2007GL031889. [PDF]

    In that article, we cited a paper in press by Bell et al. that identified a weekly cyclicity. That paper would be later published as:

  • Bell, T. L., D. Rosenfeld, K.-M. Kim, J.-M. Yoo, M.-I. Lee, and M. Hahnenberger (2008), Midweek increase in U. S. summer rain and storm heights suggests air pollution invigorates rainstorms, J. Geophys. Res., 113, D02209, doi:10.1029/2007JD008623. [PDF]

    That paper found that "Tropical Rainfall Measuring Mission (TRMM) satellite estimates of summertime rainfall over the southeast U.S. are found on average to be significantly higher during the middle of the work week than on weekends, attributable to a midweek intensification of afternoon storms and an increase in area with detectable rain."

    On 19 December, Tom Bell wrote to say that he had submitted a comment on our paper because we analyzed different periods than they did, so our results are not comparable to theirs. We eventually received their comments from the journal, which went well beyond discussing the differences in the time periods over which the analysis was done. From their abstract:

    We discuss the methods used by S07 [Schultz et al. 2007]. We suggest that a more focused approach - if guided by physical theory - can extract far more useful information from a dataset than generic statistical searches such as described by S07.
  • Bell and Rosenfeld (2008)

    After redoing the analysis to focus on the periods of overlap and address their issues, we submitted our response on 3 March 2008, linked made available publicly for the first time [PDF] [HTML]

    The heart of our argument is the following. We feel strongly that our analyses are superior to that Bell et al. because we did not assume a priori a seven-day cycle.

    "Rather than asking whether a seven-day cycle exists, S07 ask a simpler question: whether any day of the week differs from any other day of the week (in terms of precipitation amount or occurrence). S07's results (as well as the additional calculations provided in section 2 of this reply) provide no evidence for a statistically significant favored day of the week. Simply put, we believe that by B08 assuming a seven-day cycle in their analysis methodology likely biases their conclusions in favor of finding a seven-day cycle."
    On 26 March, we received a decision from the Editor:

    We have had your manuscript, 2008GL033738, "Reply to Bell and Rosenfeld," reviewed for both scientific content and GRL-specific criteria. Based on this evaluation, I am declining publication of your response in Geophysical Research Letters. The review of the Comments made by Bell are very positive and I am recommending that those comments be published without the response from you. Attached below are the review comments, which you may find helpful if you decide to pursue an independent avenue for their publication. I am sorry I cannot be more encouraging at this time.

    The reviews are here.

    I sent an email to the Editor for an explanation for why our Reply wasn't being accepted, yet the Comments were.

    As you can imagine, I had a tough time making the decision the way I did. In fact I sat on it for a few days before sending out the decision. Both reviewers raise some serious concerns about the response in that it fails to adequately address the issues raised. Given the evaluation, it is very difficult to make a different decision. I understand the consequences of not publishing the reply along with the comments, yet I need to enforce the publication standards of the journal. I know this issue will be debated and I hope that you can contribute to that in the future. As an editor of several journals, I hope you understand the difficult choices we have to make. I wish you the best.

    I emailed the Editor in Chief twice to question this practice of publishing Comments, but not the Reply. He didn't respond to either of my emails.

    My problems with this process were the following.

    1. This debate was between us and Bell and Rosenfeld. External reviewers were involved and decided upon the winner, rather than allowing the free exchange of ideas in the published literature.

    2. I believe that Bell and Rosenfeld should have every opportunity to comment on our paper in a scientific manner. Then, we should have our opportunity to respond in a scientifically professional manner. Let the readers see both the Comment and Reply.

    3. We used the same statistical analyses in the Reply as in the original paper. How come our statistical analyses were good enough for the initial paper, but not the Reply?

    4. The Editor in Chief did not respond to my concerns.

    Postscript: On 21 September 2010, Eos published a new policy: they would not be considering comments and replies for publication.

    Other stories of the Comment and Reply process:

  • Discussion at RealClimate.org
  • "How to Publish a Scientific Comment in 1 2 3 Easy Steps" by Rick Trebino
    Last update: 28 May 2012