No comment

At the risk of beating the issue to death, I offer yet another post on the question, “why don’t scientists comment on scientific articles?” Previous reflections stood within the larger context of scientific impact and article-level metrics, and I’ve also attempted some superficial analysis of commenting behavior at PLoS, BMJ, and BMC. More recently (and this is why the topic is on my mind again), a room full of bright minds at the PLoS Forum (including Cameron Neylon and Jon Eisen) scratched their heads over it and came up with pretty much the same conclusion as everyone else who’s ever thought about the problem — the costs simply outweigh the benefits.

The costs, in principle, are minimal. You might need to register for an account at the journal website and be logged on, but then all that’s needed is little more than what most of us already do multiple times a day with our email — type into a box and click “submit”. (In practice, there may be nonsensical, hidden costs that make you wonder what the folks at those journals were smoking.) So the perception that the cost-benefit equation doesn’t work speaks more to the lack of benefit than anything else.

Photo by jamesclay on flickr

In academia, this translates to “commenting on scientific articles does not advance your career.” In fact, commenting might even have a negative impact if it’s perceived as a frivolous activity (as most online activities are perceived, though this is starting to change as online reference managers and other tools become more popular). Every minute not spent doing research or applying for grants is a minute wasted, despite the fact that all but the most superhuman of us — and we all know one of those — have to sleep, eat, and use the can.

This doesn’t mean that no one comments on scientific articles. They just don’t do it online. Instead, they might spend months slogging through a convoluted review process similar to that for full-fledged papers so that their comments are published and citable in a “traditional” form (granted, some of these “comments” are barely indistinguishable from full-fledged papers). Never mind that by the time the comment is published, the discussion may no longer be relevant. Never mind that only a narrow slice of the possible spectrum of comments is represented.

People prefer this type of commenting to the relative ease and immediacy of online commenting, and it isn’t because of any real utility. Traditionally published comments are widely accepted as contributions to the scientific record, and so might be considered part of someone’s academic record as well. That little distinction is the only reason I can fathom people suffering through even a shadow of what this individual suffered through. The perceived benefit is high enough to offset what is arguably a very high cost.

And that benefit is conspicuously absent from online commenting.

Photo by the-majestic-fool on flickr

Does that have to be the case? No, not necessarily. But that’s how it is right now, and it will remain that way as long as hiring committees, tenure committees and funding agencies rely on traditional measures of academic success and contribution. We could ask courageous, aspiring researchers to embrace online commenting merely on principle, for “openness” and “betterment of science”, but, frankly, they shouldn’t have to.

Naturally, the discussion at the PLoS Forum centered around how to make online commenting more attractive, with the assumption being that providing incentives would shift the cost-benefit equation. But the main incentive suggested — a “karma” type system reminiscent of StackOverflow — still begs the question, “who cares?” Scientists aren’t going to comment simply because they get imaginary points for it — the points would have to translate to real-world recognition.

With all of this in mind, I was struck by the description of PLoS Currents, a new offering from PLoS that is part journal, part blog. The idea is to provide a venue for “fast-breaking” research that simply can’t be delayed by the traditional peer review process. Articles are composed in Google Knol with a blog editor-style interface; after light review by an expert moderator (with typically around 24 hours turnaround time), the articles are posted on the PLoS Currents website. The finished articles are barely distinguishable from articles that might appear in a peer-reviewed journal, with familiar document structure and detailed figures and tables. And, importantly, the articles are indexed on PubMed — making them archived and citable. The costs, clearly, are lower than traditional commenting, but the benefits are arguably similar.

Photo by ubermoogle on flickr

Is this the transitional form that will successfully bring science towards more flexible modes of publishing and professional advancement? I don’t know, but I like the idea. PLoS Currents might just satisfy the ingrained compulsion to make distinct, cohesive contributions to the scientific record but has none of the arbitrariness and convolution of commenting through lengthy peer review.

Only time (and perhaps some proactive efforts from PLoS) will tell. They’ve only launched one Current so far — Influenza, spurred by the H1N1 flu crisis — but it’s not hard to see how the concept could be extended to other topic areas. PLoS has also been fairly reserved in its advertising of Currents given its experimental nature, but I’d love to see what happens when they throw more weight behind it. Maybe, just maybe, it will be contender.