One of the more interesting "problems" in Science 2.0 is the lack of commenting on online articles. In particular some journals now allow one to post comments about papers published in the journal. As this friendfeed conversation asks:
Why people do not comment online articles? What is wrong with the online commenting system[s]? I think this is one of the central issues in Science 2.0.
Or as Carl Zimmer commented on comments appearing at PLOS One a few years back:
What I find striking, however, is how quiet it is over at PLOS One. Check out a few for yourself. My search turned up a lot of papers with no discussion attached. Many others had a few comments such as, "This is a neat paper." There's nothing like the tough criticism coming out about the new flagellum paper to be found at PLOS One.
If you read the discussion on friendfeed you'll see some good reasons given for lack of comments about papers which allow online commenting. Two of the main reasons given for failure of online commenting systems are:
- Community: One needs a community before commenting works. The main reasons for this seem to me to be that one doesn't want to "waste your time commenting" and in a community you are rewarded for your comments. Without a community your comments are just more sound and fury, to quote MacBeth. Community allows for easier dialogue which, after all, is the whole point, I think.
- Anonymity: Here the issue is that comment systems where one has to register and not be anonymous may hinder people using the commenting system. Do you really want to trash that paper by the super important person who might be on your next grant review? Do you really want to ask that dumb question you have about the paper and look the fool? Again, because there are no rewards for commenting, there is, right now, more risk in commenting than in just keeping quiet. This sort of criticism, it seems to me, is less accepted by science 2.0ers, perhaps because they are selected to already be over the barrier of being non-anonymous.
Now both of these are, in my mind, good reasons for commenting not being as vibrant as one might expect coming out of a community with the high ideals that scientists are supposed to be striving for. But one point that I never see raised in discussions of commenting about online papers is what actual percentage of papers actually merit commenting. In my more pessimistic days I would argue that many (most) papers are correct but boring (I dare not look at my own publication list in making this statement.) And what exactly do you comment about for papers that are correct but boring. "This paper solves a problem that many consider low hanging fruit. It seems to be correct, but doesn't advance the field." Repeat that for every such paper?
Something like nearly half of papers in the ISI citation database are only never cited at all. Numbers from within the domain of Physical Review are better, but still 30 percent of the publications receive one or fewer citations and 80 percent of the publications receive less than five citations. This is, of course, the result of the dread publish or perish model in academia (I like to call in the publish and perish model...you may get tenure but is it really worth selling your scientific soul?)
What does all this mean for online comment systems? I'm not sure. I do think, however, that there is something wrong with a model of academia in which one can only go long, in finance speak, on publications by citing them. There is no way, outside of the reviewing process, of saying "this paper is worthless." (A contrary point is that perhaps the "fluff" is needed for the "meat": that for science to work properly a large amount of useless stuff is necessary.) We all, of course, want to be cordial and professional, but the fact that so many papers receive so few citation indicates to me that there is a breakdown in the publication model we currently use. What would be really interesting would be to find studies of peoples "real" opinions about papers in their field. I searched the intertubes a bit for any papers on this topic but couldn't locate any. Does anyone know of any such study?
Of course this is different than the issue of commenting on papers online. Is there some setup which facilitates commenting on papers? I don't know. However, as Michael Nielsen points out at the end of the friendfeed discussion, there is a lot on topic substantive commenting about science going on...it's just not found on any of the publishers websites, but instead is more likely found on blogs (not this one, of course!) I suspect part of the reason for this is that blogs more readily are identified with communities, and the problems of anonymity are lessened by a fair minded moderator. So maybe the model of commenting on papers at a publishers website is just never destined to take off. Does this mean that some form of commenting won't become more important as a function of time? I suspect the answer to this is "no", but have yet to see a model which is truly successful (blogs, for all their worth, don't quite feel social network-y enough to really facilitate good dialogue: their choice of topics is dictated by the blogger, the commenters don't really have a way to follow up with colleagues about their discussions, etc.)
- Log in to post comments
Also, look at how few papers elicit commentaries and letters to the editor. It's not surprising that comments would be light or nonexistent for most papers. What I do see that is unique to online comments are 1) quick and simple nomenclature or methodology questions that would never fly as an actual letter to the editor; rather than getting several emails in private, an author can make a public clarification and be done with it, and 2) corrections of very minor errors that really don't rise to the dreaded "erratum" level. Trackbacks are also useful for plugging the papers directly into the blogosphere.
I think there's also the issue of permanence. A comment on a blog, while it may exist forever, is a bit more transient in that the link to the article will decrease. It feels most like an online lab meeting.
I'll (pseudonymously) comment about an article on a blog, but the idea of linking my comment to the permanent archives of an article is closer to writing a letter to the editor in that it's something that will permanently be linked to references to the article. The threshold for wanting to post is much higher than "I'm curious about X"
Perhaps the solution is for the journals to make forums that aren't linked to the articles. You could search the forums to find references to the articles, but they're not linked as closely.
Philosophize all you want about this, but in my opinion this is simply another symptom of society's growing lack of engagement as a whole. Here are some personal experiences of mine that make me think this:
1. I've been involved in several non-profit organizations/clubs for many years and have noticed it has become very difficult to get people to volunteer their time, particularly for the more mundane tasks associated with keeping an organization going.
2. Related to the first comment, I've been writing The Quantum Times for over three years now and, despite my pleas, have a devil of a time just getting articles, let alone help (with the exception of Barry Sanders and, more recently, Mark Wilde). In addition, while many other APS Unit newsletters have active Letters to the Editor sections and despite a plea directly from APS HQ, in three years I've received two (maybe three?) letters for publication.
3. I don't know about anyone else, but journals can't even find referees for my papers (and they're not wacko, out-in-left-field things). I suspect people just don't want to take the time. It's not personal (at least I hope not), but I'm not a collaborator or friend and so they can't be bothered.
4. Repeat number three with regard to collaborators. I didn't take a traditional path to get where I am so I don't have collaborators from grad school all set up. No one seems to be want to be bothered to hook up on some project. Again, likely not personal.
5. Too many people I know have never met their neighbors. Dude, I'm a loner who hates people and even I know my neighbors.
So, that's my theory. Take it or leave it - or ignore it and prove my point.
I'm going with the theory that people are afraid to make negative comments. Dave, here is a test of my theory: how about you create a website called "scihate", instead of "scirate". On scihate, people could give "hates", instead of "scites", as a way of making negative, rather than positive, comments. Potentially this could be broken further into "boring" and "wrong" categories. I bet you that no one uses it!
I did consider having thumbs up and thumbs down when I first setup scirate. Now I wish I'd done it just to see if people have the guts to click thumbs down.
Now I wish I'd done it just to see if people have the guts to click thumbs down.
Or, you could make thumbs-down the default. If a reviewer does nothing, that's equivalent to disapproval. The interactive options are "neutral" and "thumbs up." The reviewer is forced to interact just to express indifference.
I don't think people are afraid to make negative comments. Trust me. If anything, I think they're more inclined to in some cases. I seriously think it is because they can't be bothered.
Dave, your thumbs up/down idea probably would get more usage than the actual comments section. It's vaguely similar to the "Like" function on Facebook which often tends to get used more than actual commenting.
The problem is that it is possibly - even likely - that people will give a quick thumbs up or down simply based on a title, maybe the abstract. What I think we'd both like to see is more people actually reading the papers carefully and making thoughtful comments on them.
I think there's really another factor here: Most papers are not really fluff - they do contain some new fact or some new insight - but the paper format itself discourages comments and discussion.
A good paper is a self-contained piece of discussion in its own right, enforced by tradition and reviewer demands. The results and methods are laid out, and all the easy objections and qualifications - the stuff that could start an actual discussion - have been addressed already within the paper. Not infrequently are not only the obvious objections been exhaustively addressed with lots of references to earlier works, but thanks to the reviewers even some quite unlikely and off-center commentary has also often been covered in detail.
It means a good paper really doesn't have any easy points on which to hang a fruitful discussion. Remaining points of contention tend to be so technical and involved that they need a "Letter to the Editor" or a full paper to make the case. A 3-5 paragraph online discussion format can't address it in any effective way. The only thing left to say for a good paper is "Nice paper. I like the results."
A bad paper, on the other hand, is much like a bad but sincere post on an online forum. It doesn't attract any comments, because, well, what would be the point? There is again nothing much to discuss that the comment format can conveniently address. So just like a bad forum post it'll sink silently into obscurity.
If you want actual online discussions to happen, I think we really need a different format than the traditional paper as the starting point. I wonder what would happen if for instance PLoS One started "seeding" each paper discussion with a short, 4-5 paragraph length forum post-style assertion of the main point and method of the paper in question, written by the first author. With a forum post - as opposed to the paper - to respond to, it may be much easier to comment.
I've pondered the complete lack of commenting, or even of using the thumbs up / thumbs down I provide for each blog post - less than 5% using the one-click, and of the only three people to leave comments, two are fellow blog initiators from my department.
One interesting event arose after I blogged about a paper I liked, but still felt had a loose end that they didn't do justice to. I sent the link to the blog post to the corresponding author, hoping to get some discussion stimulated. I got a response - by private email. And it would have made a great discussion but I respect the authors choice to not make it a public one.
At least in my discipline, good papers are rarely "perfect" - there are always assumptions that could be debated. Choices that were made about why to analyze something one way vs. another - these often do not get into the paper, but could be very illuminating discussion, especially for scientists-in-training.
I think the community idea is critical, actually. I like it because it provides a testable hypothesis!
If we're professionals, we communicate. In all directions. Period.
You might want to check out the lates xkcd comic about papers and commenting:
http://www.phdcomics.com/comics/archive.php?comicid=1178
Have fun!
Comment #7 ftw, people simply can't be bothered. According to the NSF, there are more than 20 million scientists in the US alone. How many science blogs are there? Hundreds? Maybe a thousand plus? There's no incentive for commenting, no incentive for blogging. Time is better spent on more productive activities that further your career.
Factor in also the idea that scientists are no different than any other group of human beings. Most don't read blogs, most who do don't read the comments, most who read the comments don't post comments. Most don't read comments on newspaper or magazine articles, and of those who do, very few post comments themselves. Jakob Nielsen explains it with his 90-9-1 rule, where the vast majority of internet content comes from 1% of the community:
http://www.useit.com/alertbox/participation_inequality.html
> Time is better spent on more productive activities that
> further your career.
True, but there is some thought that engaging in scholarly a dialogue on concerning paper constitutes productive activity.
In any case, I still think that, generally speaking, people really can't be bothered (see comment #3).
Oy, that came out of my fingertips all mixed up. It should say: "there is some thought that engaging in a scholarly dialogue concerning a paper constitutes productive activity."
David Crotty writes: "According to the NSF, there are more than 20 million scientists in the US alone. How many science blogs are there? Hundreds? Maybe a thousand plus? There's no incentive for commenting, no incentive for blogging. Time is better spent on more productive activities that further your career."
Of 42 living Fields Medallists, 4 run blogs. (One of those is, admittedly, defunct, although the blogger in question is active on Wikipedia).
I can think of 3 Nobelists off the top of my head who blog. (Two of those are economists, and I know some people don't consider it a science, but whatever: I personally find all three blogs very valuable.)
Curious that the best scientists are disproportionately likely to spend their time on blogging.
The main reason for this is that there is currently no academic credit allocated to scientists who leave on-line comments on papers.
I like very much the writings and pictures and explanations in your adress so I look forward to see your next writings.