14 Responses to Science protocols – Recipe for success?

  1. coturnix says:

    Well, JoVE is trying to do something like that, showing videos of how stuff is done. Like a Food Channel for science.

  2. shwu says:

    That does look really useful! It’s great when people ask questions and are able to get answers (from the “authors” or others), too. I suppose it can get awkward for people to offer suggestions but it would be helpful if, say, things don’t work quite the same at higher altitudes, or if someone found that some slight tweak led to significantly better results – or even just to say “make sure you do this part exactly how they say – or else everything fails!” Some words of wisdom from others who have been there and done that to reinforce the best protocol.

  3. Crowdish says:

    “what is it about science that keeps it from exploiting and embracing the web the way practically everything else has done?”

    While I am not overly familiar with the scientific community, could it be simply that knowledge is power and that many scientists believe that by keeping the sauce secret they will remain separate from the rest of us with opposable thumbs?

    Look at some of the effects of the Internet: 10 years ago, people paid $275 commission for a stock trade, now they pay $10. Brokers lost their jobs. Look at the real estate industry today, RE agents try to maintain as much control over the transaction as possible, keeping that 6% commission intact (even though we are in the midst of a massive downturn in real estate values).

    Knowledge is power…and a job? Not sure, just my $.02 on the crowdsourcing issues.

  4. Lorrie says:

    At OpenWetware we do get interesting comments and discussion on some of our protocols, for example this one: http://openwetware.org/wiki/
    Beta-Galactosidase_Assay_A_better_Miller.

    We eventually hope to build our protocols repository into what you envision: AllRecipes for research protocols. We’re also thinking about how to offer helpful feedback, ratings, and search tools for each protocol. I’d be interested in hearing your ideas on what “AllProtocols” would look like on OpenWetWare, and how we can make our current collection better and more useful.

  5. Cameron Neylon says:

    Shirley, love the analogy (and I like cooking so that’s good as well)

    crowdish – I think its rather the reverse, there is no money (or glory) in it for scientists so they can’t be bothered. To me the solution is to tip that on its head – if we can’t get the reward culture to change (and it is but slowly) then we just have to pay for those contributions.

    I would agree with Lorrie that OpenWetWare is a good place to build such a thing, or perhaps rather OpenWetWare would be a good platform on which to build such a thing. Think of the Wiki as the back end document store with a WebApp built on top for the ranking etc.

    Another thing which would be good here is something that Kulius Lucks suggested when he and I met with the people from PLoS Comp Biol in Cambridge. Imagine if there was an autolink process between OWW and PLoS papers so that the ‘supplmenatry methods’ was hosted on OWW. Then imagine an autolink script that checks how many different papers reference this method.

    ‘This methods features in five papers in PLoS ONE and two in PLoS Biology. It is cited in 24 other papers’

    That’s the real measure of a protocol’s success. Whether it is usedin published research. So using that as the criterion could be powerful (and doesn’t require people to do the rating – which the y usually don’t in science)

  6. Jean-Claude Bradley says:

    Just like coturnix, the first thing to come to mind was JoVE and second OpenWetWare. Describing protocols is a major use of OWW.

    I also thought of Nature Protocols, but that is not free…Nature Precedings would be an alternative.

    To my mind, the biggest hurdle to doing this properly is the significant time burden. In my field, if I go to the trouble of crunching through my lab notebook I might as well write a full paper (unless I’m using JoVE). That may not be the case in fields like molecular biology.

  7. David Crotty says:

    It is a bit of a conundrum, and if you come up with the answers, I’d be thrilled. I think one obvious reason why you don’t see that many protocol repositories is that people don’t automatically think to write up their protocols in a formal manner. The vast majority of what you read in CSH Protocols is commissioned material. Once you have a need for commissioning, you have a need for an editor, and a support staff, and thus you have costs that need to be paid rather than an open site.

    You’ve also got me thinking about the nature of social networks, and whether they’re relevant or useful in professional communities. Sure, the kids love their Facebook, and job sites like LinkedIn and SciLink serve a purpose when one is hiring/looking for work. But beyond that, is there a point? What other professions communicate with each other in this manner? More thought needed, I’ll probably write more about this on my own blog soon.

  8. shwu says:

    @Lorrie and Cameron, that example is a great one – I love that someone included a sample plot! Also the fact that the original author included some background information on the assay can be very helpful. Although the wiki format has its advantages, ideally I think you should be able to do things like collapse sections or see only the X most recent comments, maybe allow tagging of comments so you can categorize by type (is this a comment suggesting a modification, an endorsement or otherwise, asking a question, or something else – like sample data?). Without these filtering mechanisms the pages will quickly become impossible to navigate. We also have to make clear that the rating system is geared towards the protocols themselves and not the original contributors of the protocols.

    With the kind of protocol attribution Cameron describe, I think it could really gain some traction. It would be useful even if they wait until after the paper is published. And linking from the paper itself is a great way to get a baseline amount of traffic. Great idea!

    @crowdish, it feels to me that scientists fear not the general populace but other scientists. Which is ironic because science also depends on collaboration. For some reason, while we (in the royal sense) can imagine all the benefits we’d glean from data and everything else being open, we balk at making our own work open.

    @David, to me Science takes aspects from both types of networking. It can be purely professional but not necessarily goal-oriented and time-restricted the way job searching is; scientists are always looking for answers to questions, people to work with, interesting problems to solve. In this way, a scientific online collaborative community could be indefinitely sustaining, and also useful to its participants, not merely a diversion. In my opinion, science is unique among the professional areas because for the most part it thrives on openness and collaboration but has yet to realize these fully in the internet era. (Some domains are more complicated, such as the medical domain.) It is one of the few places where some type of online networking could have a real impact on the work you do.

  9. David Crotty says:

    I think the problem is (and has always been for anyone who has tried in the past to build these community centers) incentivizing people to join in. I can see the incentive for someone asking a question, looking for help, but where’s the incentive for the person answering? It’s always good to be nice to your fellow man, but it’s never going to be much of a priority, particularly in this age of extremely limited funding and job availability.

    One thing I should mention, is that CSH Protocols pays a small royalty to authors of protocols (or they can choose an open access option). It’s not something we’ve publicized much, we’re waiting to see how it goes as we send out the first year’s worth of royalties. It’s not a huge amount of money but I know that as a graduate student, I certainly could have used an extra hundred bucks here and there. I’m not sure how much of a difference this will make, but we’ll see, and it’s certainly an interesting experiment in profitsharing with the community as an alternative publishing model.

  10. bgood says:

    Since I saw it presented at the Web2.0 Expo in 2007, I’ve thought that Instructables would be a fantastic platform to build a scientific recipe book with. They’ve really got all the pieces assembled. With a little more stodgy-scientist and a little less potato-gun-builder look and feel it would be good to go. If you are in the bay area, you might just drop by, I bet they would would be interested in talking.

  11. rpg says:

    I think the problem is (and has always been for anyone who has tried in the past to build these community centers) incentivizing people to join in. I can see the incentive for someone asking a question, looking for help, but where’s the incentive for the person answering?

    One might make a similar argument about peer review. I think people do it because, consciously or otherwise, they know there’s a return on the investment.

    So. Hmm. Yes. I help other folks in the lab because I’m that sort of guy, but sub-consciously maybe it is an investment and I expect help in return, whenever I need it?

  12. Jean-Claude Bradley says:

    rpg makes a good point – anonymous peer review shouldn’t work because the incentives are quite minimal (the main one probably making sure the reviewer is cited :) and it takes time. But it does “work” since publishers can find enough people to do it for free.

  13. David Crotty says:

    As far as peer review goes, it’s often an arduous process for an editor, and one regularly goes through multiple requests before one finds someone willing to review. The most common response is “I don’t have the time right now”, which is likely to be the response of most scientists to the heavy demands of social networking.

    I’d also make the argument that there are issues with expediency for the person asking the question. Sure, you can throw your problem out there and hope someone happens along who has the answer for you, but you’re much more likely to get that answer with a more directed approach. And most people can’t sit around for weeks with their experiments on hold hoping someone will solve their problems.

  14. shwu says:

    @David, I’m not sure if this would completely solve the expediency problem, but having “supernodes” in such a network (something Cameron brought up during an earlier discussion) – i.e. people who are adept at recognizing appropriate connections and can bring the relevant parties together – might be key. Not everyone has time to monitor the network for people who can help them or who they might help, but if there are some who’s “job” it is to do the monitoring then we may very well see the network working on useful timescales.

    Of course, there is still the question of “what’s in it for them?”, but it seems that so-called supernodes often enjoy their role as a connector, and we’d need relatively very few of them, so most likely reward wouldn’t be an issue. And it’s possible that “Supernode at Big Science Community – enabled hundreds of successful collaborations” could become an acceptable bullet in a CV at some point? ;)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s