Power to the Protocols!
Sharing Scientific Methods to Accelerate Science
By Fiona Griffin & Jonah Cool
How do we support the science and technology that will make it possible to cure, prevent, or manage all diseases by the end of this century? One way is through open science:
We believe the velocity of science and pace of discovery increase as scientists build on each others’ discoveries by sharing results, software, experimental methods, and biological resources as early as possible.
In this light, accelerating science is a factor not simply of landmark results — but rapid sharing of the scientific process, and building communities comfortable with rapidly and openly iterating.
CZI’s first Request For Applications centered on single cell biology, an emerging field that labs around the world are developing and implementing. But the focus was not strictly on new technologies or findings; many labs focused on the critical need to benchmark methods and help the community develop a better sense of why one method may be ideally suited for a particular application. At its core, this grant program was about process — the fine details of which may not land on the cover of a major journal, but will underpin many subsequent discoveries. That’s why we’re working to enable rapid sharing, awareness, and understanding of the progress that the 38 projects made in the last year.
We evaluate scientists, research projects, and programs by the new knowledge that they produce, emphasizing the quality, robustness, and rigor of that work. We want to track the underpinnings and process and ensure that they are shared with the community. We have come to appreciate that ensuring accessibility of all of the steps along the scientific journey is as valuable as the final publication.
We require our grantees to contribute to open science in several ways, including:
- Depositing software code to an open repository on sites such as GitHub;
- Submitting results to open-access preprint servers like bioRxiv upon submission to a peer-reviewed journal, if not earlier;
- Making experimental protocols openly accessible.
Taken together, these measures help us understand how our grantees are making progress and contributing knowledge that will move the whole scientific community forward.
Shortly after the launch of our first grant program, it became evident to us that the community was going to generate many valuable protocols, and that the process of iteration was of interest to many other scientists. Historically, researchers either share protocols via very short portions of a paper or external documents and private communications.
Here is an example of a protocol on preparing agar blocks that I (Jonah) helped develop. This method works okay, but the truth is that protocols are not flat objects that any researcher can immediately pull off of the shelf. They are dynamic and need to be adapted to a particular application. The bottom line is that my protocol is fine for me, but in its current form, it can’t be easily improved upon by multiple labs, versioned, commented on or run by another researcher.
These challenges helped us identify a new technology platform for developing and sharing protocols. Protocols.io is an open access resource that allows researchers to discover and share up-to-date science methods, similar to the way code can be shared on GitHub. A scientist can share, edit, fork (copy and adopt/modify), or develop protocols directly in the platform. Although we don’t require researchers to use Protocols.io, we have begun to support our grantees submitting their protocols to the platform because it makes it easy for others to immediately see, comment, fork or use the protocols.
Is There a Reproducibility Crisis in Science?
Nature asked 1,576 scientists for their thoughts on reproducibility. Most agree that there's a 'crisis' and over 70%…
Supporting the Human Cell Atlas Community on Protocols.io
CZI is supporting protocols.io to help assemble open protocols from online communities, including the global researchers who are working to create the Human Cell Atlas (HCA) — a new reference map of all cell types in the human body. This protocol community includes CZI grantees, but also many other labs around the world that are eager to learn about the diversity of methods.
CZI’s early support of the HCA includes 85 projects to create collaborative computational tools in support of building the HCA, and 38 pilot projects to help establish best practices and technologies.
The HCA protocols.io group of investigators has over 240 members, including some that are funded by CZI. We’re excited about how this community has rallied around open sharing, with a demonstrable appetite for using open science resources like protocols.io:
In just over a year, 79 openly accessible protocols were created, with over 27,000 views by more than 10,000 visitors.
One example is a project by awardee Steve Henikoff, a member of the Division of Basic Sciences at Fred Hutchinson Cancer Research Center and an investigator with the Howard Hughes Medical Institute. His lab automated an emerging method they previously developed that measures the residency of proteins on DNA. This is an important technique for understanding gene regulation in single cells.
The Henikoff lab made the first submission to the HCA protocols.io group in February 2018. It has since been viewed over 7,400 times and forked 26 times, with 58 bookmarks and 107 comments, suggesting widespread interest and continued development.
In November 2018, the Henikoff lab contributed their unpublished automated protocol to the community, which was viewed 600 times in the first month. This data speaks to the importance of sharing these methods openly, the positive reception from the community, and the importance of tracking this information to help us understand which methods are primed for broad adoption.
Experimental protocols aren’t the only things you can share — platforms like protocols.io give researchers the unique capability of sharing their analysis techniques as well. The Regev lab shared a collection of protocols together with an analysis pipeline developed at the Broad Institute, providing easy-to-follow instructions in protocol format. Sharing computational methods, as well as experimental methods, will allow labs all over the world to get up and running using new tools and technologies.
What’s more, the wider scientific community is using this platform, and other funders, such as the National Institutes of Health, are starting to encourage deposition. One thing that we’ve learned from our first grant program is the need for better ways of capturing and sharing the process of science. It has been amazing to see the community rallying around protocol sharing as one example. The prospect of additional funders and communities converging on solutions promises to generate a wealth of information that is accessible and easily adaptable.
As our communities of investigators grow, so, too, will our presence on protocols.io. CZI funding for protocols.io will also support collective protocol resources for new communities of CZI-funded investigators, including grantees who will work to find innovative solutions to neurodegenerative diseases as part of CZI’s Neurodegeneration Challenge Network. We’re excited to see where these projects lead, and for these grantees to start sharing their work and progress.
Jonah Cool, Science Officer, Chan Zuckerberg Initiative
Jonah is a cell biologist and geneticist by training, and is currently a Science Officer at the Chan Zuckerberg Initiative, where he leads the organization’s efforts to support the international Human Cell Atlas consortium. He was an American Heart Association fellow while completing his PhD at Duke Medical Center, with a focus on the role of vascularization during cell differentiation and organ morphogenesis, and was subsequently a Ruth Kirchstein Fellow at the Salk Institute studying nuclear organization during stem cell differentiation. Dr. Cool previously worked in intellectual property litigation, as well as ran an industry research group working toward therapeutic application of 3D bioprinted human tissue. He has a deep love of cell biology and, in particular, the origins of cellular heterogeneity and how diverse cells assemble into complex tissues.
Fiona Griffin, Program Associate, Chan Zuckerberg Initiative
Fiona has a background in neuroscience and project management, and is now working at the Chan Zuckerberg Initiative supporting its programs on the Human Cell Atlas. She previously worked on imaging systems to understand models of the cortex in vivo, as well as the neurobiology of drug and food addiction. Fiona is excited about driving science and biomedicine forward — understanding that communication and collaboration across researchers and fields is critical to that success. She wants to open the door to scientific research for diverse populations, and is excited to push these aims through new technology.