Open science

Importance
Neglectedness
Interventions
Organizations
Funders
Grants
External links
Related

Indie science could be a way beyond that. Right now, most scientists need to publish in top journals to even have a shot at a scientific career. If funding can be decoupled from publications (through mechanisms such as crowdfunding or basic income), then it may be possible for people to focus completely on science without wasting hours of time writing grants or teaching. Also, they could publish online through living documents (and since their career isn’t at stake from being open, they can afford to be as open as possible).

Open science means less “reinventing the wheel”.

Open science can include promoting new ways of producing, sharing, reviewing, and evaluating scientific research.

Importance

In 2016, GiveWell wrote that:¹ > Academics seek to publish in prestigious journals; academics largely assess each other (for purposes of awarding tenure among other things) by their records of publishing in prestigious journals. Yet the traditional system is problematic in many ways:

Journals usually charge fees for access to publications; an alternative publication system could include universal open access to academic research.

Journals use a time-consuming peer-review process that doesn’t necessarily ensure that a paper is reliable or error-free.

Journals often fail to encourage or facilitate optimal sharing of data and code (as well as preregistration), and the journal system gives authors little reason to go out of their way to share.

Journals often have conventions that run counter to the goal of producing as much social value as possible. They may favor “newsworthy” results, leading to publication bias; they may favor publishing novel analysis over replications, reanalyses and debates; they may have arbitrary length requirements that limit the amount of detail that can be included; they may have other informal preferences that discourage certain forms of investigation, even when those investigations would be highly valuable. This is particularly problematic because considerations about “what a top journal might publish” appears to drive much of the incentive structure for researchers.

From the improving science page:

If one thinks that crunch x-risks (such as resource depletion or technological arrest) are more likely than technological x-risks, then improving science as a whole is probably net positive. But x-risk researchers generally put a lot of probability mass on technological x-risks.² So improving the speed of scientific research as a whole is probably net negative from a differential progress perspective as some scientific development could bring dangerous technologies before our social structure is able to deal with them. Although from a person-affecting view, speeding up biomedical research is more important.

Neglectedness

In 2016, GiveWell wrote that:³

Some for-profit organizations have gotten significant funding; on the nonprofit side, there are several foundations working on various aspects of the problem, though most are relatively new to the space. We have the sense that there is currently little funding available for groups focused on changing incentives and doing advocacy (as opposed to building tools and platforms), though we don’t have high confidence in this view.

We see less “room for more philanthropy” in the space of supporting tools and platforms than we expected, partly because of the presence of for-profit organizations, some of which have substantial funding.

We see more such room in the space of “advocacy and incentives” than we expected, as most of the organizations in that category seem to have relatively little in terms of funding.

Interventions

In 2016, GiveWell wrote about possible interventions.⁴

Altmetrics

Metrics for evaluating the use/influence/importance of research that go beyond the traditional measures of “where a paper is published and how many citations it has.

Open access publishing

Innovative open access publishing, including preprints – Models that facilitate sharing research publicly rather than simply publishing it in closed journals, sometimes prior to any peer review occurring.

Sharing data and code

Projects that encourage researchers to share more information about their research, by providing tools to make sharing easier or by creating incentives to share.

Reproducibility

Projects that focus on assessing and improving the reproducibility of research, something that the traditional journal system has only very limited mechanisms to address.

Attribution

Tools allowing researchers to cite each others’ work in nontraditional ways, thus encouraging nontraditional practices (such as data-sharing).

Alternative publication and peer review models

Providing novel ways for researchers to disseminate their research processes and findings and have them reviewed (pre-publication).

Social networks

Platforms encouraging researchers to connect with each other, and in the process to share their research in nontraditional forums.

Organizations

The following organizations were retrieved from GiveWell’s Google Sheet Open science field on 2019-04-04.

Altmetrics

ImpactStory
Description: Open-source provider of altmetrics for individual researchers
Organizational form: Nonprofit
Budget: $625k in Sloan grants over a few years
Staff: Unclear; 2 cofounders on website
Current Funders: Sloan Foundation
Sources: http://impactstory.org/faq ; https://twitter.com/ImpactStory/status/344527516599472128

Plum Metrics
Description: Altmetric provider, typically for institutions
Organizational form: For-profit
Budget: Info not available
Staff: Unclear; 2 people on website
Current Funders: Looked and couldn’t find
Sources: http://www.plumanalytics.com/about.html

Altmetric.com
Description: Almetric provider for both individuals and institutions
Organizational form: For-profit
Budget: Info not available
Staff: 6
Current Funders: Digital Science
Sources: http://altmetric.com/about.php

PLoS – Article Level Metrics
Description: Less “alternative” and more “article level”, open source software for publishers to use for data on downloads, access, etc
Organizational form: Nonprofit project
Budget: Looked and couldn’t find
Staff: Unclear; 1 technical lead
Current Funders: PLoS
Sources: http://blogs.plos.org/mfenner/about/

Post-publication peer review

Faculty of 1,000 - F1000 Prime
Description: Paid service with ~10,000 reviewers who participate to rate articles after publication
Organizational form: For-profit project
Budget: Info not available
Staff: Looked and couldn’t find
Current Funders: F1000
Sources: http://www.givewell.org/files/conversations/Jane%20Hunter%20and%20Rebecca%20Lawrence%20(public).pdf

Journal Lab
Description: Web interface to allow people to comment on papers, vote up comments, etc.
Organizational form: For-profit
Budget: Seeking funding
Staff: Unclear; 2 cofounders
Current Funders: Seeking funding
Sources: http://www.givewell.org/files/conversations/David%20Jay%2003-14-13%20(public).pdf

Hypothes.is
Description: Annotation and conversation addon for the entire web
Organizational form: Nonprofit
Budget: ~$2m budget for 2013
Staff: 6
Current Funders: Sloan Foundation, Mellon Foundation, others
Sources: http://hypothes.is/who/ ; http://www.givewell.org/files/conversations/Dan%20Whaley%2005-08-13%20(public).pdf

Numerous other platforms
Description: https://docs.google.com/document/d/1HD-BEaVeDdFjjCNFkb0j3pvwe7MrP3PtE-bWHkkdq7Q/edit#
Organizational form: Varies
Budget: Varies
Staff: Varies
Current Funders: Varies
Sources: document initially created by Jason Priem and shared on his Twitter feed

Innovative open access publishing, including preprints

PeerJ
Description: Startup open access journal and preprint publisher, lifetime membership model
Organizational form: For-profit
Budget: Info not available
Staff: 5
Current Funders: Looked and couldn’t find
Sources: https://peerj.com/about/

eLife
Description: New open access journal for the life sciences with no article processing charges for authors, run by HHMI and Wellcome
Organizational form: Nonprofit
Budget: Looked and couldn’t find
Staff: Unclear; 13 “executive staff”
Current Funders: HHMI, Wellcome, Max Plank Society
Sources: http://www.elifesciences.org/about/elife-community/executive-staff/

PLoS
Description: Largest open-source publisher and pioneer in the field, now considered “profitable”
Organizational form: Nonprofit
Budget: ~$20m in 2011
Staff: >130
Current Funders: PLoS
Sources: http://www.plos.org/about/people/ ; http://projects.propublica.org/nonprofits/organizations/680492065

F1000Research
Description: New open access journal that requires submission of data, publishes prior to peer review, and uses no significance filter
Organizational form: For-profit project
Budget: Info not available
Staff: Looked and couldn’t find
Current Funders: F1000
Sources: http://www.givewell.org/files/conversations/Jane%20Hunter%20and%20Rebecca%20Lawrence%20(public).pdf

PressForward
Description: “A platform for overlay journals”
Organizational form: Nonprofit project
Budget: Looked and couldn’t find
Staff: Looked and couldn’t find
Current Funders: Sloan Foundation
Sources: http://www.givewell.org/files/conversations/Josh%20Greenberg%2003-13-13%20(public).pdf

ArXiv
Description: Preprint server for physics, computer science, and some other disciplines
Organizational form: Nonprofit project
Budget: ~$800K for 2013
Staff: Looked and couldn’t find
Current Funders: Simons Foundation & Cornell
Sources: https://confluence.cornell.edu/display/culpublic/Initial+Five-Year+Support+Pledges ; https://confluence.cornell.edu/download/attachments/127116484/arXiv_CY13_budget_public.pdf

Numerous other preprint servers
Description: No exhaustive list, but there are appear to be many at relatively early stages (by comparison to ArXiv)
Organizational form: Varies
Budget: Varies
Staff: Varies
Current Funders: Varies
Sources: Varies

Figshare
Description: Repository for any data, including published article data but not exclusively; gives out DOIs so that researchers can cite
Organizational form: For-profit
Budget: Info not available
Staff: Looked and couldn’t find
Current Funders: Digital Science
Sources: http://www.givewell.org/files/conversations/Kaitlin%20Thaney%2004-10-13%20(public).pdf

Dryad
Description: Repository for data from published articles, works with publishers
Organizational form: Nonprofit
Budget: $1.3m NSF grant
Staff: 7
Current Funders: NSF
Sources: http://www.nsf.gov/awardsearch/showAward?AWD_ID=1147166 ; http://datadryad.org/pages/whoWeAre#staff

Github
Description: General service for backing up, sharing, and collaborating on code. Not specific to scholarly application
Organizational form: For-profit
Budget: $100 million in VC funding
Staff: 174
Current Funders: Andreesen Horowitz
Sources: https://github.com/about ; http://www.wired.com/wiredenterprise/2012/07/github100m/

Nature Scientific Data
Description: Data journal
Organizational form: For-profit
Budget: Info not available
Staff: Looked and couldn’t find
Current Funders: Nature Publishing Group
Sources: http://www.nature.com/press_releases/scientificdata.html

Duraspace
Description: Produces software for institutional repositories
Organizational form: Nonprofit
Budget: $1.8m in 2011
Staff: 9
Current Funders: Mellon Foundation
Sources: http://www.duraspace.org/duraspace_staff ; http://projects.propublica.org/nonprofits/organizations/260389639

Registry of Research Data Repositories
Description: re3data.org
Organizational form: Nonprofit project
Budget: Looked and couldn’t find
Staff: Looked and couldn’t find
Current Funders: German Research Foundation DFG
Sources: http://www.re3data.org/about/

Dataverse Network
Description: http://thedata.org/
Organizational form: Nonprofit project
Budget: Looked and couldn’t find
Staff: 13
Current Funders: NSF grants, Harvard
Sources: http://thedata.org/book/people

Reproducibility

Projects
Description: Data/research process management software
Organizational form: For-profit project
Budget: Info not available
Staff: Looked and couldn’t find
Current Funders: Digital Science
Sources: https://projects.ac/

Center for Open Science
Description: Assessing reproducibility of psychology studies, building infrastructure for future reproducibility studies
Organizational form: Nonprofit
Budget: ~$5m grant
Staff: Looked and couldn’t find
Current Funders: Arnold Foundation
Sources: http://www.arnoldfoundation.org/center-open-science-provide-revolutionary-approach-scientific-communication

Reproducibility Initiative
Description: Effort to assess the reproducibility of a sample of volunteered biomedical lab studies
Organizational form: Nonprofit project
Budget: Seeking funding
Staff: 0 full time
Current Funders: Seeking funding
Sources: http://www.givewell.org/files/conversations/Elizabeth%20Iorns%20conversation%2002-26-13%20(public).pdf

RunMyCode
Description: Allows public reproducibility of submitted data and code
Organizational form: Nonprofit
Budget: Looked and couldn’t find
Staff: Unclear; ~13 people on website (excluding board), but many appear to be part time
Current Funders: Sloan Foundation
Sources: http://www.runmycode.org/CompanionSite/team.do

iPython Notebook
Description: Software environment for reproducible data analysis
Organizational form: Nonprofit project
Budget: Looked and couldn’t find
Staff: Looked and couldn’t find
Current Funders: Sloan Foundation
Sources: http://www.givewell.org/fi les/conversations/Josh%20Greenberg%2003-13-13%20(public).pdf

Attribution

ORCID
Description: System to create unique identifiers for researchers so that they can be cited correctly
Organizational form: Nonprofit
Budget: >$750k in contributions
Staff: 6
Current Funders: Numerous societies and publishers
Sources: http://orcid.org/about/team ; http://orcid.org/about/community/sponsors

DataCite
Description: Infrastructure to enable the citation of data
Organizational form: Nonprofit
Budget: Looked and couldn’t find
Staff: Looked and couldn’t find
Current Funders: Member organizations
Sources: http://www.datacite.org/whatdowedo

Advocacy

AllTrials
Description: Campaign to ensure that all trials are registered and fully reported in a timely manner
Organizational form: Campaign
Budget: ~$60k Just Giving target
Staff: Looked and couldn’t find
Current Funders: Looked and couldn’t find
Sources: https://www.justgiving.com/alltrials

SPARC
Description: Main open access advocate in the US
Organizational form: Nonprofit
Budget: <<$1M/year
Staff: ~5
Current Funders: OSF
Sources: http://www.givewell.org/files/conversations/Heather%20Joseph%2005-23-13%20(public).pdf

PLoS - Cameron Neylon
Description: PLoS’ advocacy team, working to support open access
Organizational form: Nonprofit
Budget: Looked and couldn’t find
Staff: 2
Current Funders: PLoS
Sources: http://www.plos.org/about/people/staff/#advocacy ; http://www.givewell.org/files/conversations/Cameron%20Neylon%2003-26-13%20(public).pdf

Retraction Watch
Description: Blog about retractions
Organizational form: Blog
Budget: ~None
Staff: 2 volunteers
Current Funders: None
Sources: http://www.givewell.org/files/conversations/Ivan%20Oransky%20Conversation%2002-21-13%20public.pdf

SF Declaration on Research Assessment
Description: Campaign against the use of journal impact factors in research evaluation
Organizational form: Campaign
Budget: Looked and couldn’t find
Staff: Looked and couldn’t find
Current Funders: Looked and couldn’t find
Sources: http://am.ascb.org/dora/

Open Knowledge Foundation
Description: Works to open both government and scientific data
Organizational form: Nonprofit
Budget: Looked and couldn’t find
Staff: Unclear; ~40 on website but many appear to be volunteers
Current Funders: Sloan Foundation, Hewlett Foundation
Sources: http://okfn.org/about/partners-and-funders/ ; http://okfn.org/about/team/

Sage Bionetworks - John Wilbanks
Description: Policy efforts to make it easier to donate medical data
Organizational form: Nonprofit
Budget: ~$6m in 2011
Staff: ~35 total, with ~1 devoted to policy work
Current Funders: Numerous biomedical funders
Sources: http://www.givewell.org/files/conversations/John%20Wilbanks%2003-25-13%20(public).pdf ; http://projects.propublica.org/nonprofits/organizations/264489946

Changing publication models

FORCE 11
Description: Hosts Beyond the PDF conferences, devoted to the next generation of scholarly communication, community website
Organizational form: Nonprofit
Budget: ~$200k grant
Staff: ~1 staff member; ~400 community members
Current Funders: Sloan Foundation
Sources: Martone notes

myExperiment
Description: http://www.myexperiment.org/
Organizational form: Nonprofit
Budget: Looked and couldn’t find
Staff: Looked and couldn’t find
Current Funders: EU
Sources: http://www.myexperiment.org/

Scalar
Description: Platform for digital publications that are not (just) journal articles
Organizational form: Nonprofit
Budget: Looked and couldn’t find
Staff: Unclear; 6 people on website
Current Funders: Mellon Foundation
Sources: http://scalar.usc.edu/people/

Workflow4Ever
Description: http://www.wf4ever-project.org/web/guest/home
Organizational form: Nonprofit
Budget: ~4m Euro grant
Staff: Looked and couldn’t find
Current Funders: EU
Sources: http://www.wf4ever-project.org/partners

Academia.edu
Description: Social network for scientists to share papers, get metrics
Organizational form: For-profit
Budget: $6.7m in VC funding
Staff: Looked and couldn’t find
Current Funders: Spark and other VCs
Sources: http://www.crunchbase.com/company/academia-edu

ResearchGate
Description: Social network for scientists to share papers, get metrics
Organizational form: For-profit
Budget: >$35m in VC funding
Staff: Looked and couldn’t find
Current Funders: Benchmark, Founders Fund,Bill Gates, other VCs
Sources: http://www.crunchbase.com/company/researchgate

Mendeley
Description: Reference manager and social network for scientists
Organizational form: For-profit
Budget: $2m in VC funding, acquired by Elsevier for ~$100m
Staff: Looked and couldn’t find
Current Funders: Elsevier
Sources: http://www.crunchbase.com/company/mendeley

Alternative peer review model

Rubriq
Description: http://www.rubriq.com/
Organizational form: For-profit
Budget: Info not available
Staff: 13
Current Funders: Looked and couldn’t find
Sources: http://www.rubriq.com/who/our-team/

Peerage of Science
Description: A collaborative peer review model in which reviews are eligible for use by multiple jounals; journals pay for the service. Peer reviews are themselves peer reviewed Organizational form: For-profit
Budget: Info not available
Staff: 2-5
Current Funders: May be profitable
Sources: http://www.peerageofscience.org/company/our-team/ ; http://www.peerageofscience.org/faq/#faq13

Iris.AI Description: Research discovery with artificial intelligence Link: https://iris.ai/

1science Description: A comprehensive suite of products based on a curated collection of 93 million articles published in peer-reviewed journals in all fields of academia and research, in all languages, and from all over the world. Link: https://www.1science.com/

Other

Inter-university Consortium for Political and Social Research (ICPSR)
Description: Data archiving
Organizational form: Nonprofit
Budget: $10 million in grants in 2012
Staff: 36
Current Funders: NIH, Gates, NSF, NCAA, Shriver Institute, Department of Justice, RWJ Foundation, and more
Sources: http://www.icpsr.umich.edu/icpsrweb/landing.jsp

Citizen Science Association
Description: Advocacy group for Citizen Science
Organizational form: Nonprofit
Budget: N/a
Staff:
Current Funders:
Sources: http://www.citizenscienceassociation.org

Citizen Science Alliance
Description: Advocacy group for Citizen Science
Organizational form: Nonprofit
Budget: N/a
Staff:
Current Funders:
Sources: http://www.citizensciencealliance.org

Citizen Science Center
Description: Advocacy for Citizen (participatory science)
Organizational form: Profit
Budget: n/a Staff:
Current Funders:
Sources: http://www.citizensciencecenter.com

Funders

The following organizations were retrieved from GiveWell’s Google Sheet Open science field on 2019-04-04.

Digital Information Technology
Funder: Alfred P. Sloan Foundation.
Description: “the program aims to maximize the efficiency and trustedness of academic research.”
Conversation notes: http://www.givewell.org/files/conversations/Josh%20Greenberg%2003-13-13%20(public).pdf

Data-Driven Discovery
Funder: Gordon and Betty Moore Foundation
Description: see http://www.moore.org/initiative.aspx?id=4775
Budget: $60M/5 years (as of 2016)
Organizations currently funding: just getting started; no public grants at last check (2016)

Research Integrity
Funder: Laura and John Arnold Foundation
Description: “The Research Integrity initiative is aimed at improving the reliability and validity of scientific evidence across fields that inform governmental policy, philanthropic endeavors and individual decision-making. As a society, we often rely on published scientific research to guide our policy, health and lifestyle choices. In some fields, the research is rigorous and thorough. In other areas, it is weak, spurious and unreliable.”
Organizations currently funding: Center for Open Science ($5M grant)

Scholarly Communications and Information Technology
Funder: Andrew W. Mellon Foundation
Description: see http://www.mellon.org/news_publications/annual-reports-essays/annual-reports/content2011.pdf
Budget: $26M in 2011
Organizations currently funding: Hypothes.is, Scalar, many others

Life Science Entrepreneurship
Funder: Ewing Marion Kauffman Foundation
Description: “Much of the progress in the prevention, diagnosis, and treatment of disease can be attributed to medical research in university labs. The Kauffman Foundation is identifying barriers that slow or deter life science innovators—the scientists in those labs—from getting their discoveries to the health care market. The Foundation’s Life Science Entrepreneurship program includes a host of initiatives designed to help life science entrepreneurs commercialize their research to benefit patients.”

Research Tools and Science Metrics divisions
Funder: Digital Science
Description: Part of Macmillan Publishers, owner of Nature, incubator and funder for new tools
Organizations currently funding: Figshare, Altmetric.com, Projects
Conversations notes: http://www.givewell.org/files/conversations/Kaitlin%20Thaney%2004-10-13%20(public).pdf

Assorted venture capitalists
Description: Benchmark, Founders Fund, numerous others
Organizations currently funding: ResearchGate, Academia.edu

Grants

Wikimedia’s Individual Engagement Grants

External links

With respect to AI safety:

Other: