What would science look like if it were invented today

2009-06-15T17:42:15Z

141.35.200.224: /* Conclusion */

''Feel free to join in to improve this blog post-to-be for the June 2009 issue of the [http://www.euroscience.org/current-issue,29004,en.html Euroscientist]. For comments, please go [http://friendfeed.com/science-2-0/0bc939a9/feedback-request-what-would-science-look-like here].''

=What would science look like if it were invented today?=
Sure, it is hard to imagine you reading this blog post in a world which hadn't yet engaged in science but in the wake of the [http://www.youtube.com/watch?v=v_UyVmITiYQ recent presentation] of the [http://www.waveprotocol.org/ Wave protocol] addressing the question "What would email look like if it were invented today", it seems reasonable to entertain some similar ideas on reinventing science today, i.e. on designing a system that creates and structures knowledge in a way that both processes can effectively feed on and adapt to each other, making use of the most appropriate technologies at hand.

How can this challenge be tackled? Certainly, it would be helpful to have operational definitions of the terms involved, particularly science, knowledge and research. However, such endeavours are the subject of ongoing debate, so it may be better to skip this step and to design our system such that it would be operational across a broad range of definitions and conceptualizations (for instance, independent of whether or not the term "science" encompasses maths, engineering, medicine, social sciences or the humanities).

==Part I: What would knowledge creation look like if it were invented today==
===Overview===
Let us start by considering scientific knowledge creation -- or research, for short. Within the framework of existing knowledge, this requires, as a first step, the identification (and perhaps further characterization) of a gap to be bridged or closed (some methodologists prefer or even have to construct their bridges before choosing a suitable place to install them, but we shall not discuss these special cases here).

Once such a gap has been identified (we will leave a detailed consideration of this process to a later post in this series), three basic components (roughly following each other as stages of a research project) are necessary to close it:
# Planning: an idea on how to bridge or close the gap
# Realization: the means to put the idea into practice
# Verification: independent assessment of the realization.

===Farewell to paper===
In this enumeration, we have consciously left out a fourth component, very prominent in contemporary science: the communication of selected details of the approach to other members of the scientific community (usually separately for each component - as grant proposals, publications, and control experiments in related studies, respectively). The decoupling of this fourth component from the other three, however, is simply a trait inherited from the era of paper-based scientific communication, and not a technical necessity today when such information can be shared instantly (with few exceptions concerning sensitive information, e.g. patient data) within and beyond the scientific community. For our purposes, we will thus reframe the concept of putting ideas or results on paper as putting them on a wiki, a blog, a dedicated online repository or successors of these (e.g. as blips or wavelets within the proposed Wave protocol).

In this kind of framework (henceforth public research environment), individual contributions can be automatically assigned a unique identifier (revision number in wikis, henceforth contribution ID) and linked to its originator (usually the user name, henceforth contributor ID). Currently, the contributor ID is generally unique within but not across individual online platforms (this differs from the paper-based system in which contributor ID is mainly based on an author's surname plus some representation -- variable across journals -- of given names, such that a single contributor ID may be shared by different individuals whose names are identical or similar, while some individuals -- especially those with multiple initials, with non-English characters, or who changed their name after marriage -- may have more than one contributor ID). For online platforms, a number of solutions towards unique identification of contributors have been implemented (e.g. [http://openid.net/ OpenID]), including some specifically targeted at scientists (e.g. [http://www.researcherid.com/ Researcher ID]).

Each contribution ID can not only be linked to its contributor but also tagged (similar to the keywords currently accompanying manuscripts or grant proposals) and have their quality assessed (or rated, for short) by individual contributors (perhaps as a function of the overlap between the tags for their personal expertise and those of the contribution under consideration) according to a pre-defined set of evaluation criteria (e.g. appropriateness to the current stage of a given project, reliability of the information supplied, or presentation with enough context to be understood by specialists and/ or the public). Some journals already allow such ratings ([http://www.plosone.org/ PLoS ONE], for instance) but none of them currently provides aggregations of the ratings by contributor, or incentives to rate (or tag, for that matter) items on their site. In spite of this, the principle feasibility of generating and aggregating such user-defined metrics has been demonstrated on multiple online platforms, especially in non-scholarly environments (tagging: [http://www.flickr.com/ Flickr]; rating: [http://www.ebay.com/ Ebay]) and also in some scholarly ones (tagging at [http://www.citeulike.org/ CiteULike]), though [http://www.sciencemag.org/cgi/content/full/311/5762/854 possible herding effects have to be taken into account]. No working implementation exists, however, that would address the lack of incentives for scientists to engage in collaborative research assessment of this sort, but given that funding agencies have managed to coerce scientists and their institutions into all sorts of behaviour during research assessment exercises in the past and present, they should have no problems providing incentives to participate in this one which has the added benefits of being both transparent and beneficial to the scientific community as a whole (it is of note in this respect that there are [http://dx.doi.org/10.1371/journal.pbio.0050107 very few incentives in the current system] to deliver timely, fair and detailed peer reviews for grant proposals or manuscripts). One way to do this would be to include both the quality and the quantity of a specific researcher's ratings (both active and passive) into the determination of the variable portion of her research funding, perhaps with some sort of normalization by the usage frequency of the tags involved (to balance between large and small fields of inquiry, and to avoid exaggerated claims). The remaining obstacles to a wider adoption of such transparent reputation schemes based on a public research environment with unique contribution and contributor ID schemes are thus not of a technical nature, and we shall assume these features to be available for the system we are about to design.

===A public research environment in practice===
With the above remarks in mind, let us now reconsider the three stages listed above:

The '''conception of ideas''' is a process very specific to the problem at hand and to the individuals (or possibly even machines) dealing with it. Though technical assistance may be available for a subset of these cases, we will not discuss them here (but in a subsequent post) and assume instead, for simplicity, that a research project is started by being entered into the public research environment and tagged as an idea with suitable keywords. Similar to current systems of journal publication alerts, scientists (and possibly other interested parties, including dedicated robots with their own contributor ID) subscribed to specific tags or contributors (or combinations thereof) will then automatically be alerted of the existence of this new project and may add to it (e.g. comments, references, extensions, limitations, illustrations, links to suitable tools or relevant legal information or related ongoing projects or previous refutations of similar ideas, offers for collaboration or funding, suggestions for a timeline, or simply further tags, or ratings of any of these), to which the original contributor and anyone else interested may respond (with some provisions to avoid spam).

As a result of these interactions, the planning of a subset of proposed projects will have taken shape, i.e. the necessary material, financial and human resources integrated with a tentative timeline to acquire some '''preliminary data'''. Once these are available, they will be posted in the same way as everything before -- with the public research environment effectively acting as an electronic lab notebook -- and immediately integrated with the relevant information available in the system by then, such that the procedures can be adapted as needed to gather the amount and quality of data necessary to bridge the targeted knowledge gap in its most recent state.

Searchable lists sortable by tags, contributors, ratings, envisioned budget or other metadata can then be compiled automatically, and science funders (which may include dedicated funding bodies but also other organizations, companies, groups of scientists or other people, or individuals) would be able to browse (potentially even with the aid of automated or semi-automated proposal crawlers) through the available proposals meeting their criteria and to either fund them directly or to signal to other funders that they would be willing to fund a proposal in part (such a practice would particularly benefit transdisciplinary projects, which often fall through the grid in traditional research funding). No technical difficulties here, just cultural ones associated with the cherished habit of keeping ideas and results private until formal publication.

It is important to note that such a public research environment would allow for '''independent verification''' right from the start in that independent samples could be investigated in parallel by independent scientists (or even robots) following the same public protocol and posting their data in public as they arise - a situation far from being common in contemporary science, although not entirely new after successful completion of large-scale collaborative initiatives like the Human Genome Project.

===Discussion===
One of the most frequently raised arguments against public research environments concerns the perceived danger of being scooped of the information laid out under the eyes of everyone and their dog. But with an attribution system as described here, it will always be possible to point out, in public, who had posted what and when, thereby severely limiting the effectiveness of any scooping attempt. Furthermore, it is probably fair to assume that way more scientists would prefer to engage in collaboration rather than scooping, and so it is much more likely that the posting of ideas, results or analytical tools will result in constructive feedback early on that may actually enhance their research.
Indeed, once the paper-based separation of the communicative component of knowledge generation has been overcome, the incentives will shift towards relevant information immediately.

Interestingly, a public research environment would work best if the initiators of a project had a certain amount of baseline funding at their disposal to bring their research through the idea stage until the first preliminary data (when it is easier to get putative funders interested in the matter). Such baseline funding is realistic: A [http://dx.doi.org/10.1080/08989620802689821 recent study] on the cost effectiveness of the [http://www.nserc-crsng.gc.ca/ Natural Sciences and Engineering Research Council of Canada] found that the costs of the research grant peer review exceeded the costs of providing every eligible researcher with a yearly baseline grant of about CAN$ 30k. Furthermore, a possibility to invest in selected projects initiated by others is perhaps even better a form of assessment than classical behind-the-doors peer review.

Given that the rating system in our public research environment is almost certainly less expensive than classical committee-based peer review of grant proposals, the new system would represent an improvement with respect to the current one, even if neither the quality of the research, nor the speed of communicating the results were affected. But both are bound to improve in the new system, leaving more money in the research funding system that can actually be spent on reserch than this is currently the case.

===Conclusion===
A small change in the design of the research system -- switching from paper-based to web-based communication of ideas, results and verifications -- may have profound consequences: within the scientific community, the permanent communication of progress during the course of a project will shorten the feedback loops, allowing to improve or update the design of any research project on the run and to link it to other gap-closing or even maintenance work on our shared corpus of knowledge. Beyond the scientific community, a scientific cycle that is completely open will allow new ways of interaction with the media and the public: Instead of maintaining a stream of "scientists found out" broadcasts as they do today, the media could add in some of the "scientists are currently investigating - let's see how they do it" variety, and everybody and their dog could join.

===Possible tags===
Google Wave, Science, Science history, Future of Science, Peer review, research funding, impact metrics, scientific collaboration, internet protocol, open science, grants, science funding, Fantasy Science Funding

==Outlook: Part II: What would knowledge structuring look like if it were invented today?==
Having considered how the switch from paper-based to web-based forms of science communication might affect knowledge creation, the next issue in this series will deal with potential effects of the same technology shift on the structuring and maintenance of knowledge and on its use in education.

IdeaWazaWiki - User contributions [en]

What would science look like if it were invented today