Nature Outlook: Assessing Science

Society benefits enormously from scientific research. We get new technologies, live longer and healthier lives, and gain deeper knowledge of our planet and the Universe. The issue of how to evaluate the fruits of research confronts scientists and policy-makers all over the world. Every country has its own set of circumstances surrounding its research infrastructure, wealth, and economic, environmental and developmental objectives — so there is no universal solution.

Earlier this year, at a symposium organized by Nature in Melbourne, Australia, a group of leading academics, funders and government advisers discussed how research outcomes are measured (see page S57). This Outlook supplement was influenced by these debates, although we at Nature take sole responsibility for its content.

As discussed at the symposium, both Australia and New Zealand have research assessment programmes that place heavy emphasis on research excellence (S52) — a qualitative determination that is heavily informed by quantitative metrics concerning, for instance, how often a paper is cited (S64). Both Australia (S67) and New Zealand (S82) have seen their global scientific standings rise in recent years — attributable at least in part to their assessment systems, even though Australia’s system offers little financial reward (S81).

Measuring research using academic yardsticks largely ignores the wider impacts of research such as new policies or improved technologies. Academics and policymakers in both countries are considering the benefits and difficulties of trying to measure such impact (S72). Could the creation of ‘citation equivalents’ enable comparison of non-academic work against peer-reviewed literature (S77)?

We hope that the intense focus on these issues in Australia and New Zealand will inform and stimulate this crucial debate throughout the scientific world.