Assessing the importance of scientific work
ONE role academic journals have come to play that was not, as it were, part of their original job-description of disseminating scientific results (see article), is as indicators of a researcher’s prowess, and thus determinants of academic careers. Publication in a top-notch title such as Nature or Science is an adornment to a scientist’s CV that is unlikely to be overlooked by an appointment committee. Using such publications as endorsements is, though, necessarily a rule of thumb. A paper’s true quality is better revealed by the number of times it is cited elsewhere (ideally, in papers other than those written by the original’s authors). But citations take time to accumulate. Other, faster means of assessment would be welcome.
That has led to the development of alternative metrics, or “altmetrics”. These extend the concept of citation beyond references in other scientific papers—by recording, for example, how often a paper is downloaded, or when the outcome of a clinical trial is used to develop guidelines for doctors, or if a piece of work is included in a course curriculum.
Altmetric.com, based in London, was one of the first companies to work in this area. It has, since 2011, tracked mentions of published papers in sources ranging from social media and Wikipedia to policy documents published by government departments. A rival firm, Plum Analytics, in Philadelphia, tracks mentions, downloads, clicks and the like of everything from preprints (papers that have been made publicly available, but are not yet formally published) and sets of raw data to non-commercial computer programs which investigators have written to assist their own endeavours.
Using altmetrics should thus indicate the importance of a wider range of research-related activities than citations manage, and do so faster. Plum Analytics was bought in February by Elsevier, one of the world’s largest scientific publishers, suggesting that altmetrics may be profitable as well as useful.
Meta, based in Toronto, takes another tack. It hopes, by bending artificial intelligence to the task, to identify important papers from the 2m or so produced every year. The firm’s computers have attempted to recognise features of widely cited papers that contributed to their success. Sam Molyneux, Meta’s boss, claims that as a result the firm’s software can now predict the impact of newly published work.
Meta, too, was bought earlier this year—in its case by the Chan Zuckerberg Initiative, a company started by Mark Zuckerberg, a founder of Facebook, and his wife Priscilla Chan, that is being run as a philanthropic operation. Mr Molyneux says he hopes, within the next two months, to make Meta’s tools available without charge to any scientist who wishes to use them.
