York librarian validates social media metrics for research impact evaluation

Xuemei Li

York Business Librarian Xuemei Li is among an early cohort of researchers who are studying and validating the usefulness of altmetrics published in an academic journal. Altmetrics is the study of social media metrics used for analyzing and informing scholarship.

Li’s first study was published in April, 2012 in Scientometrics and Li’s second study was accepted by the 17th International Conference on Science and Technology Indicators taking place September 2012.

Xuemei LiXuemei Li

“Researchers are integrating various social media tools such as blogs, wikis, Twitter and social bookmarks into their research processes to save, organize, share, and disseminate various research sources. It is even more difficult for traditional bibliometric indicators to capture the totality of research influence on the web,” Li explains. “Nevertheless, the traces left by researchers and the general public through those social media tools hold big potential for measuring different research influences, and this is what altmetrics aims to measure. Altmetrics can be used to complement traditional citation-based measurements.”

Li’s first study of altmetrics sampled 1613 papers published in Nature and Science in 2007 and compared citations with reader counts. She found significant statistical correlations between citations from Web of Science (now the Web of Knowledge) and Google Scholar and reader counts from the social media bookmark tools CiteULike and Mendeley. The findings suggest that the type of scholarly influence one’s research has  – as measured by these social media tools  – is related to traditional citation-based impact.

Li’s second study compared nearly 1400 Faculty of 1000 (F1000) post-publication peer reviews and Mendeley usage data with traditional bibliometric indicators. This study suggests that F1000 – a database that stores only the best quality biomedical articles after they’ve been published, as selected by over 10,000 faculty members worldwide – is good at acknowledging the merit of an article from (the F1000) experts’ point of view while Mendeley reader counts are more closely related to citation counts.

“Faculty are striving to demonstrate the impact of their research in a world where the web has become a critical communications channel,” says Cynthia Archer, York University librarian. “Li’s ground-breaking research serves to validate the usefulness of social media based altmetrics to monitor and track faculty research impact.”

Li and other researchers are working hard to identify, monitor and evaluate potential social media tools towards building reliable altmetric indicators.

Submitted to YFile by Jocelyn Lagerquist, communications officer, York Libraries

Comments are closed.