Citation impact

Article-level

Main article: Article-level metrics

One of the most basic citation metrics is how often an article was cited in other articles, books, or other sources (such as theses). Citation rates are heavily dependent on the discipline and the number of people working in that area. For instance, many more scientists work in neuroscience than in mathematics, and neuroscientists publish more papers than mathematicians, hence neuroscience papers are much more often cited than papers in mathematics. |url-access=subscription

Most-cited papers

The most-cited paper in history is a paper by Oliver Lowry describing an assay to measure the concentration of proteins. |doi-access=free |doi-access=free

Author-level

Main article: Author-level metrics

Total citations, or average citation count per article, can be reported for an individual author or researcher. Many other measures have been proposed, beyond simple citation counts, to better quantify an individual scholar's citation impact. |doi-access=free |doi-access=free |access-date=2009-05-27 |archive-url=https://web.archive.org/web/20100624054737/http://www.cindoc.csic.es/cybermetrics/articles/v13i1p2.html |archive-date=2010-06-24

Citations are distributed highly unequally among researchers. In a study based on the Web of Science database across 118 scientific disciplines, the top 1% most-cited authors accounted for 21% of all citations. Between 2000 and 2015, the proportion of citations that went to this elite group grew from 14% to 21%. The highest concentrations of 'citation elite' researchers were in the Netherlands, the United Kingdom, Switzerland and Belgium. 70% of the authors in the Web of Science database have fewer than 5 publications, so that the most-cited authors among the 4 million included in this study constitute a tiny fraction.

Journal-level

Main article: Journal-level metrics

The simplest journal-level metric is the journal impact factor, the average number of citations that articles published by a journal in the previous two years have received in the current year, as calculated by Clarivate. Other companies report similar metrics, such as the CiteScore, based on Scopus.

However, very high journal impact factor or CiteScore are often based on a small number of very highly cited papers. For instance, most papers in Nature (impact factor 38.1, 2016) were only cited 10 or 20 times during the reference year (see figure). Journals with a lower impact (e.g. PLOS ONE, impact factor 3.1) publish many papers that are cited 0 to 5 times but few highly cited articles. |doi-access=free

Journal-level metrics are often misinterpreted as a measure for journal quality or article quality. However, the use of non-article-level metrics to determine the impact of a single article is statistically invalid. Moreover, studies of methodological quality and reliability have found that "reliability of published research works in several fields may be decreasing with increasing journal rank", contrary to widespread expectations.

Citation distribution is skewed for journals because a very small number of articles are driving the vast majority of citations; therefore, some journals have stopped publicizing their impact factor, e.g. the journals of the American Society for Microbiology. |doi-access=free|pmc=4941020 Citation counts follow mostly a lognormal distribution, except for the long tail, which is better fit by a power law.

Other journal-level metrics include the Eigenfactor, and the SCImago Journal Rank.

Criticism

The advent of citation impact metrics starting around the 1960s created incentives and pressure from their institutions for scientists to publish works in journals known for publishing highly cited papers, which in turn increased the subscription demand and prices for those journals. Technology historian Edward Tenner points out that a paper which makes an incorrect claim concerning a fundamental topic can attract a large number of citations for the purpose of debunking it; citation impact is thus not a good measure of quality or accuracy.

Altmetrics

Main article: Altmetrics

An alternative approach to measure a scholar's impact relies on usage data, such as number of downloads from publishers and analyzing citation performance, often at article level.

As early as 2004, the BMJ published the number of views for its articles, which was found to be somewhat correlated to citations. |doi-access=free

In response to growing concerns over the inappropriate use of journal impact factors in evaluating scientific outputs and scientists themselves, Université de Montréal, Imperial College London, PLOS, eLife, EMBO Journal, The Royal Society, Nature and Science proposed citation distributions metrics as alternative to impact factors.

Citation analysis

Main article: Citation analysis

An important recent development in research on citation impact is the discovery of universality, or citation impact patterns that hold across different disciplines in the sciences, social sciences, and humanities. For example, it has been shown that the number of citations received by a publication, once properly rescaled by its average across articles published in the same discipline and in the same year, follows a universal log-normal distribution that is the same in every discipline. |doi-access=free |book-title=Proceedings of the WebSci10: Extending the Frontiers of Society On-Line |access-date=2017-02-20 |archive-url=https://web.archive.org/web/20160316092504/http://journal.webscience.org/321/2/websci10_submission_107.pdf |archive-date=2016-03-16 |doi-access=free

Research suggests the impact of an article can be, partly, explained by superficial factors and not only by the scientific merits of an article. |hdl-access=free |doi-access=free |hdl-access=free

Automated citation indexing |book-title=DL'98 Digital Libraries, 3rd ACM Conference on Digital Libraries

Some researchers also propose that the journal citation rate on Wikipedia, next to the traditional citation index, "may be a good indicator of the work's impact in the field of psychology."

According to Mario Biagioli: "All metrics of scientific evaluation are bound to be abused. Goodhart's law [...] states that when a feature of the economy is picked as an indicator of the economy, then it inexorably ceases to function as that indicator because people start to game it."{{cite journal |doi-access=free

Open Access publications

Main article: Open Access

Open access publications are accessible without cost to readers, hence they would be expected to be cited more frequently.{{Cite web|last=Hitchcock|first=Steve|title=The effect of open access and downloads ('hits') on citation impact: a bibliography of studies|url=http://opcit.eprints.org/oacitation-biblio.html|access-date=2023-01-22|website=opcit.eprints.org |doi-access=free |doi-access=free |doi-access=free |doi-access=free |doi-access=free

The evidence that author-self-archived ("green") open access articles are cited more than non open access articles is somewhat stronger than the evidence that ("gold") open access journals are cited more than non open access journals. Two reasons for this are that many of the top-cited journals today are still only hybrid open access (author has the option to pay for gold) and many pure author-pays open access journals today are either of low quality or downright fraudulent "predatory journals," preying on authors' eagerness to publish-or-perish, thereby lowering the average citation counts of open access journals.Björk, B. C., Kanto-Karvonen, S., & Harviainen, J. T. (2020). How frequently are articles in predatory open access journals cited. Publications, 8(2), 17.

References

References

1. Haustein, S.. (2012). "Multidimensional Journal Evaluation: Analyzing Scientific Periodicals beyond the Impact Factor". De Gruyter.
2. (2007). "Citation frequency: A biased measure of research impact significantly influenced by the geographical origin of research articles". Scientometrics.
3. Leydesdorff, L., & Milojević, S. (2012). Scientometrics. arXiv preprint arXiv:1208.4566.
4. Harnad, S. (2009). Open access scientometrics and the UK Research Assessment Exercise. Scientometrics, 79(1), 147-156.
5. Garfield, Eugene. (1972-11-03). "Citation Analysis as a Tool in Journal Evaluation". American Association for the Advancement of Science (AAAS).
6. Gálvez RH. (March 2017). "Assessing author self-citation as a mechanism of relevant knowledge diffusion". Scientometrics.
7. Reardon, Sara. (2021-03-01). "'Elite' researchers dominate citation space". Nature.
8. (2018). "Prestigious Science Journals Struggle to Reach Even Average Reliability". Frontiers in Human Neuroscience.
9. (2022-04-03). "Requiem for impact factors and high publication charges". Accountability in Research.
10. (2016-01-11). "Universality of Citation Distributions for Academic Institutions and Journals". Public Library of Science (PLoS).
11. Edward Tenner. (2018). "The Efficiency Paradox: What Big Data Can't Do". Knopf.
12. Veronique Kiermer. (2016). "Measuring Up: Impact Factors Do Not Reflect Article Citation Rates". [[PLOS.
13. "Ditching Impact Factors for Deeper Data". The Scientist.
14. (2016). "Scientific publishing observers and practitioners blast the JIF and call for improved metrics.". Physics Today.
15. (2021-02-16). "Psychology and Wikipedia: Measuring Psychology Journals' Impact by Wikipedia Citations". Social Science Computer Review.
16. "Psychology and Wikipedia: Measuring journals' impact by Wikipedia citations".
17. (2017-03-02). "The impact factor of an open access journal does not contribute to an article's citations". F1000Research.
18. Niyazov, Y., Vogel, C., Price, R., Lund, B., Judd, D., Akil, A., ... & Shron, M. (2016). Open access meets discoverability: Citations to articles posted to Academia. edu. PLOS ONE, 11(2), e0148257.
19. Young, J. S., & Brandes, P. M. (2020). Green and gold open access citation and interdisciplinary advantage: A bibliometric study of two science journals. The Journal of Academic Librarianship, 46(2), 102105.
20. Torres-Salinas, D., Robinson-Garcia, N., & Moed, H. F. (2019). Disentangling Gold Open Access. In Springer Handbook of Science and Technology Indicators (pp. 129–144). Springer, Cham.