Another extraordinary year for citation impact

Responding to this year's Journal Citation Reports, Nandita Quaderi explains how Covid-19 continues to affect the citation network, and introduces a new kind of citation distortion

Covid-19 changed our world in thousands of ways. The novel coronavirus caused millions of deaths, strained health systems worldwide, grounded flights and enforced isolation on communities used to living in a global and connected world. More than two years on from the first cases, we are starting to understand some of the silver linings associated with the pandemic -from unprecedented scientific collaboration to the accelerated speed of innovation and discovery for therapies and vaccines. For the second year, the Journal Citation Reports (JCR) from Clarivate demonstrates the enormous impact the Covid-19 pandemic has had on scholarly publishing. 

The annual JCR release enables the research community to evaluate the world’s high-quality academic journals using a range of indicators, descriptive data and visualizations. This year’s JCR contains more than 21,000 journals, across 254 research categories from 114 countries. It is based on 2021 data compiled from the Web of Science Core Collection, the leading collection of quality journals, books and conference proceedings in the world's largest publisher-neutral global citation database. The data include35,000 publications (journals, books and conference proceedings), 2.7 million citable items and 145 million cited references.

Through this carefully selected and curated data we can fully appreciate and understand the enormous impact of the academic community’s rapid response to the Covid-19 pandemic. Last year’s release was the first to show the impact of Covid-19 research with a rise in Journal Impact Factors (JIF) across the board. This year, we continue to observe notable increases in citation impact for journals in the fields of general medicine, critical care, public health, infectious diseases, immunology and basic biomedical sciences. 

Exceptional performance

Perhaps the most surprising increase is that of The Lancet.  The general medical journal increased its JIF to 202.731, which moves it to the #1 position in the general & internal medicine category, overtaking the New England Journal of Medicine. Industry watchers will know that the New England Journal of Medicine has been the top-ranked title in that category since the first release of the JCR 45 years ago. Of the 10 articles with the highest citation count in 2021, three appeared in The Lancet. They are all Covid-19 articles and  describe the clinical features of patients in Wuhan infected with the virus; examine risk factors for mortality; and examine the epidemiological and clinical characteristics of cases in Wuhan. 

This year Nature earns the distinction of being the first ever journal to accumulate more than one million total citations in one year. Nature published16 items with over 500 JIF citations – of which 12 items were COVID-19 related.  To put this in context, the first Journal Citation Reports in 1975 summarized the 1974 data from the Science Citation Index and included a total of four million citations in the whole data extraction (compared to 143 million citations this year).

Seven journals had JIFs of more than 100 for the first time, all of which published high quantities of Covid-19 related research. These are the Journal of the American Medical Association, The Lancet, Lancet Respiratory Medicine, Nature Reviews Drug Discovery, Nature Reviews Immunology, Nature Reviews Molecular Cell Biology and the New England Journal of Medicine.   

Upholding research integrity and a new type of citation distortion  

While it is important to acknowledge and celebrate the journals and research communities that have made exceptional impacts on their fields over the last two years, upholding the importance of research integrity and sound citation practice is undoubtedly more important to us. Every year, our editorial integrity team analyses the data to ensure that the metrics we provide accurately reflect the contribution of those journals to the literature.

Every year we identify journals where the citation data show anomalies when compared to other journals in the same category. We suppress the JIF of journals that demonstrate anomalous citation behaviour including where there is evidence of excessive journal self-citation and/or citation stacking (which involves two or more journals). 

We do not presume a motive or accuse these journals of wrongdoing – sometimes it may be accidental, sometimes it may be malicious, and it is not always black and white. Suppressed journals remain in the Web of Science Core Collection – although they may be subject to re-evaluation and will be removed from coverage if they fail – and will be eligible for inclusion in the Journal Citation Reports again the following year.

We have found a new trend in citation distortion, which we have defined as ‘self-stacking’. This is where the journal contains one ormore documents with citations that are highly concentrated to the JIF numerator of the title itself, for example a review or retrospective which predominantly includes citations that would contribute to the journal’s JIF.  This is the first year we have formally defined the criteria for self-stacking suppression, and as such we have made the decision to issue a warning to six journals rather than suppress the journal’s JIF. Going forward, continued journal self-stacking will result in suppression of JIF.    

 Scholarly publishing continues to be based on trust but unfortunately new forms of manipulation have emerged as some stakeholders seek to gain an unfair advantage and fraudulent enterprises have appeared that exploit the increased pressure to publish and be cited. One of the factors contributing to the growth of citation manipulation is the over-emphasis on bibliometric indicators during research assessment and the misuse of journal-level metrics, such as the JIF, to directly evaluate researchers.

It is wonderful to see high impact journals get the attention they deserve from the international research community, but seeking to secure high citation impact without a solid foundation should not be a goal in itself for researchers and can damage the integrity of the scholarly record. This is not a phenomenon that can be stopped solely by metrics providers, but needs cooperation and active participation from publishers, institutions, funders and governments too. We continuously review content with the goal of developing additional screening for distortions of the JIF, and will continue to do so, considering every option to promote best practice for research, citation behavior, and research evaluation. 

Nandita Quaderi is editor-in-chief and vice president of Web of Science Editorial