Does research evaluation in the sciences have a gender problem?
How do we measure and evaluate productivity in scientific research? Counting papers published per year isn’t useful. Research has shown that female researchers tend to publish less[3] than their male counterparts in several scientific fields, especially early in their careers[4].
Measuring collaboration is equally tricky. Women tend to collaborate less than their male counterparts[5] (especially less often internationally)[6] and have different collaboration strategies than their male counterparts [7].
Then there’s bibliometrics. Citation counts aren’t sexist, but citation practices can be. Many studies have found[8] that, no matter the authorship position[9] of a female researcher, she is less likely to be cited than her male counterparts.
Might altmetrics[10] be better suited to help understand the influence of research in a more gender-balanced way? Altmetrics are data from the social web that help us understand how research is discussed, shared, reviewed, rated, a nd reused by other researchers and the public.
The jury is still out on whether altmetrics show a gender advantage for male researchers over female researchers. In fact, in some fields and for certain types of altmetrics, women actually have an advantage over their male counterparts when it comes to altmetrics for their work. Bar-Ilan and van der Weijden[11] (2015)found that for papers published prior to March 2014, female astronomers and astrophysics researchers have slightly higher Mendeley readership numbers on average, but that men are better represented on the academic social bookmarking site overall.
Haustein, Paul-Hus, Sugimoto and Larivière[12] (2016) looked at articles from a larger cross-section of disciplines and found that a gender gap exists for social media altmetrics for publications from 2013, but to less of an extent than for citations. Some disciplines were found to be mostly gender-balanced (mathematics, arts, humanities, health, psychology), while others showed that both men and women lead authors had dominance based upon the social media platform studied (biology, biomedical research, earth and space sciences, engineering and technology, professional fields, and social sciences).
Interestingly, a majority of open access journals studied by Sugimoto and Larivier̀e[13] (2017) in chemistry and the interdisciplinary sciences showed an advantage for female lead authors, especially those publishing in first author positions. The authors suggest that the results show “that these venues provide particular visibility for younger female academics on social media”.
More recently, I looked at the data for how male and female lead authors compare when one examines the overall attention that their work receives online (which can be approximated by the Altmetric Attention Score), and more specifically in the media and public policy documents.
Across all papers published in 2016 from Web of Science where lead author gender was known (N = 1,849,326), male lead-authored papers were more prevalent than female lead-authored papers, by a ratio of 2.5 to 1. But we already knew that men lead author papers more often than women, didn’t we?
Looking more closely at a subset of articles published between January and March 2016, 85,277 had received attention in the sources that Altmetric tracks. Male lead-authored papers with any Altmetric data (n = 52,821) were represented 1.6 to 1 over female lead-authored papers with any Altmetric data (n = 32,456).
This gender discrepancy differs from what Haustein et al (2016) found – 29 per cent of female lead authored papers in their sample from 2013 had any Altmetric attention, compared to 19 per cent of male lead authored papers. Taken together, one finds a hint that gender discrepancies might reverse over time: my findings suggest that men’s work may be talked about more online over the first year or so after publication, but over time, Haustein et al’s data indicate that female lead authored papers eventually get more than their due.
The gender discrepancies seem to stop there. There was no difference in overall attention being paid to research online, as measured by the median Altmetric Attention Scores for the papers I looked at (a median Score of 2 was found for both male and female lead-authored papers).
Of the male lead-authored papers published in Q1 2016 with Altmetric attention, 7,165 (13.6 per cent) had at least one mention in the news, and 492 (0.9 per cent) had at least one citation in a public policy document. Female lead-authored papers had roughly similar rates of attention in the news and in public policy documents: 4,288 (13.2 per cent) had at least one news mention and 347 (one per cent) had at least one public policy citation. The median number of mentions in news articles and public policy documents showed no gender difference (a median of 1 was found for attention in both sources, for both male and female lead-authored articles).
The relative gender balance of altmetrics should be heartening to all researchers. It means that a conscious engagement and impact strategy for one’s research, carefully applied, will not necessarily be hindered by implicit bias in the same way that citations can sometimes be.
For those researchers in the fields of biology, biomedical research, earth and space sciences, engineering and technology, professional fields, and social sciences – disciplines where gender biases have unfortunately been shown to exist – carefully planned outreach strategies may help balance the playing field.
For more information on developing outreach and impact strategies for your research, check out these resources:
l The 30 Day Impact Challenge[14] (e-book, 2015): a primer on using social media and other online outreach strategies to raise your professional, scholarly profile on the Web
l The Research Impact Handbook[15] (e-book, 2016): a guide to developing strategies for long-term impact for your research in both the scholarly and public spheres.
[1] Library Pipeline www.librarypipeline.org/
[2] Results of the FORCE 2016 Innovation Challenge http://stacykonkiel.org/force2016-innovation-challenge/
[3] Gender, Children and Research Productivity https://www.researchgate.net/publication/227090293_Gender_Children_and_Research_Productivity
[4] Measures of Sex Differences in Scientific Productivity https://academic.oup.com/sf/article-abstract/71/1/159/2232637/Measures-of-Sex-Differences-in-Scientific
[5] Child Care. Research Collaboration, and Gender Differences in Scientific Productivity ftp://ftp.ige.unicamp.br/pub/CT001%20SocCiencia/25%20de%20Outubro/Kyvik%201996.pdf
[6] Bibliometrics: Global gender disparities in science http://www.nature.com/news/bibliometrics-global-gender-disparities-in-science-1.14321#/b7
[7] How do men and women differ in research collaborations? An analysis of the collaborative motives and strategies of academic researchers www.sciencedirect.com/science/article/pii/S0048733311001296
[8] Gender Differences in Publication Output: Towards an Unbiased Metric of Research Performance http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0000127
[9] Bibliometrics: Global gender disparities in science www.nature.com/news/bibliometrics-global-gender-disparities-in-science-1.14321
[10] altmetrics: a manifesto http://altmetrics.org/manifesto/
[11] Altmetric Gender Bias – An Exploratory Study www.orb-academic.org/index.php/journal-of-computer-science/article/view/122
[12] Is there a gender gap in social media metrics? www.researchgate.net/profile/Adele_Paul-Hus/publication/280080757_Is_there_a_gender_gap_in_social_media_metrics/links/55a6d21b08ae410caa7507a2/Is-there-a-gender-gap-in-social-media-metrics.pdf
[13] Altmetrics: Broadening Impact or Amplifying
Voices? http://pubs.acs.org/doi/pdf/10.1021/acscentsci.7b00249
[14] The 30-Day Impact Challenge http://blog.impactstory.org/research-impact-challenge-ebook/
[15] The Research Impact Handbook www.fasttrackimpact.com/research-impact-book
Stacy Konkiel is the director of research and education at Altmetric. She also currently chairs the Innovation committee of Library Pipeline [1] and is building the Metrics Toolkit [2].
This article forms part of a wider report issued by technology company Digital Science to coincide with Ada Lovelace Day, the annual celebration that promotes women working in Science, Technology, Engineering, and Maths (STEM). Championing The Success of Women in Science, Technology, Engineering, Maths, and Medicine includes a collection of think pieces around current issues faced by women in STEM authored by industry leaders.
