Hostname: page-component-cd9895bd7-gbm5v Total loading time: 0 Render date: 2024-12-26T05:20:02.133Z Has data issue: false hasContentIssue false

The Gender Citation Gap in Undergraduate Student Research: Evidence from the Political Science Classroom

Published online by Cambridge University Press:  01 July 2020

Li-Yin Liu
Affiliation:
University of Dayton
Christopher J. Devine
Affiliation:
University of Dayton
Heidi Gauder
Affiliation:
University of Dayton
Rights & Permissions [Opens in a new window]

Abstract

Previous studies documented a “gender citation gap” in political science whereby women are less likely to be cited in published research and course syllabi, especially by male scholars. However, no previous study examined citation patterns among students in political science courses to determine if similar patterns are evident in their research. This article analyzes an original database of individual as well as group research assignments from an undergraduate research methods course. Our analysis indicates that male students are significantly less likely than female students to cite research published by women—whether as first authors, any of the authors, or the average percentage of authors per team. However, in a subsequent group research assignment, gender diversity had no discernible effect on group citation patterns. We discuss the implications of these findings for current pedagogical practices and the future state of the discipline.

Type
Article
Copyright
© American Political Science Association 2020

Previous studies point to a “gender citation gap” in the field of political science, broadly speaking, whereby female authors are disproportionately less likely than male authors to have their work cited. Specifically, when publishing original research, political scientists are less likely to cite studies conducted by women than by men (Dion, Sumner, and Mitchell Reference Dion, Sumner and Mitchell2018; Maliniak, Powers, and Walter Reference Maliniak, Powers and Walter2013; Mitchell, Lange, and Brun Reference Mitchell, Lange and Brun2013; Peterson Reference Peterson2018; see also Esarey and Bryant Reference Esarey and Bryant2018).Footnote 1 Also, political science instructors are less likely to cite women than men in their course syllabi, and they tend to assign textbooks and other readings in which women are underrepresented (Colgan Reference Colgan2017; Diament, Howat, and Lacombe Reference Diament, Howat and Lacombe2018; Phull, Cifliki, and Meibauer Reference Phull, Cifliki and Meibauer2018).Footnote 2

However, these studies examine the gender citation gap only among political science scholars; no previous study has examined whether and to what extent this gap also exists among political science students.Footnote 3 This article presents our analysis, focusing on undergraduate student research. Specifically, we use original data from an introductory political science research methods course to examine the relationship between student gender and gendered citation patterns. Our data are from two stages of an original research project, completed by students during the course of one semester. Students completed the first stage of this research individually and the second stage in groups, thereby enabling us to analyze citation patterns as a function of an individual student’s gender as well as group gender dynamics.

…we use original data from an introductory political science research methods course to examine the relationship between student gender and gendered citation patterns.

It is important to study the gender citation gap among political science students—and not solely among scholars—for three reasons. First, gender bias is unacceptable as a matter of equity, and it undermines students’ educational experience in political science. For example, the gender citation gap may be indicative of broader gender biases that negatively affect a classroom learning environment and, as such, cannot be tolerated or ignored. Also, discounting research conducted by women necessarily limits students’ understanding of political science research as a whole, which in turn should compromise the quality of their performance on research projects and overall educational experience. Thus, as a basic pedagogical matter, it is essential for political science instructors to understand how gender bias in any form—including the gender citation gap—manifests in the classroom and negatively impacts students.

Second, studying the gender citation gap in undergraduate student research would make a valuable contribution to combating implicit gender bias in the political science classroom. Indeed, this study provides evidence that might convince skeptical students (as well as some faculty) that such problems likely exist and should be addressed. Moreover, it could provide the impetus for political science instructors and departments to explore strategies aimed at doing so. For example, instructors might use this or similar studies to engage students in discussions of the gender citation gap before conducting political science research, whether in a traditional classroom setting or for an independent study or honors thesis. Instructors might even design experiments to test which type of intervention is most effective at reducing the gender citation gap among students, such as assigning a reading on the topic versus requiring them to analyze their own citation patterns from previous assignments.

Third, this research provides essential context for previous studies of the gender citation gap. Those studies, cited previously, demonstrate that political scientists generally engage in gendered citation patterns, which—among other things—may skew political science students’ perceptions of what qualifies as the most relevant scholarship in the field. Indeed, scholars’ citation patterns alone may be enough to have an effect. However, students are not blank slates. To the extent that they enter the classroom already inclined to discount women’s research, political scientists’ gendered citation patterns actually may be exacerbating an existing bias among students of which we are not yet aware. If so, then the gender citation gap among scholars may be even more problematic than previous studies indicate. Of course, our study is not designed to test for an interactive effect of students’ and scholars’ gendered citation patterns. However, this work can serve as a necessary first step toward researching such effects in the future.

DATA AND METHODS

The objective of this study is to determine whether—and, if so, to what extent—gender bias influences students’ citation patterns, specifically when citing women. Such biases may be evident in research conducted by individual students, by groups of students, or both. In other words, gender bias may influence how students conduct their own research and/or how they collaborate with others when conducting research. Either phenomenon may limit students’ exposure to female-authored research and perpetuate the gender citation gap.

To evaluate gender bias in student citations at the individual and group levels, respectively, we propose the following hypotheses: (1) male students are less likely than female students to cite research authored by women (hypothesis 1); and (2) groups with fewer female students are less likely to cite research authored by women (hypothesis 2). Both hypotheses are consistent with previous findings regarding gendered citation patterns among scholars (Dion, Sumner, and Mitchell Reference Dion, Sumner and Mitchell2018; Mitchell, Lange, and Brun Reference Mitchell, Lange and Brun2013).

To evaluate gender bias in student citations at the individual and group levels, respectively, we propose the following hypotheses: (1) male students are less likely than female students to cite research authored by women (hypothesis 1); and (2) groups with fewer female students are less likely to cite research authored by women (hypothesis 2).

Data for this analysis are from an original database of research projects completed as part of an undergraduate political science research methods course at a private Midwestern university. Our sample included all 140 students (65 women and 75 men) enrolled in one of seven course sections offered between spring 2017 and spring 2019. Each course section was taught individually by two of this article’s coauthors (one woman and one man).Footnote 4

During the course of one semester, students followed a scaffolded approach to complete an original research project. The first stage was an annotated bibliography that each student completed individually. The final stage was a 10-page research paper consisting of a literature review, research design, and data analysis, all of which were completed in groups of three to four students.Footnote 5 We used two research assignments integral to students’ development of the final project—the annotated bibliography and the literature review—as our data sources. For the annotated bibliography, each student located and summarized four or five peer-reviewed articles directly addressing a research question developed by the group to which he or she was assigned at the beginning of the semester. After completing the annotated bibliography, students then worked with their group members to write a literature review, citing 12 scholarly articles, that later would be incorporated into their final research paper. Because most groups included four students, and most students cited at least four articles in their annotated bibliographies, each group decided which articles to eliminate or retain for the literature review. They also could add new articles, which many groups did in response to the instructor’s feedback.Footnote 6 In total, the (individual) annotated bibliographies included 624 citations and the (group) literature reviews included 483 citations.

To test our hypotheses, we began by coding each author’s gender—or, at least, students’ likely perception thereof. Two graduate students (one woman and one man) coded each author by name, exactly as it appeared in a student’s or a group’s citation, as (1) certainly a man/woman (90% to 100% confidence); (2) probably a man/woman (60% to 90% confidence); or (3) indeterminable (less than 60% confidence).Footnote 7 For this analysis, we combined the certain and probable gender identifications so that each author was coded as a woman, a man, or indeterminable.Footnote 8 Next, we generated three measures of gendered citation patternsFootnote 9: (1) percentage of articles cited by a student or group in which the first author was a woman; (2) percentage of articles in which any author was a woman; and (3) average percentage of authors who were women, per article.Footnote 10

RESULTS

Figure 1 presents an initial indication of gender’s influence on student citation patterns. The figure lists the percentage of articles that male-versus-female students cited in their annotated bibliographies in which the first author or any author was a woman, as well as the average percentage of female authors per article. Figure 1 also includes 95% confidence intervals to indicate whether the percentages differed significantly based on a student’s gender. All three measures exhibit these differences. Specifically, there is a 14-percentage-point gender gap regarding first authorship; a 16-percentage-point gap regarding any inclusion of women; and an 11-percentage-point gap regarding the average share of female authors per article. In each case, male students were less likely than female students to cite research conducted by women.

Figure 1 Citation of Women’s Research in Male-versus-Female Students’ Annotated Bibliographies

However, there are good reasons to believe that other factors might confound—or at least weaken—the bivariate relationships in figure 1. First, some students’ research interests pertain to political science subfields in which women are more active and publish more frequently than other subfields. If female students, in particular, gravitate toward studying topics within these subfields, then that may explain the gender disparities shown in figure 1. Second, the college experience should increase students’ awareness of gender and other inequities such that more advanced students may be more sensitive and responsive to concerns about diversity and inclusion. Therefore, we might expect a fourth-year student’s citation pattern to be more gender-inclusive than that of a second-year student.

To account for potential confounders, we estimated three linear regression models. The dependent variables are the same as in figure 1. The independent variable is student gender (0=male, 1=female). Also, we included two control variables. First, to account for women’s subfield presence, we identified the APSA organized section most closely related to a student’s research topic. Then we recorded the percentage of women belonging to that organized section as of early 2019.Footnote 11 For example, a student whose research most directly pertains to “Human Rights” (Section 36) would be coded as 54.8% on the subfield variable because this is the percentage of section members identified as women. Second, we controlled for the student’s class year (1=first year through 4=fourth year). Table 1 presents the results from our regression models.

The student-gender variable is statistically significant at p<0.05 in each model, even after controlling for subfield and class year.Footnote 12 However, there are differences by student gender smaller than in figure 1. That is, the citation gap between female-versus-male students is 8.9 percentage points for first authorship; 11.5 percentage points for including any women; and 6.5 percentage points for the average share of female authors per article. Again, male students were less likely than female students to cite research conducted by women. This evidence clearly supports hypothesis 1.Footnote 13

Table 1 Citation of Women in Students’ Annotated Bibliographies

Notes: Entries are linear-regression coefficients. Robust standard errors are in parentheses. All observations are clustered by research group.

***p<0.001; **p<0.01; *p<0.05; ^p<0.10. The dependent variables measure gender diversity in student citation patterns. First Author (Model 1) represents the percentage of citations with a woman as first author. Any Women (Model 2) represents the percentage of citations including any woman as an author. Average % (Model 3) represents the average percentage of authors who were women, per article.

However, does gender bias likewise affect how students conduct research in collaboration with other students? In other words, does gender diversity influence group citation patterns with respect to gender, which seems to be the case for political scientists (Dion, Sumner, and Mitchell Reference Dion, Sumner and Mitchell2018)? To answer this question, first we needed to modify our data so as to treat the group’s (i.e., literature review) rather than the student’s (i.e., annotated bibliography) submission as our unit of analysis. Our independent variable measured the percentage of women within a group (i.e., 0=no women to 100=all women) rather than an individual student’s gender.Footnote 14 Also, we measured class year as the average for all group members. The subfield variable required no modification because its original measurement was based on the group’s research topic.

Finally, we calculated the dependent variables according to the difference between the percentage of women cited in the group’s literature review and their percentage cited in all of the individual group members’ annotated bibliographies combined. For example, suppose that a group of four students submitted a literature review in which three of 12 articles (25.0%) had any female authors, whereas those students’ annotated bibliographies, combined, included six of 16 articles (37.5%) with any female authors. This group would be coded as -12.5% on the “any” dependent variable (37.5% to 25.0%).Footnote 15 In other words, the group’s literature review was 12.5 percentage points less diverse, with respect to gender, than the original pool of citations.

To evaluate the relationship between the gender diversity of a student group and its citation patterns, we estimated three linear regression models (table 2). In each case, we found no evidence of any such relationship. That is, the percentage of women within a group had no discernible effect on how many women that group cited in its literature review, relative to the pool of citations with which it started.Footnote 16 This evidence does not support hypothesis 2. The gender bias that we observed so clearly in individual students’ research apparently does not extend to the collaborative research setting.Footnote 17

Table 2 Citation of Women in Group Literature Reviews

Notes: Entries are linear-regression coefficients. Standard errors are in parentheses. ***p<0.001; **p<0.01; *p<0.05; ^p<0.10. The dependent variables measure gender diversity in group citation patterns, relative to gender diversity in the group members’ individual citation patterns. Specifically, in each case, we subtracted the relevant measure as applied to all group members’ annotated bibliographies, combined, from that measure as applied to the group’s literature review. First Author (Model 1) represents the percentage of citations with a woman as first author. Any Women (Model 2) represents the percentage of citations including any woman as an author. Average % (Model 3) represents the average percentage of authors who were women, per article.

CONCLUSION

Overall, our results indicate that gender bias does influence undergraduate students’ citation patterns—but only at the individual level. Male students were less likely than female students to cite research conducted by women. However, within groups, we found no evidence that gender dynamics influenced the collaborative process such that male-dominated groups were more likely to exclude research conducted by women. The former result is consistent with previous evidence regarding political scientists’ citation patterns (Mitchell, Lange, and Brun Reference Mitchell, Lange and Brun2013) whereas the latter is not (Dion, Sumner, and Mitchell Reference Dion, Sumner and Mitchell2018).

Overall, our results indicate that gender bias does influence undergraduate students’ citation patterns—but only at the individual level.

We can only speculate as to why our evidence does not support hypothesis 2. One possibility is that whatever their private inclinations may be, students genuinely strive to include all members and their perspective when participating in group work—and they succeeded in doing so. Another possibility is that students simply were more intentional and more attentive to various details about a given article, including author gender, when conducting research individually compared to working in a group setting in which citation decisions might be deferred to other students. The latter explanation is plausible and could be tested easily using data from an alternative research assignment—for instance, one that requires the group to work together to locate articles for its literature review rather than drawing on a pool of citations from individuals’ annotated bibliographies. Future research would be valuable in exploring these possible explanations and in replicating our results across a diverse range of assignments and disciplines. Moreover, we recognize that this study is limited in terms of generalizability because our data are from seven sections of one course taught at one private Midwestern university. Additional studies replicating or extending our research design in other courses and institutions would strengthen our understanding of undergraduate students’ citation patterns and further address the gender citation gap in political science.

To the extent that women, in fact, are underrepresented in undergraduate student research, the question becomes: What do we, as a discipline, do about this? One practical response is for political scientists to evaluate whether they are setting a good example for their students by measuring the citation gap in their own research and syllabi (Sumner Reference Sumner2018) and making appropriate modifications. Another practical response is for instructors to explicitly address the gender citation gap with their students and encourage them to consider how gender biases might influence their work. Indeed, these are the types of interventions recommended to address the gender citation gap only among political scientists (Brown and Samuels Reference Brown and Samuels2019; Peterson Reference Peterson2018). If similar biases are evident among students, surely they would deserve our attention as well.

Supplementary Materials

To view supplementary material for this article, please visit http://dx.doi.org/10.1017/S1049096520000426.

Footnotes

1. However, these patterns seem to depend on author gender; women are more likely than men to cite other women’s research whether authored by one women or an all-woman team. In that case, “most of the citation gap is driven by men and mixed author teams underciting the work of women rather than women overciting work by other female scholars” (Dion, Sumner, and Mitchell Reference Dion, Sumner and Mitchell2018, 324). Women also are significantly less likely than men to cite their own work (Maliniak, Powers, and Walter Reference Maliniak, Powers and Walter2013).

2. Again, these patterns seem to vary by gender, with female instructors citing more female-authored research in their syllabi and senior male instructors assigning fewer articles by female authors (Colgan Reference Colgan2017; Phull, Cifliki, and Meibauer Reference Phull, Cifliki and Meibauer2018).

3. In fact, the only previous studies of political science students’ citation patterns focus on understanding how they find and evaluate sources (Datig Reference Datig2016; Hendley Reference Hendley2012).

4. Among the students taught by the female professor (i.e., one section, 21 students), 47.6 % were female; among the students taught by the male professor (i.e., six sections, 119 students), 46.2 % were female.

5. There were 38 groups, in total, with an average of 3.7 students per group.

6. Instructor feedback was based on the quality of students’ assignments and the applicability of the article selection to the research topic; no gender-citation–related information was given. Moreover, neither instructor incorporated any intervention mechanisms for reducing the gender citation gap in their classroom.

7. The latter category includes authors with gender-ambiguous names (e.g., Jamie); names that would be unfamiliar to most Americans (e.g., Xuesong); and those who were identified by only their initials (e.g., P.K.).

8. Intercoder agreement was very high (91.3%). In cases of disagreement, we made a final determination about students’ likely perception of the author’s gender.

9. We used multiple dependent variables, in part, because previous studies of the gender citation gap yield different results depending on the number of authors included in the analysis (Dion, Sumner, and Mitchell Reference Dion, Sumner and Mitchell2018; Esarey and Bryant Reference Esarey and Bryant2018).

10. For example, suppose that a student cited four articles with the following number of female authors: 0/1, 0/2, 1/2, 3/3. The “average” percentage of female authors would be 37.5% ((0% + 0% + 50% + 100%)/4) and the percentage of “any” female authors would be 50% (two of four articles).

12. The subfield variable is significant at p<0.05 in two models and at p<0.10 in one model, with the correct sign in each case. The class-year variable is significant only at p<0.10 in one model but incorrectly signed.

13. Appendix table A1 presents the same models with the professor’s gender added as a control variable. In each model, this variable is not statistically significant and the student-gender variable remains significant. Also, the R-squared statistic increases slightly in Model 2.

14. The median group was 50% female. Seven groups included no women (18%) and five included only women (13%).

15. A more obvious measurement strategy would be to calculate the percentage of women authors in the group’s literature review without reference to the annotated bibliographies. However, this would treat the former as if it were independent of the latter, which is wrong. In fact, most if not all of a group’s literature review citations came from its members’ annotated bibliographies. Given this interdependence, it is more appropriate to evaluate the group’s citation patterns relative to the pool of citations with which it started.

16. One explanation for these null findings may be that our sample size (N=38) is too small to detect the hypothesized relationship. However, the subfield variable attains conventional significance levels in one model. Also, if we exclude the subfield variable from the average-difference model, the group-gender variable becomes statistically significant at p<0.05 (but incorrectly signed). In that case, it seems unlikely that our null findings can be attributed to a lack of statistical power.

17. Appendix table A2 presents the same models with the professor’s gender added as a control variable. In each model, this variable is not statistically significant and the group-gender variable is not significant. Also, the adjusted R-squared statistic increases slightly in Models 1 and 2.

References

REFERENCES

Brown, Nadia E., and Samuels, David. 2019. “Beyond the Gender Citation Gap: Comments on Dion, Sumner, and Mitchell.” Political Analysis 26 (3): 328–30.CrossRefGoogle Scholar
Colgan, Jeff. 2017. “Gender Bias in International Relations Graduate Education? New Evidence from Syllabi.” PS: Political Science & Politics 50 (2): 456–60.Google Scholar
Datig, Ilka. 2016. “Citation Behavior of Advanced Undergraduate Students in the Social Sciences: A Mixed-Method Approach.” Behavioral & Social Sciences Librarian 35 (2): 6480.CrossRefGoogle Scholar
Diament, Sean M., Howat, Adam J., and Lacombe, Matthew J.. 2018. “Gender Representation in the American Politics Canon: An Analysis of Core Graduate Syllabi.” PS: Political Science & Politics 51 (3): 635–40.Google Scholar
Dion, Michelle L., Sumner, Jane L., and Mitchell, Sara M.. 2018. “Gendered Citation Patterns across Political Science and Social Science Methodology Fields.” Political Analysis 26 (3): 312–27.CrossRefGoogle Scholar
Esarey, Justin, and Bryant, Kristin. 2018. “Are Papers Written by Women Authors Cited Less Frequently?Political Analysis 26 (3): 331–34.CrossRefGoogle Scholar
Hendley, Michelle. 2012. “Citation Behavior of Undergraduate Students: A Study of History, Political Science, and Sociology Papers.” Behavioral & Social Sciences Librarian 31 (2): 96111.CrossRefGoogle Scholar
Maliniak, Daniel, Powers, Ryan, and Walter, Barbara F.. 2013. “The Gender Citation Gap in International Relations.” International Organization 67 (4): 889922.CrossRefGoogle Scholar
Mitchell, Sara M., Lange, Samantha, and Brun, Holly. 2013. “Gendered Citation Patterns in International Relations Journals.” International Studies Perspectives 14 (4): 485–92.CrossRefGoogle Scholar
Peterson, David A. M. 2018. “Historical Disparities and Gendered Citation Patterns.” Political Analysis 26 (3): 338–44.CrossRefGoogle Scholar
Phull, Kiran, Cifliki, Gokhan, and Meibauer, Gustav. 2018. “Gender and Bias in the International Relations Curriculum: Insights from Reading Lists.” European Journal of International Relations 25 (2): 383407.CrossRefGoogle Scholar
Sumner, Jane L. 2018. “The Gender Balance Assessment Tool (GBAT): A Web-Based Tool for Estimating Gender Balance in Syllabi and Bibliographies.” PS: Political Science & Politics 51 (2): 396400.Google Scholar
Figure 0

Figure 1 Citation of Women’s Research in Male-versus-Female Students’ Annotated Bibliographies

Figure 1

Table 1 Citation of Women in Students’ Annotated Bibliographies

Figure 2

Table 2 Citation of Women in Group Literature Reviews

Supplementary material: PDF

Liu et al. supplementary material

Appendix

Download Liu et al. supplementary material(PDF)
PDF 69.7 KB